Understanding the English Advantage in Large Language Models

Large Language Models (LLMs) such as GPT-4, Claude, Gemini, and Llama are often described as multilingual models capable of understanding and generating text in dozens of languages. However, multilingual does not imply equal performance across all languages. In practice, these models consistently demonstrate superior performance in English compared to many other languages.

This performance gap is not accidental. It is largely a consequence of how foundation models are trained, the availability of multilingual data on the internet, and the efficiency of language tokenization. Understanding these factors helps explain why prompts written in English often produce more accurate, consistent, and detailed responses than equivalent prompts in lower-resource languages.

The Internet Speaks English

The performance of modern LLMs is fundamentally determined by their training data. Most general-purpose models are pretrained on enormous collections of publicly available internet content, books, research papers, and source code. One of the most influential datasets used during pretraining is Common Crawl, a large-scale archive of publicly accessible web pages.

According to Lai et al. (2023), English accounts for 45.88% of the Common Crawl dataset, making it by far the most represented language. In comparison, Russian—the second most common language, accounts for only 5.97%, while Chinese represents approximately 4.87%. Even widely spoken languages such as Portuguese and Vietnamese each contribute only around 1% of the dataset.

This imbalance means that an LLM encounters English examples far more frequently during training than any other language. Consequently, the model develops richer statistical representations, stronger linguistic patterns, and broader domain knowledge in English.

In other words, LLMs are not inherently designed to prefer English; they simply learn most effectively from the language they encounter most often.

Under-Represented Languages Face a Data Scarcity Problem

The imbalance becomes even more pronounced for many languages spoken by hundreds of millions of people.

Punjabi, for example, is spoken by more than 113 million people worldwide but represents only 0.0061% of Common Crawl. Similarly, Urdu has over 230 million speakers but contributes merely 0.0274% of the dataset. Languages such as Telugu, Marathi, Gujarati, Kannada, and Swahili are also significantly under-represented relative to their global populations.

Researchers often refer to these as low-resource languages, meaning that comparatively little high-quality digital text is available for model training.

The implication is straightforward: if a model rarely encounters a language during training, it has fewer opportunities to learn its vocabulary, grammar, idiomatic expressions, cultural context, and reasoning patterns. As a result, performance naturally declines.

Benchmark Evaluations Confirm the Performance Gap

The effect of training data imbalance is clearly reflected in benchmark evaluations.

OpenAI reported that GPT-4 achieved substantially higher scores on the Massive Multitask Language Understanding (MMLU) benchmark when questions were presented in English than when identical questions were translated into lower-resource languages such as Telugu.

Similar findings were reported by Yennie Jun, who evaluated GPT-4 on mathematical reasoning tasks using Project Euler problems. GPT-4 solved English problems more than three times as often as equivalent problems written in Armenian or Farsi, while failing all tested questions in Burmese and Amharic.

These results suggest that multilingual capability is not uniformly distributed across languages. Instead, language performance closely mirrors the distribution of training data available during pretraining.

More Data Is Not the Only Explanation

Although training data plays a central role, it is not the sole factor influencing multilingual performance.

Languages differ significantly in grammatical structure, writing systems, morphology, and cultural conventions. Some languages encode social relationships directly within pronouns or verb forms, while others rely heavily on contextual information.

Vietnamese provides an illustrative example. Rather than using a single equivalent of the English pronouns I and you, Vietnamese employs numerous pronouns that express age, family relationships, and social hierarchy. When translated into English, much of this relational information is inevitably lost.

Consequently, even perfect translation cannot always preserve the full semantic richness of the original language.

Translation Is Helpful but Imperfect

Because English generally yields stronger model performance, many users adopt a simple workflow:

1. Translate the prompt into English.
2. Ask the LLM in English.
3. Translate the response back into the original language.

While this strategy often improves answer quality, it introduces several limitations.

First, translation itself requires a model capable of accurately understanding lower-resource languages. If the translation contains subtle errors, the downstream response may already be compromised.

Second, translation inevitably loses linguistic and cultural nuances that have no direct equivalent in English. Idioms, honorific systems, implicit relationships, and culturally specific expressions may disappear during translation, reducing the fidelity of the interaction.

Therefore, translation should be viewed as a practical workaround rather than a complete solution.

Language Also Affects Cost and Latency

An often-overlooked consequence of multilingual processing is computational efficiency.

LLMs process text as tokens rather than words. API pricing and inference latency are generally proportional to the number of tokens processed.

Interestingly, expressing the same meaning requires vastly different numbers of tokens across languages.

Using the MASSIVE multilingual dataset, Yennie Jun found that the median prompt length was approximately 7 tokens in English, compared to 32 tokens in Hindi and 72 tokens in Burmese.

Assuming similar generation speeds, processing Burmese text may require roughly ten times more computation than equivalent English text. For token-based APIs, this also translates directly into higher inference costs.

This difference highlights that language choice can affect not only model quality but also operational efficiency.

The Rise of Specialized Multilingual Models

Recognizing these limitations, researchers have begun developing language-specific foundation models rather than relying exclusively on English-centric LLMs.

Examples include ChatGLM and Llama-Chinese for Chinese, CroissantLLM for French, PhoGPT for Vietnamese, and Jais for Arabic. These models allocate substantially more training data to their target languages, enabling stronger performance within specific linguistic communities.

Rather than attempting to optimize equally for every language, these specialized models prioritize depth over breadth.

Conclusion

The superior performance of English in modern Large Language Models is primarily a consequence of training data distribution. English dominates publicly available internet content, giving foundation models significantly more opportunities to learn its vocabulary, syntax, and knowledge than most other languages.

However, training data is only one piece of the puzzle. Linguistic complexity, cultural context, tokenization efficiency, and translation fidelity all contribute to the multilingual performance gap observed in today's LLMs.

As multilingual datasets continue to grow and language-specific foundation models become more common, this gap is likely to narrow. Nevertheless, understanding these underlying factors enables AI practitioners to design better prompting strategies, make informed deployment decisions, and develop more inclusive AI systems for global users.