# How LLMs Shrink Global Linguistic Diversity

> Published 2026-08-30 · https://www.promptzone.com/noor_eriksson/how-llms-shrink-global-linguistic-diversity-ink

A Nature Human Behaviour paper documents measurable contraction in linguistic variety as LLMs scale. The study, first flagged on Hacker News with 16 points and 3 comments, analyzed output distributions across 20+ languages and found consistent homogenization toward high-resource English patterns.

## What the Data Show

Researchers measured lexical diversity, syntactic structures, and morphological features in model generations versus human baselines. Outputs from models trained on web-scale corpora exhibited 18-34% lower type-token ratios in non-English languages compared with pre-2020 human text.

The effect appears strongest in mid-resource languages such as Korean, Turkish, and Swahili. Low-resource languages showed even steeper drops when models were prompted in English first.

## How LLMs Produce the Effect

Training data skews heavily toward English web text. During alignment and instruction tuning, English preference signals further reinforce dominant patterns. Decoding strategies that optimize for fluency then suppress rarer constructions that appear less often in the training distribution.

The paper quantifies this through controlled generation experiments: identical prompts in different languages produced outputs whose n-gram overlap with English increased by an average of 22% after 2022 model releases.

## Benchmarks and Measurements

| Language | Pre-LLM Diversity Score | 2023 LLM Score | Drop |
|----------|-------------------------|----------------|------|
| Korean   | 0.71                    | 0.47           | 34%  |
| Turkish  | 0.68                    | 0.49           | 28%  |
| Swahili  | 0.64                    | 0.42           | 34%  |
| Spanish  | 0.75                    | 0.61           | 19%  |

Scores reflect normalized type-token ratio on 10,000-sentence samples. The study used the same evaluation pipeline across years to isolate model-driven change.

## Practical Steps for Teams

- Audit training mixtures for language balance before scaling.
- Add explicit diversity penalties during decoding when generating non-English text.
- Maintain parallel human-written corpora for languages showing the largest measured drops.
- Test prompts in the target language rather than translating from English.

These steps require only modest additional compute and can be inserted into existing fine-tuning pipelines.

## Who Should Pay Attention

Teams building multilingual products or localization tools face direct output quality risks. Researchers studying cultural representation in AI should incorporate the reported metrics into evaluation suites. Developers working exclusively on English-only applications can deprioritize this finding for now.

## Comparison with Earlier Trends

Pre-LLM machine translation systems already narrowed some language pairs, yet the contraction remained smaller and more localized. Current LLM scaling amplifies the effect across all generation tasks, not only translation. No other publicly documented study has tracked the same languages with identical metrics across the 2020-2024 window.

> **Bottom line:** The measured homogenization is real, reproducible, and tied directly to current training and decoding practices.

The trend is likely to continue unless deliberate countermeasures are adopted at the data and decoding stages.