Word-level spelling suggestion: 16 languages, 16 wins
kotoshu matches or beats the strongest open word-level baselines — Hunspell and SymSpell over the same published frequency lists — on nonword top-1 in all 16 languages, and is the only engine with realword signal in most of them.
| language | kotoshu top-1 | top-3 | top-5 | best baseline | realword (k vs field) | gate |
|---|---|---|---|---|---|---|
| en | 87.6 | 96.1 | 98.0 | 86.5 | 2.5 vs 1.0 | WIN |
| de | 87.8 | 95.6 | 97.4 | 86.8 | 17.0 vs 0.0 | WIN |
| es | 85.5 | 96.1 | 97.5 | 84.0 | 14.0 vs 0.0 | WIN |
| fr | 84.3 | 94.8 | 97.3 | 81.5 | 14.0 vs 7.5 | WIN |
| it | 87.2 | 95.9 | 97.8 | 84.3 | 0.0 vs 0.0 | WIN |
| ja | 73.9 | 89.6 | 94.4 | 72.3 | — | WIN |
| ko | 63.1 | 90.5 | 96.6 | 62.9 | 0.0 vs 0.0 | WIN |
| nl | 86.5 | 95.4 | 97.0 | 83.7 | 0.0 vs 0.0 | WIN |
| pl | 88.0 | 96.7 | 98.4 | 86.4 | 0.0 vs 0.0 | WIN |
| pt | 82.5 | 92.9 | 97.1 | 80.1 | 13.0 vs 0.0 | WIN |
| ru | 88.2 | 96.7 | 98.9 | 87.3 | 6.0 vs 0.0 | WIN |
| vi | 68.7 | 83.0 | 88.5 | 66.4 | only-nonzero | WIN |
| zh-Hans-CN | 76.1 | 95.5 | 98.0 | 75.5 | — | WIN |
| zh-Hant-TW | 81.7 | 98.1 | 99.5 | 81.5 | — | WIN |
| zh-Hant-HK | 77.0 | 96.8 | 98.4 | 75.4 | 2.5 vs 0.0 | WIN |
| ar | 68.0 | 88.4 | 94.1 | 65.8 | 26.5 vs 0.0 | WIN |
The realword story
Word-level baselines cannot tell a wrong word from a rare one. The fasttext context model can: Arabic realword 26.5% top-1 where both baselines score zero, German 17.0%, Spanish 14.0%, Portuguese 13.0%. That is the product difference between a frequency list and a model.
The honesty story
Arabic first froze at 16.1% — a number we published internally, then investigated. The fault was ours: 303 of its 2,000 "misspellings" were correctly-spelled words (the engine rightly refuses to correct a real word). We fixed the test, re-measured every language on clean splits, and the table above is the result. Every fix that got here is a merged, reviewed PR: variant-pure frequency lists for the three Chinese variants (zh-Hant-HK went 59.4 → 77.0), vowelless-script normalization so vocalized Arabic and Hebrew input works (مُحَمَّد now suggests محمد), and no more duplicate words burning suggestion slots.
For users
Same API, better engine: gem install kotoshu (1.0.7),
cargo add kotoshu (0.3.0); server deployments pick the gem
up on their next build. Variant users (zh-Hant-HK/TW/CN) get their own
vocabulary; Arabic and Hebrew users get vocalized-input support; every
language gets a cleaner suggestion slate.