{"course":{"slug":"choosing-your-tools","title":"Choosing and Using the Tools","url":"https://addaly.com/learn/choosing-your-tools","level":"beginner","track":"work"},"lesson":{"slug":"testing-your-language","title":"Testing the language you actually work in","url":"https://addaly.com/learn/choosing-your-tools/testing-your-language","position":13,"of":76,"minutes":9,"answer":"Language quality varies between models far more than English benchmarks suggest, and non-Latin scripts also cost several times more tokens for the same text, so test drafting, register.","keyIdea":"Language quality varies between models far more than English benchmarks suggest, and non-Latin scripts also cost several times more tokens for the same text, so test drafting, register, code-switching and numbers in the language you actually use."},"body":{"format":"markdown","text":"## The largest undiscussed difference\n\nTwo models can sit within a point of each other on English benchmarks and be a world apart in Marathi, Yoruba, Vietnamese, Amharic or Quechua. This gap is bigger than almost any other difference between tools, it decides whether the tool is usable for a great many people, and it appears on no comparison page anywhere, because the people writing those pages work in English.\n\nIf you work in another language, this lesson is the most important test in the course for you.\n\n## Why the gap exists\n\n**Training mixture.** Models are trained on text scraped largely from the web, and the web is wildly unequal by language. English is dominant, a dozen languages have substantial presence, and hundreds have very little. A model's fluency broadly tracks how much of that language it saw.\n\n**Deliberate emphasis.** Some makers invest specifically. Models from Chinese labs are usually stronger in Chinese; some are also unusually strong across Asian languages generally. European labs put weight on European languages. This is a real, checkable difference between families, not marketing.\n\n**The tokeniser, which also costs you money.** Text is split into tokens, and tokenisers are fitted mostly to Latin-script text. The same sentence in Hindi in Devanagari script can take two to four times as many tokens as its English translation. That means three things at once: you pay several times more for the same content, you fit far less into the context window, and generation is slower. Try it in any tokeniser tool. The number surprises everybody the first time.\n\n## Six tests worth running\n\n**Ordinary drafting.** A real message you would send. Judge whether it reads as written by a speaker or translated from English. Translated-English is the characteristic failure: grammatically correct, oddly formal, wrong idiom.\n\n**Register.** Most languages carry formality distinctions English handles loosely: the pronoun choice, honorifics, the difference between how you address a customer and a cousin. Ask for the same message twice, formal and casual, and see whether the difference is real.\n\n**Code-switching.** Hinglish, Spanglish, Taglish, Arabic-French mixing. A great deal of real communication does this, and models vary from natural to comically wrong. Give it a mixed input and see whether the reply mixes the same way.\n\n**Script and transliteration.** Roman-script Hindi, Arabizi, romanised Cantonese. Many people type this way. Some models handle it fluently; some silently answer in the wrong script.\n\n**Reasoning in the language.** Ask it to work out a problem *in* your language, not to translate a solution. Models often reason better in English and produce a weaker answer when made to think in another language. Worth knowing which of your tools does this.\n\n**Names, places, dates and numbers.** Local name spellings, address formats, the lakh-crore system, date orders, currency conventions. A tool that renders 1,50,00,000 as 150,000 will cause you a real problem eventually.\n\n## The workaround that is not a failure\n\nIf the best tool for your task is weak in your language, a two-step approach is often better than either alone: work in English and translate the final output yourself, or use a dedicated translation system for the last step. It is not elegant and it is frequently the right answer, especially for technical or legal material where the reasoning matters more than the phrasing.\n\nFree paths for the translation step are strong. **NLLB-200** from Meta and **MADLAD-400** from Google are openly released translation models covering 200 and 400 languages respectively, and both run on ordinary hardware. **Argos Translate** packages offline translation for the desktop. For everyday needs, the free tiers of the major translation services remain hard to beat.\n\n## One more thing to check: what it does with your language by default\n\nSome products detect the language of your message and reply in it; some reply in English unless told otherwise; some reply in the language of their own system prompt. In a mixed-language conversation this gets erratic, and it is worth knowing which behaviour you are buying, because correcting the language every few messages is a genuine daily irritation.\n\nAlso check what happens on a long input. A twenty-page document in a non-Latin script consumes far more of the window than its English equivalent, so a tool that handles a fifty-page English contract comfortably may truncate the same contract in Bangla. Test with a real document rather than a paragraph.\n\n## Writing it down\n\nAdd a line to your prompt file recording which tool won in your language and on what date, because this is the fastest-moving part of the whole field. Open models from labs in India, China, Africa and the Gulf are being released specifically to close these gaps, and a language that was badly served eighteen months ago may be well served now by a model you can download for nothing.\n\nCheck it yourself. Nobody is going to check it for you."},"check":{"question":"The same paragraph costs three times as many tokens in Devanagari as in English. What are the practical consequences?","options":[{"text":"It costs more per message, less of a document fits in the window, and generation is slower.","correct":true},{"text":"Answers in that language will be less accurate, because more tokens means more chances for the model to make a mistake.","correct":false,"tempting":"Draws a quality conclusion from a counting fact; accuracy is driven by how much of the language was in training, not by how finely the text is chopped."},{"text":"The model must translate internally into English first, which is why the token count rises.","correct":false,"tempting":"Reverses cause and effect: token counts come from how the tokeniser was fitted, and nothing about the count implies an internal translation step."},{"text":"Nothing changes in practice, because providers price by message or by request rather than by the number of tokens consumed.","correct":false,"tempting":"Assumes a billing model that does not hold on any major API, where input and output tokens are exactly what is metered."}]},"prev":{"slug":"known-answer-probes","title":"The prompts where you already know the answer","url":"https://addaly.com/learn/choosing-your-tools/known-answer-probes"},"next":{"slug":"blind-comparison","title":"Taking the brand off the answers","url":"https://addaly.com/learn/choosing-your-tools/blind-comparison"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Testing the language you actually work in', in Choosing and Using the Tools. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/choosing-your-tools/testing-your-language (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Testing the language you actually work in","courseTitle":"Choosing and Using the Tools","module":{"index":2,"slug":"testing-it-yourself","title":"Testing a tool yourself"},"answer":null,"keyIdea":"Language quality varies between models far more than English benchmarks suggest, and non-Latin scripts also cost several times more tokens for the same text, so test drafting, register, code-switching and numbers in the language you actually use.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-largest-undiscussed-difference","runs":[{"text":"The largest undiscussed difference"}]},{"t":"p","runs":[{"text":"Two models can sit within a point of each other on English benchmarks and be a world apart in Marathi, Yoruba, Vietnamese, Amharic or Quechua. This gap is bigger than almost any other difference between tools, it decides whether the tool is usable for a great many people, and it appears on no comparison page anywhere, because the people writing those pages work in English."}]},{"t":"p","runs":[{"text":"If you work in another language, this lesson is the most important test in the course for you."}]},{"t":"h2","id":"why-the-gap-exists","runs":[{"text":"Why the gap exists"}]},{"t":"p","runs":[{"text":"Training mixture.","b":true},{"text":" Models are trained on text scraped largely from the web, and the web is wildly unequal by language. English is dominant, a dozen languages have substantial presence, and hundreds have very little. A model’s fluency broadly tracks how much of that language it saw."}]},{"t":"p","runs":[{"text":"Deliberate emphasis.","b":true},{"text":" Some makers invest specifically. Models from Chinese labs are usually stronger in Chinese; some are also unusually strong across Asian languages generally. European labs put weight on European languages. This is a real, checkable difference between families, not marketing."}]},{"t":"p","runs":[{"text":"The tokeniser, which also costs you money.","b":true},{"text":" Text is split into tokens, and tokenisers are fitted mostly to Latin-script text. The same sentence in Hindi in Devanagari script can take two to four times as many tokens as its English translation. That means three things at once: you pay several times more for the same content, you fit far less into the context window, and generation is slower. Try it in any tokeniser tool. The number surprises everybody the first time."}]},{"t":"h2","id":"six-tests-worth-running","runs":[{"text":"Six tests worth running"}]},{"t":"p","runs":[{"text":"Ordinary drafting.","b":true},{"text":" A real message you would send. Judge whether it reads as written by a speaker or translated from English. Translated-English is the characteristic failure: grammatically correct, oddly formal, wrong idiom."}]},{"t":"p","runs":[{"text":"Register.","b":true},{"text":" Most languages carry formality distinctions English handles loosely: the pronoun choice, honorifics, the difference between how you address a customer and a cousin. Ask for the same message twice, formal and casual, and see whether the difference is real."}]},{"t":"p","runs":[{"text":"Code-switching.","b":true},{"text":" Hinglish, Spanglish, Taglish, Arabic-French mixing. A great deal of real communication does this, and models vary from natural to comically wrong. Give it a mixed input and see whether the reply mixes the same way."}]},{"t":"p","runs":[{"text":"Script and transliteration.","b":true},{"text":" Roman-script Hindi, Arabizi, romanised Cantonese. Many people type this way. Some models handle it fluently; some silently answer in the wrong script."}]},{"t":"p","runs":[{"text":"Reasoning in the language.","b":true},{"text":" Ask it to work out a problem "},{"text":"in","i":true},{"text":" your language, not to translate a solution. Models often reason better in English and produce a weaker answer when made to think in another language. Worth knowing which of your tools does this."}]},{"t":"p","runs":[{"text":"Names, places, dates and numbers.","b":true},{"text":" Local name spellings, address formats, the lakh-crore system, date orders, currency conventions. A tool that renders 1,50,00,000 as 150,000 will cause you a real problem eventually."}]},{"t":"h2","id":"the-workaround-that-is-not-a-failure","runs":[{"text":"The workaround that is not a failure"}]},{"t":"p","runs":[{"text":"If the best tool for your task is weak in your language, a two-step approach is often better than either alone: work in English and translate the final output yourself, or use a dedicated translation system for the last step. It is not elegant and it is frequently the right answer, especially for technical or legal material where the reasoning matters more than the phrasing."}]},{"t":"p","runs":[{"text":"Free paths for the translation step are strong. "},{"text":"NLLB-200","b":true},{"text":" from Meta and "},{"text":"MADLAD-400","b":true},{"text":" from Google are openly released translation models covering 200 and 400 languages respectively, and both run on ordinary hardware. "},{"text":"Argos Translate","b":true},{"text":" packages offline translation for the desktop. For everyday needs, the free tiers of the major translation services remain hard to beat."}]},{"t":"h2","id":"one-more-thing-to-check-what-it-does-with-your-language-by-d","runs":[{"text":"One more thing to check: what it does with your language by default"}]},{"t":"p","runs":[{"text":"Some products detect the language of your message and reply in it; some reply in English unless told otherwise; some reply in the language of their own system prompt. In a mixed-language conversation this gets erratic, and it is worth knowing which behaviour you are buying, because correcting the language every few messages is a genuine daily irritation."}]},{"t":"p","runs":[{"text":"Also check what happens on a long input. A twenty-page document in a non-Latin script consumes far more of the window than its English equivalent, so a tool that handles a fifty-page English contract comfortably may truncate the same contract in Bangla. Test with a real document rather than a paragraph."}]},{"t":"h2","id":"writing-it-down","runs":[{"text":"Writing it down"}]},{"t":"p","runs":[{"text":"Add a line to your prompt file recording which tool won in your language and on what date, because this is the fastest-moving part of the whole field. Open models from labs in India, China, Africa and the Gulf are being released specifically to close these gaps, and a language that was badly served eighteen months ago may be well served now by a model you can download for nothing."}]},{"t":"p","runs":[{"text":"Check it yourself. Nobody is going to check it for you."}]}],"check":{"question":[{"text":"The same paragraph costs three times as many tokens in Devanagari as in English. What are the practical consequences?"}],"options":[{"text":[{"text":"It costs more per message, less of a document fits in the window, and generation is slower."}],"correct":true,"tempting":null},{"text":[{"text":"Answers in that language will be less accurate, because more tokens means more chances for the model to make a mistake."}],"correct":false,"tempting":[{"text":"Draws a quality conclusion from a counting fact; accuracy is driven by how much of the language was in training, not by how finely the text is chopped."}]},{"text":[{"text":"The model must translate internally into English first, which is why the token count rises."}],"correct":false,"tempting":[{"text":"Reverses cause and effect: token counts come from how the tokeniser was fitted, and nothing about the count implies an internal translation step."}]},{"text":[{"text":"Nothing changes in practice, because providers price by message or by request rather than by the number of tokens consumed."}],"correct":false,"tempting":[{"text":"Assumes a billing model that does not hold on any major API, where input and output tokens are exactly what is metered."}]}],"order":[1,0,3,2],"why":null},"prev":{"slug":"known-answer-probes","title":"The prompts where you already know the answer","minutes":8},"next":{"slug":"blind-comparison","title":"Taking the brand off the answers","minutes":7}}}