When your user changes language mid-sentence

Ask a bilingual colleague what language they speak at work and they will give you an answer that their actual sentences contradict within a minute. The nouns come from one language, the verbs from another, and the switch happens mid-clause without either speaker noticing.

This is normal speech for a large share of the world. It is also the case that most speech pipelines handle worst, because the standard design makes a single language decision at the start of an utterance and never revisits it.

The same phenomenon, many names#

Hinglish mixes Hindi and English. Spanglish mixes Spanish and English, Taglish mixes Tagalog and English, Franglais mixes French and English, and across North Africa, Arabic mixes with French while across the Gulf and the Levant it mixes with English. Speakers of each of these would tell you they are speaking one language. Several of the countries where this is most common are also the markets where adoption is running ahead, so the two problems arrive together.

The scale is not marginal. The authors of a Hinglish speech corpus note that “more than 250 million people are estimated to engage in code-switched communication, especially blending English with Hindi (Hinglish), making it one of the largest bilingual populations globally” (Singh et al., Data in Brief, 17 July 2025).

Code-switching sits inside the wider question of whether a speech stack covers the varieties you ship to, and it is the part of that question most often left untested. For an app, this is a product-defining fact rather than a linguistic curiosity. If a quarter of your users mix languages in ordinary requests and your pipeline treats mixing as noise, a quarter of your users get a worse product and none of them will file a bug that says so.

What the switch costs, measured#

The Hinglish corpus above was built for exactly this measurement. It holds 3,318 audio segments from adults and 1,858 from children, 5.24 hours of read and spontaneous speech in total, and the authors report that standard monolingual models underperformed “by approximately 42 % WER on our test set”.

Two features of that study are worth keeping. The corpus size is small, which is itself the finding: a benchmark for one of the world’s largest bilingual populations runs to five hours. And it includes children, who are usually absent from evaluation data and who switch more freely than adults.

Look at which words switch. In requests, the switched spans are overwhelmingly nouns and technical terms: invoice, settlement, delivery, appointment, card, refund. They are the object of the sentence and frequently the argument your tool call needs. A model that renders the Arabic perfectly and mangles the English word has lost the part of the utterance that decides which tool to call.

The pipeline shape that fails#

The common design samples the opening audio, picks a language, loads that language’s decoder and proceeds. It works well for monolingual speech and it has one structural flaw: the decision is made once, on the least informative part of the utterance, and there is no mechanism for revisiting it.

The silence is the dangerous part. A decoder constrained to Arabic does not return an error when it meets an English word. It returns the Arabic words that sound closest, which reads as a plausible sentence and means something different. A confidence score on the utterance will often look fine, because the model is confident about the wrong thing.

Vendors are moving in the right direction. Google’s Live API documents conversation in 70 supported languages and automatic language detection in its transcription (Gemini Live API documentation). Whisper was trained on 680,000 hours of multilingual supervision (Radford et al., 6 December 2022). Multilingual capability is not the same as intra-utterance switching, and the only way to know which one you have bought is to test the switched case directly.

Your archive recordings understate it#

Teams that do build a test set usually build it from whatever audio they already hold, which quietly biases the result.

The Casablanca corpus documents the effect. Across eight Arabic dialects it counted 234 English code-switching segments totalling about 22 minutes, against 1,220 French segments totalling an hour and 44 minutes. When the authors checked the dates of the source material, the pattern was clear: Egyptian episodes from 1997 to 2018 carried 72 switching instances, Jordanian from 1985 to 2000 carried 52 and Emirati from 1995 to 2009 carried 59, while Algerian episodes from 2004 to 2017 carried 586 and Moroccan from 2016 to 2018 carried 598 (Talafha et al., EMNLP 2024).

The corpus also records something most evaluations ignore, which is that code-switched words appear written in both the Latin and the Arabic script. A user who types the switched word will sometimes type it one way and sometimes the other, and a system that treats those as different tokens will fail to match the same product name to itself.

Building the test set#

Record it now, from the users you actually have, in the settings they use the app in. Switching rates vary by age, region and topic, and the switching your team does in a meeting is not the switching your users do while ordering groceries.

Score the switched spans separately from the rest. An overall word error rate averages the easy monolingual stretches together with the hard switch points and reports something in between that describes neither. The number you want is the error rate on the spans where a language boundary is crossed.

Weight the words your product cannot mishear. Fifty terms, comprising your product names, action verbs, currencies and the handful of English nouns your users always reach for. An error rate on those is worth more than the aggregate.

Then measure the thing one layer up, which is whether the agent still called the right tool. A mistranscribed word that the model resolves from context costs nothing. The same error on an amount is a different category of problem, and it is the same asymmetry that governs shipping to a variety the corpora barely contain.

Designing for the switch#

Accept mixed input and answer in one language. Our production rule is to reply in the user’s language and, for Arabic, to reply in Modern Standard Arabic rather than the dialect the user spoke. Understanding tolerates the mixture; the reply is clearer and safer to synthesise in one variety, and the synthesis constraints that drive that choice are in what a speech engine needs from a hard language.

Keep product terms in the script users read them in. A payments merchant who says “settlement” in the middle of an Arabic sentence expects to see the English word on screen, and transliterating it back into Arabic script makes the widget harder to scan.

Make repair cheap. A user who sees a switched noun rendered wrongly should be able to correct that one span rather than repeat the sentence, which is the same repair affordance dialect variation demands once a product is live.

And do not tell people to pick a language. The instruction to speak one language at a time is the interface equivalent of asking a user to write their address in block capitals. It works, in the sense that compliant users comply, and it turns a bilingual feature into a monolingual one.

Common questions#

What is code-switching in speech recognition? It is a speaker changing language inside a single utterance, often mid-clause. It is ordinary behaviour for bilingual populations, and it breaks pipelines that assign one language per utterance.

How much does code-switching degrade recognition? On a published Hinglish corpus, standard monolingual models underperformed by roughly 42 percent word error rate on the code-switched test set. The degradation concentrates on the switched spans rather than spreading evenly.

Will automatic language detection solve it? Not on its own. Detection picks a language for the utterance and the switch happens after that decision. Detection helps when someone changes language between turns, which is a different problem from changing language inside one sentence.

Which words tend to switch? Nouns and technical terms, overwhelmingly. In app requests those are the objects of the sentence and frequently the arguments a tool call needs, which is why the error rate on switched spans matters more than the aggregate.

Can I build a test set from recordings I already have? Only carefully. Analysis of one multi-dialect corpus found newer source material carrying an order of magnitude more switching than older material from the same region, so an archive-based benchmark is likely to understate what current users do.

Sources#

• Singh, Singh and Kadyan, HiACC: Hinglish adult & children code-switched corpus, Data in Brief, 17 July 2025. Accessed 12 September 2026.
• Talafha et al., Casablanca: Data and Models for Multidialectal Arabic Speech Recognition, EMNLP 2024. Accessed 12 September 2026.
• Google, Live API documentation. Accessed 12 September 2026.
• Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision, 6 December 2022. Accessed 12 September 2026.