Apple researchers have published a method that detects when a speaker switches languages mid-sentence directly from audio, without transcribing it first, in a paper posted to Apple’s Machine Learning Research site in October 2026. Tested on 14 language pairs, the best version scored a mean F1 of 0.798, ahead of two conventional baselines.
The paper, Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models, is by Jie Chi, Zakaria Aldeneh, Masha Fedzechkina, Maureen de Seyssel and Natalie Schluter. The arXiv version lists Apple as their address and is dated October 14, 2026.
Why mixed-language speech is a problem
Code-switching, the paper explains, is “the alternation between languages within an utterance.” It is “common in multilingual speech but underrepresented in pretraining corpora,” so models trained on those corpora hear little of it.
Don’t miss the best of The Mac Observer
Set us as a preferred source and our Apple reporting ranks higher in your Google Search results and Discover feed — one tap, no account changes.
The authors point to how scarce natural examples are. Only 8% of utterances in the Miami corpus are code-switched, they write. Even in SEAME, a Mandarin and English corpus where some interview questions were designed to draw out switching, the reported share is 52%.
More examples could be mined from large speech collections, but the usual way to find them is costly. The paper cites a recent pipeline that uses speech recognition transcripts, a 14-billion-parameter language model and bilingual human feedback. Apple’s team wanted a lightweight filter that could flag likely candidates first, before the expensive steps run.
How the detector works
The method relies on what pretrained speech models already know. The researchers look inside an encoder for the small number of internal dimensions whose values differ most between two languages, using only recordings in one language at a time. At test time they watch those same dimensions across short windows of audio. If enough windows lean toward each language, the clip is marked as code-switched.
The paper says the approach uses “approximately 1%” of one Whisper-small encoder layer. Whisper-small covers about 100 languages, according to the paper. The team also tested two other open speech encoders, mHuBERT-147 and MMS-1B.
The results
On the read-speech part of the CS-FLEURS benchmark, which pairs English with 14 other languages, Whisper-small reached a mean F1 of 0.798. A CNN baseline scored 0.776 and a HuBERT linear baseline 0.751, both trained for monolingual language identification.
Scores varied by language. These are Whisper-small’s results for a selection of the 14 pairs, from the paper’s per-pair table:
| Language paired with English | Whisper-small F1 |
|---|---|
| Telugu | 0.870 |
| Hindi | 0.846 |
| Arabic | 0.832 |
| Russian | 0.827 |
| Korean | 0.800 |
| Japanese | 0.785 |
| Spanish | 0.775 |
| Mandarin Chinese | 0.762 |
| German | 0.754 |
Whisper-small had the top score on 11 of the 14 pairs. mHuBERT-147 did better on Portuguese and Spanish, and MMS-1B posted the highest single-pair score, 0.918 on Telugu and English. On conversational Mandarin and English speech from SEAME, the Whisper-small detector scored 0.734.
The authors also found that the same dimensions keep turning up. The most frequently selected Whisper-small dimension ranked in the top 1% for 55.2% of language pairs, against 5.3% under random selection. Leaving both target languages out when choosing dimensions lowered the CS-FLEURS score by only 0.003.
The paper is clear about its limits: the results come from benchmark datasets, and the team has “not yet tested the detector as part of a mining pipeline on unlabelled, in-the-wild audio.”
A second October paper on mixing languages
Apple’s research site also posted Mix, Don’t Tune: Bilingual Pre-Training Outperforms Hyperparameter Search in Data-Constrained Settings in October. It looks at languages with little training text, using Arabic as the low-resource target and English as the helper language, across four model sizes from 150M to 1.43B parameters and about 1,000 pre-training runs.
Its finding is that mixing in English data beats aggressive tuning. The authors say mixing gave a boost equivalent to two to three times the unique target data on validation loss, and two to 13 times on downstream task accuracy, with the gain growing with model size. Their advice: “mix in a high-resource language, prioritize the mixing ratio over hyperparameter tuning.”
What this means for Apple users
Neither paper names an Apple product. Both are about the data that multilingual models learn from, which matters to anyone who speaks more than one language at home.
Apple’s own HomeView page says its Siri AI is “available in English to start.” We looked at what that means for multilingual households in our report on HomeView’s English-only Siri AI. Research like this is about the training data such systems would need to handle mixed-language speech.
