Earlier this week, I stopped for breakfast in Sunset Park, Brooklyn, a largely Hispanic neighborhood where street vendors sell tamales and rice pudding out of orange Gatorade coolers. I speak some Spanish, but I wanted to test out Apple’s new “Live Translation” feature, which has been advertised as a sort of interpreter in your ears. I popped in my AirPods, pulled up the Translate app, and approached.
As I opened my mouth, the AirPods blared a message into my ears: AMBIENT SOUND LEVELS ARE HIGH. TRY MOVING iPHONE CLOSER TO THE AUDIO SOURCE TO CONTINUE TRANSLATION. The vendor had already begun explaining her offerings to me in a mix of Spanish and English, but the AirPods drowned out most of her words. I asked a question, in English, about the tamale fillings, then realized that I had to press an on-screen “Play” button for it to be read aloud by my device in Spanish. The vendor smiled (or maybe grimaced) and then responded. After a few seconds of delay, her Spanish was translated into my ears: “Green sauce, slices with cheese, slices with chicken, Molly, juaquillo.”
That final word is neither translated nor a word, but I took it to mean guajillo, a kind of chili pepper. Slices seemed to be a misunderstanding of the Spanish word rajas, which does technically mean “slices” but has a regional meaning—as in rajas con crema, or strips of roasted poblano peppers with cream (and in this case, cheese or chicken). The vendor said some other things that were unintelligible because of the overlap between her live speech and the lagging translation. Finally, I assumed that Molly was a bad translation of mole, the sauce, but I didn’t want to risk a morning dose of MDMA in the off chance that Apple was right. I opted for the salsa verde.
Another day, I spotted a flower store selling vibrant bouquets and wanted to know what kind of plants were included in an orange display. The proprietor laughed as my AirPods didn’t pick up, and thus could not translate, my own English words, so I just gestured to the flowers, which she called “cempasúchil, o flor de muerto.” Too much ambient sound, my AirPods said, so I lifted the phone closer to her face and asked, in English, if she could repeat herself. When Apple translated what she said, it didn’t actually provide an English word for cempasúchil—instead, it produced sampasúchil, an unusual spelling. I didn’t know what this meant, but the flowers were pretty and I bought them anyway. Later, I Googled the word. (It corrected to cempasúchil, which means “marigolds.”)
Apple did not respond to a request for comment on my experience with Live Translation. In the company’s defense, it notes that the feature is in beta mode, even though Live Translation is available to all customers outside the European Union, and says that the product’s “outputs may be inaccurate, unexpected, or offensive. Check important information for accuracy.” But Apple also announced the feature as a “transformational, hands-free capability” to help you “understand another language and communicate with others.” In the accompanying promotional video, an English-speaking woman buys flowers from a Spanish speaker, just as I had, though her experience is completely seamless.
Perhaps the market in Apple’s video was library-quiet, and each character had a perfectly pitched voice. (My own is an admittedly quiet baritone.) The feature did well when I tried it on a well-produced YouTube video in a soundproof room. But I soon realized that ambient noise was not the only issue the AirPods had: Live Translation specifically notes that the Spanish it translates is the kind spoken in Spain (population 49 million), not the kind spoken in Mexico or anywhere else in Latin America and the Caribbean (population 663 million). Although my technical problems could not be chalked up to vocabulary alone, it is true that several of the words that my AirPods stumbled on were those that are specifically rooted in Mexican Spanish: rajas, guajillo, mole, cempasúchil.
This explanation raised another question. Why would Apple, an American company headquartered in California, optimize for a relatively less common form of Spanish rather than the kind most likely to be spoken in and around the United States? It could have to do with the relative popularity of the iPhone in Spain, though the country’s residents don’t even currently have access to Live Translation because of restrictions put in place by the EU’s Digital Markets Act, which requires Apple software to work on non-Apple devices. For translation software to struggle with non-European vocabulary and phrases is not uncommon, but two other top translation tools, Google Translate and DeepL, translate rajas con crema to “strips of peppers with cream” and “chili peppers with cream,” respectively. Oddly, Apple offers Live Translation in Portuguese but as spoken in Brazil—the world’s seventh-most-populated nation.
Whatever the reason, the situation points to larger issues with AI translation. Another scene in Apple’s marketing video shows two people using AirPods to interpret between English and Portuguese as they have a conversation. The video offers the science-fictional promise of the Babel Fish from the novel The Hitchhiker’s Guide to the Galaxy—a fish that, when placed into a character’s ear, allows them to understand any language. But a language is more than a dictionary and syntax: It is made up of histories and cultures. Gestures, colloquialisms, facial expressions, local cuisine, and the like are not incidental to a tongue but constitute it; sometimes, to capture a word or phrase, in writing or in an algorithm, is to stamp out its meaning.
Still, even with these limitations in mind, Live Translation could have provided an impressive synthesis of AI software with existing hardware, a sort of science-fiction dream. It does not. The microphone in both the AirPods and the iPhone doesn’t seem up to the task; the constant warnings about background noise (triggered, in my experience, by traffic, wind, the hum of a fridge in a bakery, and the whir of an air conditioner in a grocery store) drown out and distract from any phrases or gestures you might understand naturally. Asking someone to read from your screen, or breaking eye contact to press a “Play” button, triples the duration of every utterance and quadruples everyone’s discomfort. God forbid that multiple people should speak at once. As far as translation goes, AI software has been able to expertly convert basic Spanish to English or German to French (the other two languages currently available in the AirPod translation feature) for many years; Apple hasn’t really attempted an algorithmic leap there.
When using the AirPods to translate Spanish, I appeared stiff and bumbling at best, terribly rude at worst. The Babel Fish is invisible and instantaneous, not science fiction so much as fantasy. Your throat and tongue and ears, and of course language itself, are of the physical world—unpredictable, fallible, and beautiful for it. Live Translation, in aiming to fix all that messiness in an algorithm streamed through Bluetooth earbuds, is not opening human communication up so much as flattening it.
