Glossika Italian and how AI can leave you lost in translation

As some readers will be aware, I have been learning Italian for some time now, with the ultimate aim of translating professionally from Italian to English. Aside from personal reasons to learn (of which I have plenty), it’s also is a logical string to add to my linguistic bow, especially as a football and motorsport specialist.

If you’ve learnt a foreign language to a high level before, you will know that you tend to improve very quickly at the beginning, but that your rate of progress usually flattens out as you get closer to fluency. This is definitely true of my Italian; I went from scratch to a decent intermediate level in about four months, but taking the step from intermediate to advanced is proving much trickier. I would like to get over this ‘intermediate hump’ by taking lessons in Italy in due course, but for the time being I’m doing my best to make progress from home.

Just before Christmas of 2023 I came across Glossika, a web-based program that promises to use spaced repetition techniques to teach you thousands of sentences in your chosen language, thus taking your language from A1 level in the European framework for languages (complete beginner) to C1 (advanced, and the minimum level I would normally expect to have before I would call myself anything like fluent). ‘Finally!’ I thought, ‘Something I can use on my own that might help me over the “intermediate hump”’ What’s more, it was reasonably priced, at USD 79 for a year’s access, so I decided to take the plunge. What followed was an object lesson in how language learners (and indeed translation buyers) can be short-changed by technology – and how hard it is to spot when you’ve been had.

The good

I dived straight in at an intermediate level, and initially I was very impressed. The interface is welcoming, uncluttered and easy to use, and the spaced repetition method really works; if you practise regularly, you will remember what you learn. As well as having you read and write the examples it gives you, Glossika also makes you repeat them aloud in the microphone, occasionally giving you just the English and asking you to come up with the Italian from memory. I am a firm believer that you need to speak a foreign language in order to learn it, and this feature gives Glossika something different in comparison with heavily reading-based language-learning apps like LingQ, which I have been using on and off for various languages since I was an undergraduate student.

By the time I’d been dutifully completing 50 ‘reps’ (repetitions of sentences) a day for about a month, I had progressed to what Glossika calls Upper B2 level, and I felt confident enough in what I had learnt to try it out on a real Italian, in the shape of my friend and colleague Silvia Martinelli MA – and I got a bit of a shock. As often as not, when I repeated some of the material I’d been learning, her instant reaction was “We would never say that!” or “That sounds really weird.” This prompted us to take a closer look at the full list of sentences – and uncover a host of problems that had been hiding right under my nose.

The bad

By the time Silvia had painstakingly taken me through the full list of sentences presented at B2 level, we had discarded fully 20% of the entire database as unsuitable and not worth learning. Some of the errors we found included:

·?????? Grammatically inaccurate sentences, such as “Siamo andati in crociera settimana scorsa e non c'erano molte persone sulla nave.” In Italian, you have to say “la settimana scorsa” – the ‘la’ cannot be omitted.

·?????? Mistranslations, such as Glossika translating ‘police officer’ as “ufficiale della polizia.” This is a word-for-word, literal translation, and you’re unlikely to hear it used by a native-speaker because it’s not idiomatic Italian. The formal translation should be “agente della polizia”, although you’re much more likely to come across “poliziotto/poliziotta” (policeman/policewoman).

·?????? Sentences that are not exactly wrong, but that sound very odd to an Italian, such as “Quanto costano i farmaci per questa prescrizione?” We Googled this sentence, and we got 0 hits, because an Italian just wouldn’t ask how much a prescription costs this way. For one thing, the Italian for a ‘prescription’ in this context is usually “ricetta”, not “prescrizione.” Again, this error is probably the result of a literal translation from US English.

·?????? Sentences you will never have any reason to use in Italy, such as “Portami allo Starbucks” (“Take me to the Starbucks”). Starbucks famously fell flat on its face in the home of real coffee and had to withdraw from the market with its tail between its legs.

·?????? Sentences taken directly from newspaper headlines. These examples were frequently not grammatical, precisely because they were written as headlines and not full sentences. Fortunately, they were easy to spot, because they were shown complete with the dates on which they appeared in the press.

These examples turned out to be symptomatic of a structurally flawed database that undermines the entire Glossika program.? Aside from the kinds of inaccuracies listed above, their approach to categorising their data is highly questionable at best. One example of this is the enormous number of medical-related sentences “taught” at B2 level. By my reckoning, approximately half of all the material at this level comes from a medical context, and it covers things like asking when you last had a tetanus jab, describing your diabetes or high blood pressure, or telling your doctor that your sinuses ache. Content like this is useful for medical professionals, but it’s not a priority for most learners, and I feel like I’ve learned an awful lot of specialist vocabulary I will never use.

The way Glossika assigns its sentences to different difficulty levels is also highly suspect. I’d love to find out why a sentence as simple as “Ho l’asma” can be considered upper-intermediate, but the real disappointment is the advanced C1 level database, the existence of which was the main reason I signed up in the first place. When I finally graduated to this level at towards the end of February, I discovered that the entire sentence list for C1 contains about 20 entries, and the level Glossika calls “Upper C1” has just one example in it. That sentence is “L’Acropli si trova ad Atene” (“the Acropolis is in Athens”), which is the kind of simple material I’d expect to be learning as a beginner.

The ugly

In addition to these errors, there were other, more minor annoyances that made for an increasingly frustrating experience as time wore on. One of the most irritating of these was the blanket use of US English, despite the English sentences being read by a voice with a British accent. Hearing a Brit talk about ‘looking under the hood’ and ‘turning left at this intersection’ really grated on my Limey ears.? A potentially more serious issue was that some of the English translations, while not inaccurate as such, were very free renderings of the original Italian. For instance, the (American) English sentence ‘Could you take a look under the hood?’ was translated as “Potrebbe dare un’occhiata al motore?”. In my view, there’s no reason not to translate this sentence as “Could you take a look at the engine?”, which is much closer to the Italian and therefore much more appropriate for a language-learning app. There were a few technical bugs, too, such as parts of the interface not loading properly, which forced me to keep refreshing the page until I was able to see what I was supposed to be working on.

Conclusion

To be blunt, I cannot recommend Glossika to anyone looking to study Italian at any level. The underlying spaced repetition method is highly effective, but the material it “teaches” is so flawed that you have to? run literally every new sentence past a native speaker to check whether it’s accurate, and no learner should have to do that. Moreover, despite what Glossika’s interface and marketing material tell you, Glossika Italian does not have a C1 level worth the name at the time of writing; for Glossika to claim otherwise is misleading at best, and left me feeling as if I’d been duped out of the purchase price. If you have money to invest in learning Italian, try something like Rocket Italian instead – I moved on to that when I finished Glossika and I’m finding it a much more reliable way of learning the language (and, to be clear, they haven’t paid me to write that).

Postscript: What Glossika tells you about language learning and translation

As I worked my way up Glossika’s levels and became less and less confident in what it was showing me, I began to wonder how on Earth they could have got so much, so wrong. The answer, it turns out, was staring me in the face all along: Glossika relies heavily on AI, as the URL of its homepage will tell you. Once I spotted that, I began to notice that a lot of the problems I experienced with the system are the same ones we grapple with in the translation industry.

I’ve been dealing with machine translation on a regular basis for the last seven years, and I am trained to pick out the kinds of mistakes that only a computer makes. In hindsight, those tell-tale signs were all over Glossika, from outright mistranslations to strange word order and changing register within the same sentence. Nevertheless, it was only when I started asking Silvia to assess every single sentence that I actually spotted what was going on, and as a language professional, that got me thinking.

When I started working with Glossika, I wanted a language product I could rely on, but my Italian wasn’t good enough to work out that what it was teaching me was, in many cases, not fit for purpose. ?It’s the same for a lot of translation buyers – and that’s one reason why there are so many bad translations in circulation.

Let’s imagine you need a translation from Italian into English, but you don’t speak any English yourself. You would probably be tempted to save money by feeding the text into a machine like Google Translate or DeepL, and if you did, you would get something that looked like English. However, unless you are native-speaker of English, you are very unlikely to be able to spot where the machine has made mistakes. The first clue you get that your shiny new translation isn’t as good as it looks might well be when a native-speaker tells you “We would never say that!” – just like I did when I first started running Glossika’s sentences past Silvia.

AI can produce a text, but unless you’re a native-speaker of the relevant language, you’ll struggle to judge whether that text is any good. A native-speaker can spot poorly translated copy a mile off, and if they do, they’re likely to be unimpressed. As former West German Chancellor Willy Brandt famously pointed out, if you’re selling, you have to speak your customers’ language. ?Computers can’t do that – only humans can.

If you need a translation that really resonates with your target market, hire a professional linguist who will craft a text to your specific requirements and really understands your business. It won’t be cheap, but you’ll get top-quality copy that will really impress your customers, not a Glossika-style collection of random sentences.