Why Latin Google Translate is so bad.

Google Translate for Latin is completely statistical. It has no model of grammar, syntax, or meaning. All it does its correlate sequences of up to five consecutive words in texts that have been manually translated into two or more languages.

More precisely, it has built up a hidden Markov model from all the manual translations that have been fed into it. Google calls this the Phrase-Based Machine Translation Model, or PBMTM. In November 2016, Google updated Translate to use the improved Neural Machine Translation Model for some languages, which does not work like this - but Latin is not among them.

Here's how the PBMTM works, roughly speaking. It assumes that people speak by randomly choosing one word after another, with the probabilities determined by the previous word spoken. For example, if you said "two", there is a certain probability that the next word will be "or". If you just said "or", there is a certain probability that the next word will be "butane". You can calculate an estimate of these probabilities by looking at all those texts which you fed in earlier. You can then generate random but just slightly coherent gibberish according to these probabilities:

Two

Two or

Two or butane

Two or butane gas

...

Two or butane gas can be attacked

If you use a contextual 'window' of more words - say, the previous three, four, or five - to find the next one, the resulting gibberish will look more likely to have been written by a schizophrenic than by an aphasic. Here's an example, with the contextual window highlighted.

I heard about

I heard about six

I heard about six today

I heard about six today, we

I heard about six today, we seek

...

Notice how each bolded phrase could have been in a real sentence: "Yesterday

That's a Markov model. The "hidden" part adds some complexity, which I'll defer to the end of this post. The basic idea is: Google Translate's PBMTM tries to choose the most probable next word, based on estimates of probability derived from five-word sequences from the corpus it possesses of texts in the input and target language (not only actual texts, but also many crowd-sourced translations).

What is Google Translate good for, then?

What Google Translate is most reliable for is translating documents produced by the United Nations between the languages in use there. This is because UN documents have provided a disproportionately large share of the manually translated texts from which Google Translate draws its five-word sequences, as UN documents are readily available in many different languages (as, for example, all official United Nations documents, meeting records and correspondence at the UN Headquarters is translated into at least Arabic, Chinese, English, French, Russian, and Spanish).

Witness what happens when I type this in:

À l'exception de ce qui peut être convenu dans les accords particuliers de tutelle conclus conformément aux Articles 77, 79 et 81 et plaçant chaque territoire sous le régime de tutelle, et jusqu'à ce que ces accords aient été conclus, aucune disposition du présent Chapitre ne sera interprétée comme modifiant directement ou indirectement en aucune manière les droits quelconques d'aucun État ou d'aucun peuple ou les dispositions d'actes internationaux en vigueur auxquels des Membres de l'Organisation peuvent être parties.

It gives me:

Except as may be agreed upon in the special guardianship agreements concluded in accordance with Articles 77, 79 and 81 and placing each territory under the trusteeship system, and until such agreements have been concluded, This Chapter shall not be construed as directly or indirectly modifying in any way the rights of any State or any people or the provisions of international instruments in force to which Members of the Organization may be parties.

Perfect! (Almost).

This is one reason why its Latin translations tend to be so poor: it has a very thin corpus of human-made translations of Latin to base its hidden Markov models on—oh, and it's using hidden Markov models.

So, until the United Nations starts doing its business in Latin, Google Translate's statistical model is not going to do a very good job. And even then, don't expect much unless you're translating text pasted directly from UN documents.

Further details on translation for the curious.

A hidden Markov model adds "states". The speaker is assumed to randomly transition from one "state" to another, and each state has its own set of probabilities for what word it will "emit". Thus a hidden Markov model is a statistical guess about what are the most likely states, transition probabilities, and emission probabilities that would produce a given set of sequences—assuming that they were produced in this random fashion.

Google Translate therefore calculates: "Given that the author in language A just said (up to) these five words, what is the most likely state that the author is in? OK, now, from the corresponding state in language B, what is the most likely word to output next?"

Here's an illustration of the five-word contextual window. If we enter the following:

Pants, as you expected, were worn.

Pants were worn.

Pants, as you expected, are worn.

The Latin translations (with manual translations back to English), are:

Anhelat quemadmodum speravimus confecta. (He is panting just as we hoped accomplished.)

Braccas sunt attriti. (The trousers have been worn away [like "attrition"]).

Anhelat, ut spe teris. (He is panting, just as, by hope, you are wearing [something] out.)

Notice that the first and third sentences border on ungrammatical nonsense. There aren't any five-word sequences in Google Translate's English database that line up well with "pants as you expected were/are," so it's flailing. Notice that in the third sentence, by the time it got to "worn", it had forgotten which sense of "pants" it chose at the start of the sentence. Or rather, it didn't forget, because it never tracked it. It only tracked five-word sequences. It gives the second sentence some sort of meaning, but even then, it is still very wrong - not only does it give 'worn' the wrong meaning (since, as I said before, it makes no semantic links between 'pants' and 'worn' that would imply the other definition of 'to wear), but it fails entirely to have the gender of the noun and verb agree, or to have the subject in the correct case.

So, whether the sentence makes sense sort of affects whether the translation means anything, but it's worse than that. What matters the most is exact, word-for-word matching with texts in the database.

Entering Latin into Google Translate (with words changed from the first sentence shown in bold):

Abraham vero aliam duxit uxorem nomine Cetthuram.

Quintilianus vero aliam duxit uxorem nomine Cetthuram.

Abraham vero aliam duxit uxorem nomine Iuliam.

Abraham vero canem duxit uxorem nomine Fido.

English output:

And Abraham took another wife, and her name was Keturah.

Quintilian, now the wife of another wife, and her name was Keturah.

And Abraham took another wife, and the name of his wife, a daughter named Julia.

And Abraham took a wife, and brought him to a dog by the name of Fido.

The Vulgate and the ASV translation (or similar) are among Google Translate's source texts, so it is very good at directly translating them - notice, though, what happens when the input is off by as little as one word. No longer able to spot the similarity, the software begins to translate smaller sentence fragments instead of the whole sentence: for example the fragment "uxorem nomine Cetthuram" is translated in both of the above sentences where it appears as "another wife, and her name was Keturah" despite the change of context.

This method of translation is why Google Translate works relatively well for more analytic languages , where strict word order is the most important for meaning, but dreadfully for the more synthetic languages such as Latin, where inflections define meaning. This is because in such analytic languages, subsequent words are more likely to have a semantic link that will allow retention of meaning when reproduced in the target language.

The Neural Machine Translation Model

The Neural Machine Translation Model has moved beyond simple statistical models for translation, using instead machine learning and neural networks.

According to a Google blog post on the subject:

At a high level, the Neural system translates whole sentences at a time, rather than just piece by piece. It uses this broader context to help it figure out the most relevant translation, which it then rearranges and adjusts to be more like a human speaking with proper grammar.

Even more advanced is Google's Zero-Shot Translation with Google’s Multilingual Machine Translation System, which can be thought of as translating the input phrases to its own semantic computer 'interlingua', and then to the output language. This is what permits the 'zero-shot translation' between language pairs it has ever analysed before. The example cited in the Google report demonstrated reasonable Korean-Japanese translation having only ever been trained on Japanese-English and Korean-English sentence pairs. Unfortunately, neither of these look to be coming to Latin any time soon.

If you want to see Markov chains working in English on reddit, r/SubredditSimulator is a subreddit entirely filled with Markov chain bots (but not hidden Markov chain models) seeded by the contents of subreddits. The post titles there are created by a Markov chain length (contextual 'window' of words) of two, meaning that every sequence of three words existed at some point in the subreddit from which the bot took its source. The comments are the created by the same method, except for the longer ones, which have a Markov chain length of three.

TL;DR. Just read it. In summary, though, Google's translation system for Latin does not have grammatical or semantic analysis at any level, but just a statistical model of the most likely word to appear next based on analysis of the corpus of works it has in both the input language and Latin. This works relatively well for languages like English, where word order is the most important thing, but is dreadful for languages like Latin.

So, NEVER use Google Translate for Latin if you want any sort of actual translation.

As the sidebar says:

Google Translate is always wrong, always. Don't even bother turning to Google Translate before asking us for help with a translation.

Entirely plagiarised from Ben Kovitz's excellent post on the Latin StackExchange, and then slightly extended.