Automatic translation is close to fluent between certain pairs and unreliable between others. The variation tracks the available training material more closely than it tracks any linguistic difficulty.
Parallel text is the scarce resource
Translation systems learn from documents existing in two languages with sentences aligned between them.
That material accumulates where institutions have translated at volume: parliamentary records, treaty texts, technical documentation and subtitle collections.
Pairs covered by such archives have vastly more training material than pairs spoken by comparable numbers of people without an institution translating between them.
Many pairs are translated through a third language
Direct material between two less-resourced languages is often unavailable, so systems route through a high-resource language, usually English.
Each leg introduces its own errors, and distinctions absent from the intermediate language are lost permanently at the first step.
Politeness levels, evidential markers and grammatical gender that the pivot does not encode cannot be recovered on the second leg, because the information is already gone.
Domain matters as much as the pair
Because the training archives are institutional, systems are strongest on formal registers and weakest on everyday speech.
Legal and technical text between well-resourced pairs is handled well, while colloquial conversation in the same pair is noticeably worse.
A single quality figure for a language pair therefore describes an average across registers that differ widely in reliability.
Idiom, humour and regional usage sit furthest from the institutional archives, which is why translations that handle a contract cleanly still fail on a message from a friend.
Multilingual models transfer capability unevenly
Training one model across many languages at once allows knowledge to transfer between related ones, which has improved lower-resource languages considerably.
The benefit depends on having a well-resourced relative in the same model, so a language with close relatives gains far more than an isolated one.
Capacity is also shared, and adding languages to a fixed-size model degrades the ones already present, which is why coverage and quality trade against each other.
Evaluation is worst where it is most needed
Judging translation quality requires fluent speakers of both languages assessing output, and that is expensive.
Low-resource pairs consequently have the smallest test sets, so their reported scores are the least reliable figures in the field.
The practical consequence is that confidence in a translation should scale with how much material exists in that direction, and the systems themselves give no indication of which case you are in.