International English gives better machine translation than standard English
There are six pairs of texts. The first text in a pair is standard English. The second text in a pair is equivalent to the standard English text, but it is optimised for machine translation. In January 2009, Google Translate was used to translate the texts into Bulgarian and into Spanish (www.google.co.uk/language_tools?hl=en).
Professional translators evaluated the translation for fluency and for accuracy of meaning.
Figure 1 summarises the results of the evaluation for fluency.
Figure 1. Fluency of machine translation for standard English and for international English
The increase in fluency for machine translation of international English is small. However, for machine translation, fluency is less important than accuracy of meaning.
Figure 2 summarises the results of the evaluation for accuracy.
Figure 2. Accuracy of machine translation for standard English and for international English
The increase in accuracy for machine translation of international English is large. However, machine translation of international English does not always give good results, as with sentence 5b for the Spanish translation.
Bulgarian
A professional translator evaluated the machine translations in February 2009.
| Ref. | English source | Bulgarian machine translation | Fluency | Accuracy |
|---|---|---|---|---|
| 1a | The request came out of the blue. | В искането се излезе от синьо. | Perfect fluency | Nonsense |
| 1b | The request came unexpectedly. | Искането дойде неочаквано. | Perfect fluency | Correct meaning |
| 2a | If the bid falls through, we will lose much money. | Ако офертата попада чрез, ние ще загубим много пари. | Low-quality fluency | Nonsense |
| 2b | If the bid is not successful, we will lose much money. | Ако офертата не е успешен, ние ще загубим много пари. | Acceptable fluency | Correct meaning |
| 3a | If the material is hard, do not study it. | Ако материалът е твърд, не проучване него. | Low-quality fluency | Nonsense |
| 3b | If the material is difficult, do not study it. | Ако материалът е трудно, да не го проучване. | Incomprehensible | Nonsense |
| 4a | The machine on the left is broken. | Машината в ляво е счупена. | Perfect fluency | Correct meaning |
| 4b | The machine that is on the left is broken. | Машината, която е в ляво е счупена. | Perfect fluency | Correct meaning |
| 5a | Our education zone gives you an insight into how various people within the organisation work, from the people known as the uniformed officers who check your bags at ports and airports, to the ‘The VAT Man’. | Нашето образование зона ви дава поглед към това как отделните хора в организацията работа, от хора, познати като униформени служители, които проверяват чантите на пристанищата и летищата, на "ДДС Man". | Low-quality fluency | Nonsense |
| 5b | Our education zone tells you about the work of people in the organisation, such as the uniformed officers who check your bags at ports and airports, and the ‘The VAT Man’. | Нашето образование зона ви казва за работата на хората в организацията, като униформени служители, които проверяват чантите на пристанищата и летищата, и "на ДДС Man". | Acceptable fluency | Correct meaning |
| 6a | They brought the matter of expenses up. | Те донесоха на разходите се. | Acceptable fluency | Nonsense |
| 6b | They brought up the matter of expenses. | Те повдигат въпроса на разноски. | Perfect fluency | Correct meaning |
Spanish
A professional interpreter evaluated the machine translations in February 2009. (An interpreter is a person who translates speech.)
| Ref. | English source | Spanish machine translation | Fluency | Accuracy |
|---|---|---|---|---|
| 1a | The request came out of the blue. | La petición llegó de la nada. | Perfect fluency | Correct meaning |
| 1b | The request came unexpectedly. | La petición llegó inesperadamente. | Perfect fluency | Correct meaning |
| 2a | If the bid falls through, we will lose much money. | Si se inscribe a través de la oferta, vamos a perder mucho dinero. | Perfect fluency | Incorrect meaning |
| 2b | If the bid is not successful, we will lose much money. | Si la oferta no tiene éxito, vamos a perder mucho dinero. | Perfect fluency | Correct meaning |
| 3a | If the material is hard, do not study it. | Si el material es duro, no estudiarla. | Acceptable fluency | Correct meaning |
| 3b | If the material is difficult, do not study it. | Si el material es difícil, no estudiarla. | Acceptable fluency | Correct meaning |
| 4a | The machine on the left is broken. | La máquina de la izquierda está roto. | Low-quality fluency | Ambiguous meaning |
| 4b | The machine that is on the left is broken. | La máquina que está a la izquierda está roto. | Perfect fluency | Correct meaning |
| 5a | Our education zone gives you an insight into how various people within the organisation work, from the people known as the uniformed officers who check your bags at ports and airports, to the ‘The VAT Man’. | Nuestra zona de la educación le da una idea de cómo varias personas dentro de la organización de trabajo, de la gente conoce como la agentes uniformados que comprobar su equipaje en los puertos y aeropuertos, a la «El Hombre del IVA». | Low-quality fluency | Nonsense |
| 5b | Our education zone tells you about the work of people in the organisation, such as the uniformed officers who check your bags at ports and airports, and the ‘The VAT Man’. | Nuestra zona de educación dice usted acerca de la labor de las personas en la organización, como los oficiales uniformados que comprobar su equipaje en los puertos y aeropuertos, y el "El Hombre del IVA». | Low-quality fluency | Nonsense |
| 6a | They brought the matter of expenses up. | Se planteó la cuestión de los gastos arriba. | Perfect fluency | Incorrect meaning |
| 6b | They brought up the matter of expenses. | Se planteó la cuestión de los gastos. | Perfect fluency | Correct meaning |
Notes
The sample size is too small for the evaluation to have statistical validity. To increase the quality of the evaluation, we must do the following things:
- Increase the size of the sample.
- Evaluate many languages.
- For each language, use many translators to evaluate the translations.
- Compare different machine translation tools.