Potential and limits of using post-edits as reference translations for MT evaluation
Popovic, Maja Arcan, Mihael Lommel, Arle
Popovic, Maja
Arcan, Mihael
Lommel, Arle
Loading...
Repository DOI
Publication Date
2016-05-30
Type
Conference Paper
Downloads
Citation
Popovic, Maja, Arcan, Mihael, & Lommel, Arle. (2016). Potential and limits of using post-edits as reference translations for MT evaluation. Paper presented at the EAMT 2016 The 19th Annual Conference of the European Association for Machine Translation, Riga, Latvia, 30 May - 01 June.
Abstract
This work investigates the potential use of post-edited machine translation (MT) outputs as reference translations for automatic machine translation evaluation, focusing mainly on the following important question: Is it necessary to take into account the machine translation system and the source language from which the given post-edits are generated? In order to explore this, we investigated the use of post-edits originating from different machine translation systems (two statistical systems and two rule-based systems), as well as the use of post-edits originating from two different source languages (English and German). The obtained results shown that for comparison of different systems using automatic evaluation metrics, a good option is to use a post-edit originating from a high-quality (possibly distinct) system. A better option is to use it together with other references and post-edits, however post-edits originating from poor translation systems should be avoided. For tuning or development of a particular system, post-edited output of this same system seems to be the best reference translation.
Publisher
Vilnius University, University of Latvia, Latvia University of Agriculture, Institute of Mathematics and Informatics of University of Latvia
Publisher DOI
Rights
Attribution-NonCommercial-NoDerivs 3.0 Ireland