How to evaluate the quality of MT output

Outline with detail

Introduction: the importance of MT evaluation

Evaluation tasks and scenarios, Human measures and automated metrics

Human MT Evaluation Measures

Adequacy and Fluency, Translation Ranking methods, Post-editing measures, Task-based evaluations, Error analysis taxonomies, executing human assessments, crowd-sourcing for MT evaluation, inter-coder agreement measures and normalization.

Automated MT Evaluation Metrics

Conceptual idea behind automated metrics, overview of the major commonly used automated metrics (BLEU, METEOR, TER), appropriate uses for automated metrics, known metric limitations, current research issues.

MT Evaluation within Commercial Translation Operations

Evaluating alternative possible MT engines, Contrasting MT system performance before and after development and/or customization, Evaluating post-editing, Asessing productivity gains.