Machine Translation for Indian Languages

Evaluation

The machine translation models will be evaluated using the following metrics:

1. CHRF++: ChrF and ChrF++ are two MT evaluation metrics. They both use the F-score statistic for character n-gram matches, and ChrF++ adds word n-grams as well which correlates more strongly with direct assessment.
2. BLEU: a metric for automatically evaluating machine-translated text. The BLEU score is a number between zero and one that measures the similarity of the machine-translated text to a set of high quality reference translations.
3. TER: TER (Translation Edit Rate, also called Translation Error Rate) is a metric to quantify the edit operations that a hypothesis requires to match a reference translation.

However, the submitted models will be ranked based on CHRF++ scores.
For simulating the evaluation SacreBLEU library can be used.Use this Link