ROUGE (metric)

Short description: Metric used for testing NLP models

ROUGE, or Recall-Oriented Understudy for Gisting Evaluation,[1] is a set of metrics and a software package used for evaluating automatic summarization and machine translation software in natural language processing. The metrics compare an automatically produced summary or translation against a reference or a set of references (human-produced) summary or translation. ROUGE metrics range between 0 and 1, with higher scores indicating higher similarity between the automatically produced summary and the reference.

## Contents

* 1 Metrics
* 2 See also
* 3 References
* 4 External links

## Metrics

The following five evaluation metrics are available.

* ROUGE-N: Overlap of n-grams[2] between the system and reference summaries.
* ROUGE-1 refers to the overlap of unigrams (each word) between the system and reference summaries.
* ROUGE-2 refers to the overlap of bigrams between the system and reference summaries.
* ROUGE-L: Longest Common Subsequence (LCS)[3] based statistics. Longest common subsequence problem takes into account sentence-level structure similarity naturally and identifies longest co-occurring in sequence n-grams automatically.
* ROUGE-W: Weighted LCS-based statistics that favors consecutive LCSes.
* ROUGE-S: Skip-bigram[3] based co-occurrence statistics. Skip-bigram is any pair of words in their sentence order.
* ROUGE-SU: Skip-bigram plus unigram-based co-occurrence statistics.

## See also

* BLEU
* F-Measure
* METEOR
* NIST (metric)
* Noun-phrase chunking
* Word error rate (WER)

## References

1. ↑ Lin, Chin-Yew. 2004. ROUGE: a Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain, July 25 - 26, 2004.
2. ↑ Lin, Chin-Yew and E.H. Hovy 2003. Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In Proceedings of 2003 Language Technology Conference (HLT-NAACL 2003), Edmonton, Canada, May 27 - June 1, 2003.
3. ↑ 3.0 3.1 Lin, Chin-Yew and Franz Josef Och. 2004. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL 2004), Barcelona, Spain, July 21 - 26, 2004.

## External links

* ROUGE Usage Tutorial
* Java Implementation of ROUGE

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/ROUGE (metric). Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=ROUGE_(metric)&oldid=193380"