ROUGE 2.0 is a Java Package for Evaluation of Summarization Tasks building on the Perl Implementation of ROUGE.

ROUGE stands for Recall-Oriented Understudy for Gisting Evaluation. It consists of a set of metrics for evaluating automatic summarization of texts as well as machine translation. It works by comparing an automatically produced summary or translation against a set of reference summaries (typically human-produced) or translations.

ROUGE 2.0 is a lightweight open-source tool that allows for easy evaluation of summaries or translation by limiting the amount of formatting needed in terms of reference summaries as well as system summaries. In addition, it also allows for evaluation of unicode texts known to be an issue with other implementations of ROUGE. One can also add new evaluation metrics to the existing code base or improve on existing ones.


Features

ROUGE 2.0 currently supports the following:

  • Evaluation of ROUGE-N (unigram, bigrams, trigrams, etc)
  • Stemming for different languages
  • Stopword removal with customizable stop words
  • Evaluation of unicode texts (e.g. Persian)
  • Synonyms capture for better agreement between system and reference summaries [English only and requires WordNet installation]
  • Evaluation of specific parts of speech (e.g. NN) [Uses Stanford POS Tagger]

This is an open-source project, so if you want to implement new measures, extend existing measures or improve on the code please read the documentation on how to contribute to this project.


Quick Start

To start evaluating your summaries using ROUGE, please move on to the usage instructions. To build on the source code or make contributions, please read the installation instructions and notes on contributions on Github.


License

This package is licensed under GNU GPL Version 3.


Download / Links