Abstract

We explain the ideas of automatic text summarization approaches and the taxonomy of summary evaluation methods. Moreover, we propose a new evaluation measure for assessing the quality of a summary. The core of the measure is covered by Latent Semantic Analysis (LSA) which can capture the main topics of a document. The summarization systems are ranked according to the similarity of the main topics of their summaries and their reference documents. Results show a high correlation between human rankings and the LSA-based evaluation measure. The measure is designed to compare a summary with its full text. It can compare a summary with a human written abstract as well; however, in this case using a standard ROUGE measure gives more precise results. Nevertheless, if abstracts are not available for a given corpus, using the LSA-based measure is an appropriate choice.

Keywords

Cite this article

@article{cai-1726,
  author  = {Josef Steinberger and Karel Ježek},
  title   = {Evaluation Measures for Text Summarization},
  journal = {Computing and Informatics},
  volume  = {28},
  number  = {2},
  pages   = {251--275},
  year    = {2009},
  issn    = {1335-9150},
  url     = {https://cai.type.sk/content/2009/2/evaluation-measures-for-text-summarization/}
}
Download .bib