ABSTRACT
The Fourth Message Understanding Conference (MUC-4) is the latest in a series of conferences that concern the evaluation of natural language processing (NLP) systems. These conferences have reported on progress being made both in the development of systems capable of analyzing relatively short English texts and in the definition of a rigorous performance evaluation methodology. MUC-4 was preceded by a period of intensive system development by each of the participating organizations and blind testing using materials prepared by NRaD and SAIC that are described in this paper, other papers in this volume, and the MUC-3 proceedings [1].
- Proceedings of the Third Message Understanding Conference (MUC-3), May, 1991, Morgan Kaufmann.Google Scholar
- Sundheim, B., Plans for a Task-Oriented Evaluation of Natural Language Understanding Systems, in Proceedings of the Speech and Natural Language Workshop, February, 1989, Morgan Kaufmann, pp. 197--202. Google ScholarDigital Library
- Chinchor, N., MUC-4 Evaluation Metrics (in this volume).Google Scholar
- Lewis, D. and Tong, R., Text Filtering in MUC-3 and MUC-4 (in this volume).Google Scholar
- Krupka, G. and Rau, L., GE Adjunct Test Report: Object-Oriented Design and Scoring for MUC-4 (in this volume).Google Scholar
- Hirschman, L., Comparing MUCK-II and MUC-3: Assessing the Difficulty of Different Tasks, in Proceedings of the Third Message Understanding Conference (MUC-3), May, 1991, Morgan Kaufmann, pp. 25--30. Google ScholarDigital Library
- Chinchor, N., Statistical Significance of MUC-4 Results (in this volume).Google Scholar
- Chinchor, N., MUC-3 Linguistic Phenomena Test Experiment, in Proceedings of the Third Message Understanding Conference (MUC-3), May, 1991, Morgan Kaufmann, pp. 31--45. Google ScholarDigital Library
- Hirschman, L., An Adjunct Test for Discourse Processing in MUC-4 (in this volume).Google Scholar
- Sundheim, B., Overview of the Third Message Understanding Evaluation and Conference, in Proceedings of the Third Message Understanding Conference (MUC-3), May, 1991, Morgan Kaufmann, pp. 3--16. Google ScholarDigital Library
Comments