Abstract
In this paper, we attempt to make a formal analysis of the performance in automatic part of speech tagging. Lower and upper bounds in tagging precision using existing taggers or their combination are provided. Since we show that with existing taggers, automatic perfect tagging is not possible, we offer two solutions for applications requiring very high precision: (1) a solution involving minimum human intervention for a precision of over 98.7%, and (2) a combination of taggers using a memory based learning algorithm that succeeds in reducing the error rate with 11.6% with respect to the best tagger involved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Ali, K., and Pazzani, M. Error reduction through learning multiple descriptions. Machine Learning 24, 3 (1996), 173–202.
Brants, T. Tnt-a statistical part-of-speech tagger. In Proceedings of the 6th Applied NLP Conference, ANLP-2000 (Seattle, WA, May 2000).
Brill, E. Transformation-based error driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics 21, 4 (December 1995), 543–566.
Brill, E., and Wu, J. Classifier combination for improved lexical disambiguation. In In Proceedings of the Seventeenth International Conference on Computational Linguistics COLING-ACL’ 98 (Montreal, Canada, 1998).
Daelemans, W., Zavrel, J., van der Sloot, K., and van den Bosch, A. Timbl: Tilburg memory based learner, version 4.0, reference guide. Tech. rep., University of Antwerp, 2001.
Florian, R., Cucerzan, S., Schafer, C., and Yarowsky, D. Combining classifiers for word sense disambiguation. JNLE Special Issue on Evaluating Word Sense Disambiguation Systems (2002). forthcoming.
Gale, W., Church, K., and Yarowsky, D. Estimating upper and lower bounds on the performance of word-sense disambiguation programs. In Proceedings of the 30th Annual Meeting of the Association for Computational Linguistics (ACL-92) (1992).
Klein, D., Toutanova, K., Ilhan, I., Kamvar, S., and Manning, C. Combining heterogeneous classifiers for word-sense disambiguation. In Proceedings of the ACL Workshop on “Word Sense Disambiguatuion: Recent Successes and Future Directions” (July 2002), pp. 74–80.
Light, M., Mann, G., Riloff, E., and Breck, E. Analyses for elucidating current question answering technology. Journal of Natural Language Engineering (forthcoming) (2002).
Marcus, M., Santorini, B., and Marcinkiewicz, M. Building a large annotated orpus of english: the Penn Treebank. Computational Linguistics 19, 2 (1993), 313–330.
Mihalcea, R., and Bunescu, R. Levels of confidence in building a tagged corpus. Technical Report, SMU, 2000.
Ratnaparkhi, A. A maximum entropy part-of-speech tagger. In Proceedings of the Empirical Methods in Natural Language Processing Conference (Philadelphia, May 1996), pp. 130–142.
Schmid, H. Probabilistic part-of-speech tagging using decision trees. In International Conference on New Methods in Language Processing (Manchester, UK, 1994).
14. van Halteren, H., Zavrel, J., and Daelemans, W. Improving data driven wordclass tagging by system combination. In In Proceedings of the Seventeenth International Conference on Computational Linguistics (COLING-ACL’ 98) (Montreal, Canada, 1998).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mihalcea, R. (2003). Performance Analysis of a Part of Speech Tagging Task. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2003. Lecture Notes in Computer Science, vol 2588. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36456-0_16
Download citation
DOI: https://doi.org/10.1007/3-540-36456-0_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00532-2
Online ISBN: 978-3-540-36456-6
eBook Packages: Springer Book Archive