Abstract
In this paper we develop the general framework for text representation based on fuzzy set theory. This work is extended from our original ideas [5],[4], in which a document is represented by a set of fuzzy concepts. The importance degree of these fuzzy concepts characterize the semantics of documents and can be calculated by a specified aggregation function of index terms. Based on this representation, a general framework is proposed and applied to text categorization problem. An algorithm is given in detail for choosing fuzzy concepts. Experiments on the real-world data set show that the proposed method is superior to the conventional method for text representation in text categorization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Billhardt, H., Bonajo, D., Maojo, V.: A context vector model for information retrieval. Journal of the American Society for Information Science and Technology (JASIST) 53(3), 236–249 (2002)
Buell, D.A.: An analysys of some fuzzy subsets application to information retrieval systems. Fuzzy Sets and Systems 7(1), 35–42 (1982)
Deerwester, S., Furnas, G.W., Dumais, S., Landauer, T.K.: Indexing by latent semantic indexing. Journal of the American Society for Information Science and Technology (JASIST) 41(6), 391–407 (1990)
Doan, S.: A fuzzy-based approach to text representation in text categorization. In: Proceeding of 14th IEEE Int’l. Conference on Fuzzy Systems - FUZZ-IEEE 2005, Nevada, U.S., pp. 1008–1013 (2005)
Doan, S., Horiguchi, S.: A new text representation using fuzzy concepts in text categorization. In: Proceeding of 1st International Conference on Fuzzy Set and Knowledge Discovery (FSKD), Singapore, vol. 2, pp. 514–518 (2002)
CMU Text Learning Group. 20newsgroups dataset, http://www.cs.cmu.edu/~textlearning
Joachims, T.: Text categorization with support vector machines: Learning with many relevant features. In: Proceedings 10th European Conference on Machine Learning (ECML), pp. 137–142 (1998)
Lewis, D.: Representation and Learning in Information Retrieval. PhD thesis, Graduate School of the University of Massachusetts (1991)
Lucarella, D., Marara, R.: First: fuzzy informatioon retrieval system. Journal of Information Science 17(2), 81–91 (1991)
Manning, C.D., Schutze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)
Miyamoto, S.: Fuzzy Sets in Information Retrieval and Cluster Analysis. Kluwer Academic Publishers, Dordrecht (1990)
Molinari, A., Pasi, G.: A fuzzy representation of html document for information retrieval system. In: Proceeding of 5th IEEE Int’l. Conference on Fuzzy Systems, pp. 107–112 (1996)
Moulinier, I.: A framework for comparing text categorization approaches. In: AAAI Symposium on Machine Learning and Information Access. Stanford University (1996)
Moulinier, I., Ganascia, J.G.: Applying an existing machine learning algorithm to text categorization. In: Wermter, S., Riloff, E., Schaler, G. (eds.) Connectionist, Statistical and Symbolic Approaches to Learning for Natural Language Processing, pp. 343–354. Springer, Heidelberg (1996)
Murai, T., Miyakoshi, M., Shimbo, M.: A fuzzy document retrieval method based on two-valued indexing. Fuzzy Sets and Systems 30(2), 103–120 (1989)
Rocchio, J.: Relevance feedback in information retrieval. In: Salton, G. (ed.) The SMART retrieval system: Experiments on Automatic Document Processing, ch. 14, pp. 313–323. Prentice-Hall, Englewood Cliffs (1971)
Salton, G., Buckley, C.: Term weighting approaches in automatic text retrieval. Information Processing and Management 24(5), 513–523 (1988)
Salton, G., Wong, A., Yang, C.S.: A vector space model for automatic indexing. Communications of the ACM 18(11), 613–620 (1975)
Sebastiani, F.: Machine learning in automated text categorization. ACM computing survey 34(1), 1–47 (2002)
Sparck-Jones, K.A.: A statistical interpretation of term specifility and its application in retrieval. Journal of Documentation 28(1), 11–20 (1972)
Witte, R., Bergler, S.: Fuzzy coreference resolution for summarization. In: Proceedings of 2003 International Symposium on Reference Resolution and Its Applications to Question Answering and Summarization (ARQAS), Venice, Italy, June 23–24, 2003, pp. 43–50. Università Ca’ Foscari (2003), http://rene-witte.net
Yang, Y.: An evaluation of statistical approaches to text categorization. Information Retrieval Journal 1, 69–90 (1999)
Yang, Y., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: Proceeding of the 14th International Conference on Machine Learning (ICML 1997), pp. 412–420 (1997)
Zadeh, L.A.: Fuzzy sets. Information Control 8, 338–353 (1965)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Doan, S., Ha, QT., Horiguchi, S. (2006). A General Fuzzy-Based Framework for Text Representation and Its Application to Text Categorization. In: Wang, L., Jiao, L., Shi, G., Li, X., Liu, J. (eds) Fuzzy Systems and Knowledge Discovery. FSKD 2006. Lecture Notes in Computer Science(), vol 4223. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11881599_73
Download citation
DOI: https://doi.org/10.1007/11881599_73
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45916-3
Online ISBN: 978-3-540-45917-0
eBook Packages: Computer ScienceComputer Science (R0)