Influence of Word Normalization on Text Classification

Influence of Word Normalization on Text Classification

In this paper we focus our attention on the comparison of various lemmatization and stemming algorithms, which are oftenused in nature language processing (NLP). We describe thealgorithm in detail and compare it with other widely used algorithms for word normalization on two different corpora. Wepresent promising results obtained by our EWN-based lemmatization approach in comparison to other techniques. We alsodiscuss the influence of the word normalization on classification task in general.

Keywords: lemmatization, classification, EuroWordNet, stemming, word normalization

Year: 2006

Download: download Full text [167 kB]

Authors of this publication:


Michal Toman


E-mail: mtoman@kiv.zcu.cz

Michal graduated at UWB in 2003, specialized in software engineering. Currently, he is a PhD student interested in information retrieval, multilingual text processing, word sense disambiguation and knowledge discovery.

Roman Tesař


Phone: +420 377632479
E-mail: roman.tesar@gmail.com
WWW: http://www.sweb.cz/romant1/CV.pdf

Roman is a PhD student at the Department of Computer Science and Engineering, Faculty of Applied Sciences, University of West Bohemia in Pilsen, Czech Republic. His work is focused on the utilization of word n-grams in text classification and document filtering.

Karel Ježek


Phone:  +420 377632475, 377632400
E-mail: jezek_ka@kiv.zcu.cz
WWW: http://www-kiv.zcu.cz/~jezek_ka/

Karel is a group coordinator and a supervisor of PhD students working at research projects of this Group.

Related Projects:


Project

Document Classification

Authors:  Jiří Hynek, Karel Ježek, Michal Toman, Roman Tesař, Zdeněk Češka, Petr Grolmus
Desc.:Use of inductive machine learning methods in classification of short text documents.