| Journal: | Polibits |
| Database: | PERIÓDICA |
| System number: | 000368533 |
| ISSN: | 1870-9044 |
| Authors: | Ekbal, Asif1 Bandyopadhyay, Sivaji1 |
| Institutions: | 1Jadavpur University, Department of Computer Science and Engineering, Calcuta, Bengala Occidental. India |
| Year: | 2008 |
| Season: | Ene-Jun |
| Number: | 37 |
| Country: | México |
| Language: | Inglés |
| Document type: | Artículo |
| Approach: | Experimental, aplicado |
| English abstract | Lexicon development and Part of Speech (POS) tagging are very important for almost all Natural Language Processing (NLP) applications. The rapid development of these resources and tools using machine learning techniques for less computerized languages requires appropriately tagged corpus. We have used a Bengali news corpus, developed from the web archive of a widely read Bengali newspaper. The corpus contains approximately 34 million wordforms. This corpus is used for lexicon development without employing extensive knowledge of the language. We have developed the POS taggers using Hidden Markov Model (HMM) and Support Vector Machine (SVM). The lexicon contains around 128 thousand entries and a manual check yields the accuracy of 79.6%. Initially, the POS taggers have been developed for Bengali and shown the accuracies of 85.56%, and 91.23% for HMM, and SVM, respectively. Based on the Bengali news corpus, we identify various word–level orthographic features to use in the POS taggers. The lexicon and a Named Entity Recognition (NER) system, developed using this corpus, are also used in POS tagging. The POS taggers are then evaluated with Hindi and Telugu data. Evaluation results demonstrates the fact that SVM performs better than HMM for all the three Indian languages |
| Disciplines: | Ciencias de la computación |
| Keyword: | Procesamiento de datos, Lingüística computacional, Procesamiento de lenguaje natural, Léxico multilingüe, Modelos ocultos de Markov, Máquinas vectoriales de soporte |
| Keyword: | Computer science, Data processing, Computing linguistics, Natural language processing, Multilingual lexicon, Hidden Markov models, Support vector machines |
| Full text: | Texto completo (Ver HTML) |
