| Journal: | Computación y sistemas |
| Database: | |
| System number: | 000560162 |
| ISSN: | 1405-5546 |
| Authors: | Viveros Jiménez, Francisco1 Sanchez Perez, Miguel A1 Gómez Adorno, Helena1 Posadas Durán, Juan Pablo2 Sidorov, Grigori1 Gelbukh, Alexander1 |
| Institutions: | 1Instituto Politécnico Nacional, Centro de Investigación en Computación, Ciudad de México. México 2Instituto Politécnico Nacional, Escuela Superior de Ingeniería Mecánica y Eléctrica, Ciudad de México. México |
| Year: | 2018 |
| Season: | Abr-Jun |
| Volumen: | 22 |
| Number: | 2 |
| Pages: | 483-489 |
| Country: | México |
| Language: | Inglés |
| Document type: | Artículo |
| English abstract | It is well-known that the lack of quality data is a major problem for information retrieval engines. Web articles are flooded with non-relevant data such as advertising and related links. Moreover, some of these ads are loaded in a randomized way every time you hit a page, so the HTML document will be different and hashing of the content will be not possible. Therefore, we need to filter the non-relevant text of documents. The automatic extraction of relevant text in on-line text (news articles, etc.), is not a trivial task. There are many algorithms for this purpose described in the literature. One of the most popular ones is Boilerpipe and its performance is one of the best. In this paper, we present a method, which improves the precision of the Boilerpipe algorithm using the HTML tree for selection of the relevant content. Our filter greatly increases precision (at least 15%), at the cost of some recall, resulting in an overall F1-measure improvement (around 5%). We make the experiments for the news articles using our own corpus of 2,400 news in Spanish and 1,000 in English. |
| Disciplines: | Ciencias de la computación |
| Keyword: | Programación |
| Keyword: | Boilerplate removal, News extraction, HTML tree structure, Boilerpipe, Programming |
| Full text: | Texto completo (Ver HTML) Texto completo (Ver PDF) |
