Publication Details

Box clustering segmentation: A new method for vision-based web page preprocessing

ZELENÝ Jan, BURGET Radek and ZENDULKA Jaroslav. Box Clustering Segmentation: A New Method for Vision-based Page Preprocessing. Information Processing and Management, vol. 53, no. 3, 2017, pp. 735-750. ISSN 0306-4573. Available from: http://www.sciencedirect.com/science/article/pii/S0306457316301169
Czech title
Box Clustering Segmentation: Nová metoda vizuálně orientovaného předzpracování stránek
Type
journal article
Language
english
Authors
URL
Keywords

box clustering, graph clustering, vision-based page segmentation, VIPS

Abstract

This paper presents a novel approach to web page segmentation, which is one of substantial preprocessing steps when mining data from web documents. Most of the current segmentation methods are based on algorithms that work on a tree representation of web pages (DOM tree or a hierarchical rendering model) and produce another tree structure as an output. In contrast, our method uses a rendering engine to get an image of the web page, takes the smallest rendered elements of that image, performs clustering using a custom algorithm and produces a flat set of segments of a given granularity. For the clustering metrics, we use purely visual properties only: the distance of elements and their visual similarity. We experimentally evaluate the properties of our algorithm by processing 2400 web pages. On this set of web pages, we prove that our algorithm is almost 90% faster than the reference algorithm. We also show that our algorithm accuracy is between 47% and 133% of the reference algorithm accuracy with indirect correlation of our algorithms accuracy to the depth of inspected page structure. In our experiments, we also demonstrate the advantages of producing a flat segmentation structure instead of an hierarchy.

Published
2017
Pages
735-750
Journal
Information Processing and Management, vol. 53, no. 3, ISSN 0306-4573
Publisher
Elsevier Science
DOI
UT WoS
000396972300012
EID Scopus
BibTeX
@ARTICLE{FITPUB10821,
   author = "Jan Zelen\'{y} and Radek Burget and Jaroslav Zendulka",
   title = "Box clustering segmentation: A new method for vision-based web page preprocessing",
   pages = "735--750",
   journal = "Information Processing and Management",
   volume = 53,
   number = 3,
   year = 2017,
   ISSN = "0306-4573",
   doi = "10.1016/j.ipm.2017.02.002",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/10821"
}
Back to top