A New Method to Cluster HTML Documents Using Mixed Algorithms

Abstract

Given the high volume of web information, more attention has been paid to the automatic data extraction systems. One of the most important methods of data extraction is clustering. Today, many clustering methods are provided which are mostly based on vector models. In these models, each document is treated like a set of words, and the sequence of words in the sentence is ignored. Since the meanings in the natural language are completely dependent on the sequence of words, a great deal of shortcomings is observed in these methods. To overcome these shortcomings, this paper presents a new method for clustering HTML documents in which STC algorithm is considered for clustering snippets. This method, called clustering based on KS_STC key sentences, provides a weighted vector for each document and using this vector, the key sentences of each text are extracted from the document. Finally, these key sentences are given for clustering to the STC algorithm.

Keywords

• : html document clustering
• data mining
• information extraction systems
• classification
• information overload