Reducing hardware hit by queries in web search engines

Autoren Marcelo Mendoza
Mauricio Marín
Verónica Gil-Costa
Flavio Ferrarotti
Titel Reducing hardware hit by queries in web search engines
Typ Artikel
Journal Information Processing & Management
Nummer 6
Band 52
DOI 10.1016/j.ipm.2016.04.008
Monat November
Jahr 2016
Seiten 1031-1053
SCCH ID# 16094

In this paper, we introduce a new collection selection strategy to be operated in search engines with document partitioned indexes. Our method involves the selection of those document partitions that are most likely to deliver the best results to the formulated queries, reducing the number of queries that are submitted to each partition. This method employs learning algorithms that are capable of ranking the partitions, maximizing the probability of recovering documents with high gain. The method operates by building vector representations of each partition on the term space that is spanned by the queries. The proposed method is able to generalize to new queries and elaborate document lists with high precision for queries not considered during the training phase. To update the representations of each partition, our method employs incremental learning strategies. Beginning with an inversion test of the partition lists, we identify queries that contribute with new information and add them to the training phase. The experimental results show that our collection selection method favorably compares with state-of-the-art methods. In addition our method achieves a suitable performance with low parameter sensitivity making it applicable to search engines with hundreds of partitions.