Conference paper Open Access

Adaptive focused crawling using online learning: A study on content related to Islamic extremism

Iliou, Christos; Tsikrika, Theodora; Kalpakis, George; Vrochidis, Stefanos; Kompatsiaris, Ioannis

Focused crawlers aim to automatically discover online content resources relevant to a domain of interest by automatically navigating through the Web link structure and selecting which hyperlinks to follow based on an estimation of their relevance to the topic of interest; to this end, classifier-guided approaches are typically employed for identifying hyperlinks having the higher likelihood of leading to relevant content. However, the training data used for building these classifiers might not be entirely representative of the domain of interest, or the domain of interest might change over time. To meet these challenges, this work proposes a novel adaptive focused crawling framework that allows the classifiers that underlie the hyperlink selection policy to be adapted based on the evidence they encounter during their crawls. Our frame-
work uses two different approaches to retrain its models: (i) Interactive Adaptation, where a user manually evaluates the discovered resources, and (ii) Automatic Adaptation, where the framework uses the already trained classifiers to assess the relevance of newly discovered resources.The evaluation experiments in the domain of Islamic extremism indicate
the effectiveness of online learning in focused crawling.

Files (1.1 MB)
Name Size
INSCI2018_final_submission.pdf
md5:51138867d13cacae34cce393497a594a
1.1 MB Download
29
22
views
downloads
All versions This version
Views 2929
Downloads 2222
Data volume 25.1 MB25.1 MB
Unique views 2626
Unique downloads 2020

Share

Cite as