does websearching provide 'hints' to studies overlooked by bibliographic database searching?

Background: In a previous case study,[1] the effect of geographic location on websearches using Google.com was assessed. In this case study, the effectiveness of the websearches from the previous case study were examined. The lens for this examination was the idea presented by Eynsenbach et al. that a contribution of websearching may be to examine hints to eligible studies if not the studies themselves.

Research aim: To explore the contribution of hints in web-searching: namely, are hints to studies frequent and do they help identify studies missed by bibliographic database searching?

A hint was defined as a link leading to identification of a relevant item.

Methods: All websearch results from the previous case study were assessed by the author to a predefined screening criterion. Eligible items were either studies or systematic reviews and these were categorised as either primary reports or hints. Three comparisons were made in analysis:

1. The prevalence of hints to primary reports;

2. The incidence of where a hint would have been necessary to identify the primary report based only on websearches in the original case study; and

3. The incidence of a hint where the primary report was not available in MEDLINE, Embase or The Cochrane Library.

Findings: Ten items fulfilled inclusion. These related to two studies and three systematic reviews. Seven hints compared to three primary reports were identified. All studies and two systematic reviews were available for retrieval in core bibliographic databases. It is possible that the systematic review not indexed in core databases (Pavey et al.) would be identified by a hint.

Conclusion: It was not possible to determine the importance of hints based on the sampling frame used in the previous case study but a role for hints is perceived in Health Technology Assessment, in particular to identify conference proceedings. With the pivot to on-line only conferences this may be valuable to test further.  

The websearches identified no studies uniquely. For searchers located outside of the UK, it is possible that the primary report of a systematic review by Pavey would have been missed by websearches, but they would have identified a copy of this review, so this finding is unremarkable.

The finding that websearches identified no unique studies or systematic reviews aligned with prior expectation since it is anticipated that studies/reviews would be identified by bibliographic database searching in systematic reviews of medicine or Health Technology Assessment. 

[1] Cooper, Chris, Theo Lorenc, and Ute Schauberger. "What You See Depends on Where You Sit: The Effect of Geographical Location on Web-Searching for Systematic Reviews: A Case Study." Research Synthesis Methods https://doi.org/10.1002/jrsm.1485