A survey of current practice of data search services

Relevancy ranking is an important component of making a data repository's search system
responsive to data seekers’ needs. The Research Data Alliance (RDA) Data Discovery Paradigms
Interest Group is a collaborative activity within our data community which aims to improve data
searchability. This survey is intended to gather information about the current practices and lessons
learnt by data repositories in implementing relevancy ranking in search systems. We expect that
analysis of the survey results will:

• Help data repositories choose appropriate technologies when implementing or improving their search functionality;
• Provide a means for sharing experiences in improving relevancy ranking;
• Capture the aspirations, successes and challenges encountered from research data repository managers;
• Help the Data Discovery Paradigms Interest group align future activities on data search improvement with the interests of data search service providers.

For the above the purpose, we designed a survey instrument to answer the following topics (the numbers in brackets indicate the number of questions asked per topic):

1. What are characteristics of each repositories (5)?
2. What are system configurations (e.g., ranking model, index methods, query methods) (7)?
3. Evaluation methods and benchmark (10)
   • What has been evaluated?
   • What evaluation methods have been applied?
   • How was the evaluation collection built?
   • What is approximate  performance range of search systems with certain configuration?
4. What methods have been used to boost searchability to web search engines (e.g., Google, Bing) (2)
5. What other technologies or system configurations have been employed (5)?
6. Wish list for future activities for the RDA relevance task force (2)?

Notes: Survey instruments from Version 1 and Version 2 have same questions, but order questions slightly different.  Version 2 has the one as instrumented to participants.