10.1007/978-3-030-35288-2_24
https://zenodo.org/records/3923572
oai:zenodo.org:3923572
Nicos Isaak
Nicos Isaak
Open University of Cyprus
Loizos Michael
Loizos Michael
Open University of Cyprus and Research Center on Interactive Media, Smart Systems, and Emerging Technologies
WinoFlexi: A Crowdsourcing Platform for the Development of Winograd Schemas
Zenodo
2019
2019-11-25
eng
https://zenodo.org/communities/eu
Accepted pre-print
Creative Commons Attribution Non Commercial No Derivatives 4.0 International
The Winograd Schema Challenge, the task of resolving pronouns in certain carefully-structured sentences, has received considerable interest in the past few years as an alternative to the Turing Test. Systems developed to tackle this challenge have typically been evaluated on a small set of hand-crafted collections of sentences, since the development of new sentences by individuals is itself a rather challenging task, requiring care and creativity. In this paper we approach the problem of developing Winograd schemas via the introduction of WinoFlexi, a flexible online crowdsourcing system. Our empirical evaluation of the system’s performance suggests that WinoFlexi allows crowdworkers to develop Winograd schemas of quality similar to that of most typical existing collections.
This work has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and under Grant Agreement No 823783 and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.
European Commission
10.13039/501100000780
823783
WeNet - The Internet of US
European Commission
10.13039/501100000780
739578
Research Center on Interactive Media, Smart System and Emerging Technologies