Conference paper Open Access

Using the Winograd Schema Challenge as a CAPTCHA

Nicos Isaak; Loizos Michael

CAPTCHAs have established themselves as a standard technology to condently distinguish humans from bots. Beyond the typical use for security reasons, CAPTCHAs have helped promote AI research in challenge tasks such as image classication and optical
character recognition. It is, therefore, natural to consider what other challenge tasks for AI could serve a role in CAPTCHAs. The Winograd Schema Challenge (WSC), a certain form of hard pronoun resolution tasks, was proposed by Levesque as such a challenge task to promote research in AI. Based on current reports in the literature, the WSC remains a challenging task for bots, and is, therefore, a candidate to serve as a form of CAPTCHA. In this work we investigate whether this a priori appropriateness of the WSC as a form of CAPTCHA can be justied in terms of its acceptability by the human users in relation to existing CAPTCHA tasks. Our empirical study involved a total of 329 students, aged between 11 and 15, and showed that the WSC is generally faster and easier to solve than, and equally entertaining with, the most typical existing CAPTCHA tasks.

This work has been partly supported by the project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 739578 (RISE – Call: H2020-WIDESPREAD-01-2016-2017-TeamingPhase2) and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development. ©The authors
Files (722.5 kB)
Name Size
722.5 kB Download
  • Adam Richard-Bollans, L Gomez Alvarez, and Anthony G Cohn. The Role of Pragmatics in Solving the Winograd Schema Challenge. In Proceedings of 13th International Symposium on Commonsense Reasoning (Commonsense-2017). CEUR Workshop Proceedings, 2017.

  • Altaf Rahman and Vincent Ng. Resolving Complex Cases of Denite Pronouns: The Winograd Schema Challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '12, pages 777{789, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics.

  • Arpit Sharma, Nguyen H Vo, Somak Aditya, and Chitta Baral. Towards Addressing the Winograd Schema Challenge - Building and Using a Semantic Parser and a Knowledge Hunting Module. In Proceedings of the Twenty-Fourth International Joint Conference on Articial Intelligence, IJCAI, pages 25{31, 2015.

  • By Liam Tung. Google algorithm busts CAPTCHA with 99.8 percent accuracy, 2017. [Online; accessed August-2018].

  • By Rob Verger. Google just made the Internet a tiny bit less annoying, 2017. [Online; accessed August-2018].

  • By Tara Seals. unCAPTCHA Defeats Google CAPTCHA, 2017. [Online; accessed August-2018].

  • Carlos Javier Hernandez-Castro and Arturo Ribagorda. Pitfalls in CAPTCHA design and imple- mentation: The Math CAPTCHA, a case study. computers & security, 29(1):141{157, 2010.

  • Christiana Themistocleous. Written Cypriot Greek in online chat: Usage and attitudes. In Pro- ceedings of the 8th International Conference on Greek Linguistics, volume 30, pages 473{488. University of Ioannina Ioannina, 2009.

  • Christos A Fidas, Artemios G Voyiatzis, and Nikolaos M Avouris. On the necessity of user-friendly CAPTCHA. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2623{2626. ACM, 2011.

  • David Bender. Establishing a Human Baseline for the Winograd Schema Challenge. In MAICS, pages 39{45, 2015.

  • Ernest Davis, Leora Morgenstern, and Charles Ortiz. Human tests of materials for the Winograd Schema Challenge 2016. 2016.

  • Evan Ackerman. Winograd Schema Challenge Results: AI Common Sense Still a Problem, for Now. Spectrum, 2016.

  • Haoruo Peng, Daniel Khashabi, and Dan Roth. Solving Hard Coreference Problems. Urbana, 51:61801, 2015.

  • Hector J. Levesque. The Winograd Schema Challenge. In AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, number SS-11-06. American Association for Articial Intelligence, 2011.

  • Jeff Yan and Ahmad Salah El Ahmad. Breaking Visual CAPTCHAs with Naive Pattern Recog- nition Algorithms. In Computer Security Applications Conference, 2007. ACSAC 2007. Twenty- Third Annual, pages 279{291. IEEE, 2007.

  • Jeff Yan and Ahmad Salah El Ahmad. Usability of CAPTCHAs or usability issues in CAPTCHA design. In Proceedings of the 4th symposium on Usable privacy and security, pages 44{52. ACM, 2008.

  • Jeremy Elson, John JD Douceur, Jon Howell, and Jared Saul. Asirra: A CAPTCHA that Exploits Interest-Aligned Manual Image Categorization. 2007.

  • Kevin Bock, Daven Patel, George Hughey, and Dave Levin. unCaptcha: A Low-Resource Defeat of reCaptcha's Audio Challenge. In Proceedings of the 11th USENIX Conference on Oensive Technologies, pages 7{7. USENIX Association, 2017.

  • Luis Von Ahn, Manuel Blum, Nicholas J Hopper, and John Langford. CAPTCHA: Using Hard AI Problems For Security. In International Conference on the Theory and Applications of Cryp- tographic Techniques, pages 294{311. Springer, 2003.

  • Marios Belk, Panagiotis Germanakos, Christos Fidas, Andreas Holzinger, and George Samaras. Towards the Personalization of CAPTCHA Mechanisms Based on Individual Dierences in Cog- nitive Processing. In Human Factors in Computing and Informatics, pages 409{426. Springer, 2013.

  • Nicos Isaak and Loizos Michael. A Data-Driven Metric of Hardness for WSC Sentences. In Proceedings of the 4th Global Conference on Articial Intelligence (GCAI 2018). EasyChair, 2018.

  • Nicos Isaak and Loizos Michael. Tackling the Winograd Schema Challenge Through Machine Logical Inferences. In David Pearce and Helena Soa Pinto, editors, STAIRS, volume 284 of Frontiers in Articial Intelligence and Applications, pages 75{86. IOS Press, 2016.

  • Pascal Amsili and Olga Seminck. A Google-Proof Collection of French Winograd Schemas. In Second Workshop on Coreference Resolution beyond OntoNotes, page 24, 2017.

  • Sandhya Tarar Anvesh Sinha. Review Paper on Dierent CAPTCHA Techniques. IJCST Vol. 7, 2016.

  • Suphannee Sivakorn, Jason Polakis, and Angelos D Keromytis. I'm not a human: Breaking the Google reCAPTCHA. Black Hat,(i), pages 1{12, 2016.

  • Google no Captcha + INVISIBLE reCaptcha { First Experience Results Review, 2017. [Online; accessed August-2018].

  • Vu Duc Nguyen, Yang-Wai Chow, and Willy Susilo. On the security of text-based 3D CAPTCHAs. Computers & Security, 45:84{99, 2014.

  • Walid Khalifa Abdullah Hasan. A Survey of Current Research on Captcha. International Journal of Computer Science and Engineering Survey (IJCSES), 7(3):141{157, 2016.

Views 71
Downloads 45
Data volume 32.5 MB
Unique views 63
Unique downloads 42


Cite as