Conference paper Open Access

A Data-Driven Metric of Hardness for WSC Sentences

Nicos Isaak; Loizos Michael


MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="http://www.loc.gov/MARC21/slim">
  <leader>00000nam##2200000uu#4500</leader>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Evan Ackerman. Winograd Schema Challenge Results: AI Common Sense Still a Problem, for Now. Spectrum, 2016.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Dan Bailey, Amelia Harrison, Yuliya Lierler, Vladimir Lifschitz, and Julian Michael. The Winograd Schema Challenge and Reasoning about Correlation. In In Working Notes of the Symposium on Logical Formalizations of Commonsense Reasoning, 2015.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">David Bender. Establishing a Human Baseline for the Winograd Schema Challenge. In MAICS, pages 39{45, 2015.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Eric Bengtson and Dan Roth. Understanding the Value of Features for Coreference Resolution. In EMNLP, 10 2008.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Tejas Ulhas Budukh. An intelligent co-reference resolver for Winograd schema sentences containing resolved semantic entities, 2013.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Nicos Isaak and Loizos Michael. Tackling the Winograd Schema Challenge Through Machine Logical Inferences. In David Pearce and Helena Soa Pinto, editors, STAIRS, volume 284 of Frontiers in Articial Intelligence and Applications, pages 75{86. IOS Press, 2016.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Nicos Isaak and Loizos Michael. Using the Winograd Schema Challenge as a CAPTCHA. In Proceedings of the 4th Global Conference on Articial Intelligence (GCAI 2018). EasyChair, 2018.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Hector J. Levesque. The Winograd Schema Challenge. In AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, number SS-11-06. American Association for Articial Intelligence, 2011.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55{60, 2014.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Loizos Michael. Reading Between the Lines. In Proceedings of the 21st International Joint Con- ference on Articial Intelligence (IJCAI 2009), pages 1525{1530, July 2009.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Loizos Michael. Partial observability and learnability. Artif. Intell., 174(11):639{669, 2010.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Loizos Michael. Machines with Websense. In Proc. of 11th International Symposium on Logical Formalizations of Commonsense Reasoning (Commonsense 13), 2013.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Loizos Michael and Leslie G. Valiant. A First Experimental Demonstration of Massive Knowl- edge Infusion. In Proceedings of the 11th International Conference on Principles of Knowledge Representation and Reasoning (KR 2008), pages 378{388. AAAI Press, September 2008.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Haoruo Peng, Daniel Khashabi, and Dan Roth. Solving Hard Coreference Problems. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 809{819, 2015.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Altaf Rahman and Vincent Ng. Resolving Complex Cases of Denite Pronouns: The Winograd Schema Challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '12, pages 777{789, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Adam Richard-Bollans, L Gomez Alvarez, and Anthony G Cohn. The Role of Pragmatics in Solving the Winograd Schema Challenge. In Proceedings of 13th International Symposium on Commonsense Reasoning (Commonsense-2017). CEUR Workshop Proceedings, 2017.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Arpit Sharma, Nguyen H Vo, Somak Aditya, and Chitta Baral. Towards Addressing the Winograd Schema Challenge - Building and Using a Semantic Parser and a Knowledge Hunting Module. In Proceedings of the Twenty-Fourth International Joint Conference on Articial Intelligence, IJCAI, pages 25{31, 2015.</subfield>
  </datafield>
  <datafield tag="999" ind1="C" ind2="5">
    <subfield code="x">Leslie G. Valiant. Knowledge Infusion. In Proceedings of the 21st National Conference on Articial Intelligence - Volume 2, AAAI'06, pages 1546{1551. AAAI Press, 2006.</subfield>
  </datafield>
  <datafield tag="041" ind1=" " ind2=" ">
    <subfield code="a">eng</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Winograd Schema Challenge</subfield>
  </datafield>
  <controlfield tag="005">20191111070827.0</controlfield>
  <datafield tag="500" ind1=" " ind2=" ">
    <subfield code="a">This work has been partly supported by the project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 739578 (RISE – Call: H2020-WIDESPREAD-01-2016-2017-TeamingPhase2)  and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.

©The authors</subfield>
  </datafield>
  <controlfield tag="001">2671586</controlfield>
  <datafield tag="711" ind1=" " ind2=" ">
    <subfield code="d">18-21 September, 2018</subfield>
    <subfield code="g">GCAI-2018</subfield>
    <subfield code="a">4th Global Conference on Artificial Intelligence</subfield>
    <subfield code="c">Luxembourg City, Luxembourg</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Open University of Cyprus, Nicosia, Cyprus &amp; Research Center on Interactive Media, Smart Systems, and Emerging Technologies</subfield>
    <subfield code="a">Loizos Michael</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">1452161</subfield>
    <subfield code="z">md5:31cf508726996e3dcc2ee913a1fd3989</subfield>
    <subfield code="u">https://zenodo.org/record/2671586/files/A_Data-Driven_Metric_of_Hardness_for_WSC_Sentences.pdf</subfield>
  </datafield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  </datafield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2018-09-17</subfield>
  </datafield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">user-rise-teaming-cyprus</subfield>
    <subfield code="o">oai:zenodo.org:2671586</subfield>
  </datafield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="u">Open University of Cyprus, Nicosia, Cyprus</subfield>
    <subfield code="a">Nicos Isaak</subfield>
  </datafield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">A Data-Driven Metric of Hardness for WSC Sentences</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">user-rise-teaming-cyprus</subfield>
  </datafield>
  <datafield tag="536" ind1=" " ind2=" ">
    <subfield code="c">739578</subfield>
    <subfield code="a">Research Center on Interactive Media, Smart System and Emerging Technologies</subfield>
  </datafield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="u">http://creativecommons.org/licenses/by-nc-nd/4.0/legalcode</subfield>
    <subfield code="a">Creative Commons Attribution Non Commercial No Derivatives 4.0 International</subfield>
  </datafield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2">opendefinition.org</subfield>
  </datafield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">&lt;p&gt;The Winograd Schema Challenge (WSC) | the task of resolving pronouns in certain sentences where shallow parsing techniques seem not to be directly applicable | has been proposed as an alternative to the Turing Test. According to Levesque, having access to a large corpus of text would likely not help much in the WSC. Among a number of attempts to tackle this challenge, one particular approach has demonstrated the plausibility of using commonsense knowledge automatically acquired from raw text in English Wikipedia. Here, we present the results of a large-scale experiment that shows how the performance of that particular automated approach varies with the availability of training material. We compare the results of this experiment with two studies: one from the literature that investigates how adult native speakers tackle the WSC, and one that we design and undertake to investigate how teenager non-native speakers tackle the WSC. We nd that the performance of the automated approach correlates positively with the performance of humans, suggesting that the performance of the particular automated approach could be used as a metric of hardness for WSC instances.&lt;/p&gt;</subfield>
  </datafield>
  <datafield tag="773" ind1=" " ind2=" ">
    <subfield code="g">107-120</subfield>
    <subfield code="b">Easy Chair</subfield>
    <subfield code="t">GCAI-2018. 4th Global Conference on Artificial Intelligence, EPiC Series in Computing, Volume 55</subfield>
  </datafield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.29007/398z</subfield>
    <subfield code="2">doi</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">conferencepaper</subfield>
  </datafield>
</record>
18
17
views
downloads
Views 18
Downloads 17
Data volume 24.7 MB
Unique views 18
Unique downloads 17

Share

Cite as