June 14, 2020 Conference paper Open Access

Schöch, Christof; van Dalen-Oskam, Karin; Antoniak, Maria; Jannidis, Fotis; Mimno, David

MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="http://www.loc.gov/MARC21/slim">
  <leader>00000nam##2200000uu#4500</leader>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Digital Humanities</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Digital Humanities Conference 2020 (DH2020)</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Replication</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Computational Literary Studies</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Panel</subfield>
  </datafield>
  <controlfield tag="005">20200614101822.0</controlfield>
  <controlfield tag="001">3893428</controlfield>
  <datafield tag="711" ind1=" " ind2=" ">
    <subfield code="d">July 20-25, 2020</subfield>
    <subfield code="g">DH2020</subfield>
    <subfield code="a">Digital Humanities Conference 2020</subfield>
    <subfield code="c">Ottawa, Canada</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Huyghens ING and University of Amsterdam, The Netherlands</subfield>
    <subfield code="0">(orcid)0000-0001-7697-4512</subfield>
    <subfield code="a">van Dalen-Oskam, Karin</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Cornell University, USA</subfield>
    <subfield code="a">Antoniak, Maria</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">University of Würzburg, Germany</subfield>
    <subfield code="0">(orcid)0000-0001-6944-6113</subfield>
    <subfield code="a">Jannidis, Fotis</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Cornell University, USA</subfield>
    <subfield code="0">(orcid)0000-0001-7510-9404</subfield>
    <subfield code="a">Mimno, David</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">4675</subfield>
    <subfield code="z">md5:ad6a7b9f7ce3c09c952f1a3239ea4b60</subfield>
    <subfield code="u">https://zenodo.org/record/3893428/files/replication-and-computational-literary-studies.bibliography.zip</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">1734142</subfield>
    <subfield code="z">md5:58558685eeb7f84a1dc799bc422753ed</subfield>
    <subfield code="u">https://zenodo.org/record/3893428/files/replication-and-computational-literary-studies.docx</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">1727779</subfield>
    <subfield code="z">md5:9d664bd9e1712719b784059a997128a1</subfield>
    <subfield code="u">https://zenodo.org/record/3893428/files/replication-and-computational-literary-studies.fidus</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">2349676</subfield>
    <subfield code="z">md5:7e7b1bd7799a2cfb610dc23224dff97b</subfield>
    <subfield code="u">https://zenodo.org/record/3893428/files/replication-and-computational-literary-studies-figure-1.jpg</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">1941614</subfield>
    <subfield code="z">md5:2a5e9bbf817f816cba040a147a820b98</subfield>
    <subfield code="u">https://zenodo.org/record/3893428/files/replication-and-computational-literary-studies.html.zip</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">1719743</subfield>
    <subfield code="z">md5:5e9faa89d0ed3477e29071fb966e0115</subfield>
    <subfield code="u">https://zenodo.org/record/3893428/files/replication-and-computational-literary-studies.jats.zip</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">2969068</subfield>
    <subfield code="z">md5:360bd1fdfa62500195794b022cfc9027</subfield>
    <subfield code="u">https://zenodo.org/record/3893428/files/replication-and-computational-literary-studies.pdf</subfield>
  </datafield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="y">Conference website</subfield>
    <subfield code="u">https://dh2020.adho.org/</subfield>
  </datafield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2020-06-14</subfield>
  </datafield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">openaire</subfield>
    <subfield code="o">oai:zenodo.org:3893428</subfield>
  </datafield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="u">Trier University, Germany</subfield>
    <subfield code="0">(orcid)0000-0002-4557-2753</subfield>
    <subfield code="a">Schöch, Christof</subfield>
  </datafield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">Replication and Computational Literary Studies</subfield>
  </datafield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="u">https://creativecommons.org/licenses/by/4.0/legalcode</subfield>
    <subfield code="a">Creative Commons Attribution 4.0 International</subfield>
  </datafield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2">opendefinition.org</subfield>
  </datafield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">&lt;p&gt;The &amp;quot;replication crisis&amp;quot; that has been raging in fields like Psychology (Open Science Collaboration 2015) or Medicine (Ioannidis 2005) for years has recently reached the field of Artificial Intelligence (Barber 2019). One of the key conferences in the field, NeurIPS, has reacted by appointing &amp;#39;reproducibility chairs&amp;#39; in their organizing committee&amp;nbsp;&lt;a href="#fn-14m11"&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;. In the Digital Humanities, and particularly in Computational Literary Studies (CLS), there is an increasing awareness of the crucial role played by replication in evidence-based research. Relevant disciplinary developments include the increased importance of evaluation in text analysis and the increased interest in making research transparent through publicly accessible data and code (open source, open data). Specific impulses include Geoffrey Rockwell and St&amp;eacute;fan Sinclair&amp;#39;s re-enactments of pre-digital studies (Sinclair and Rockwell 2015) or the recent replication study by Nan Z. Da (Da 2019). The paper has been met by an avalanche of responses that pushed back several of its key claims, including its rather sweeping condemnation of the replicated papers. However, an important point got buried in the process: that replication is indeed a valuable goal and practice.&amp;nbsp;&lt;a href="#fn-2b52z"&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt; As stated in the Open Science Collaboration paper: &amp;quot;Replication can increase certainty when findings are reproduced and promote innovation when they are not&amp;quot; (Open Science Collaboration 2015, 943).&lt;/p&gt;

&lt;p&gt;As a consequence, the panel aims to raise a number of issues regarding the place, types, challenges and affordances, both on a practical and on a policy or community level, of replication in CLS. Several impulse papers will address key aspects of the issue: recent experience with attempts at replication of specific papers; policies dealing with replication in fields with more experience in the issue; conceptual and terminological clarification with regard to replication studies; and proposals for a way forward with replication as a community task or a policy issue.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bibliography&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Barber, Gregory. 2019. &amp;ldquo;Artificial Intelligence Confronts a &amp;lsquo;Reproducibility&amp;rsquo; Crisis.&amp;rdquo; &lt;em&gt;Wired&lt;/em&gt;, 2019. https://www.wired.com/story/artificial-intelligence-confronts-reproducibility-crisis/.&lt;/p&gt;

&lt;p&gt;Da, Nan Z. 2019. &amp;ldquo;The Computational Case Against Computational Literary Studies.&amp;rdquo; &lt;em&gt;Critical Inquiry&lt;/em&gt; 45 (3): 601&amp;ndash;39. https://www.journals.uchicago.edu/doi/abs/10.1086/702594.&lt;/p&gt;

&lt;p&gt;Ioannidis, J.A. 2005. &amp;ldquo;Contradicted and Initially Stronger Effects in Highly Cited Clinical Research.&amp;rdquo; &lt;em&gt;JAMA&lt;/em&gt;, no. 294/2: 218&amp;ndash;28. https://doi.org/https://doi:10.1001/jama.294.2.218.&lt;/p&gt;

&lt;p&gt;Open Science Collaboration. 2015. &amp;ldquo;Estimating the Reproducibility of Psychological Science.&amp;rdquo; &lt;em&gt;Science&lt;/em&gt;, no. 349 (6251). https://science.sciencemag.org/content/349/6251/aac4716.&lt;/p&gt;

&lt;p&gt;Sinclair, St&amp;eacute;fan, and Geoffrey Rockwell. 2015. &amp;ldquo;Epistemologica.&amp;rdquo; 2015. https://github.com/sgsinclair/epistemologica.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;1. See: https://nips.cc/Conferences/2019/Committees.&lt;/p&gt;

&lt;p&gt;2. For a selection of responses, see &lt;a href="https://culturalanalytics.org/articles?tag=replication"&gt;relevant contributions to &lt;/a&gt;&lt;em&gt;&lt;a href="https://culturalanalytics.org/articles?tag=replication"&gt;Cultural Analytics&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;</subfield>
  </datafield>
  <datafield tag="773" ind1=" " ind2=" ">
    <subfield code="n">doi</subfield>
    <subfield code="i">isVersionOf</subfield>
    <subfield code="a">10.5281/zenodo.3893427</subfield>
  </datafield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.5281/zenodo.3893428</subfield>
    <subfield code="2">doi</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">conferencepaper</subfield>
  </datafield>
</record>