There is a newer version of this record available.

Software Open Access

Software for: "It is just a flu: Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations"

Kostantinos Papadamou; Savvas Zannettou; Jeremy Blackburn; Emiliano De Cristofaro; Gianluca Stringhini; Michael Sirivianos

DataCite XML Export

<?xml version='1.0' encoding='utf-8'?>
<resource xmlns:xsi="" xmlns="" xsi:schemaLocation="">
  <identifier identifierType="DOI">10.5281/zenodo.4580892</identifier>
      <creatorName>Kostantinos Papadamou</creatorName>
      <affiliation>Cyprus University of Technology</affiliation>
      <creatorName>Savvas Zannettou</creatorName>
      <affiliation>Max Planck Institute</affiliation>
      <creatorName>Jeremy Blackburn</creatorName>
      <affiliation>Binghamton University</affiliation>
      <creatorName>Emiliano De Cristofaro</creatorName>
      <affiliation>University College London</affiliation>
      <creatorName>Gianluca Stringhini</creatorName>
      <affiliation>Boston University</affiliation>
      <creatorName>Michael Sirivianos</creatorName>
      <affiliation>Cyprus University of Technology</affiliation>
    <title>Software for: "It is just a flu: Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations"</title>
    <subject>YouTube Videos</subject>
    <subject>YouTube's Recommendation Algorithm</subject>
    <subject>Pseudoscientific Misinformation</subject>
    <subject>Watch History</subject>
    <subject>Flat Earth</subject>
    <date dateType="Issued">2021-03-04</date>
  <resourceType resourceTypeGeneral="Software"/>
    <alternateIdentifier alternateIdentifierType="url"></alternateIdentifier>
    <relatedIdentifier relatedIdentifierType="DOI" relationType="IsSupplementTo" resourceTypeGeneral="Preprint">10.5281/zenodo.4484045</relatedIdentifier>
    <relatedIdentifier relatedIdentifierType="DOI" relationType="IsSupplementedBy" resourceTypeGeneral="Dataset">10.5281/zenodo.4558469</relatedIdentifier>
    <relatedIdentifier relatedIdentifierType="DOI" relationType="IsVersionOf">10.5281/zenodo.4580891</relatedIdentifier>
    <rights rightsURI="">Creative Commons Attribution 4.0 International</rights>
    <rights rightsURI="info:eu-repo/semantics/openAccess">Open Access</rights>
    <description descriptionType="Abstract">&lt;p&gt;&lt;strong&gt;Abstract:&lt;/strong&gt; The role played by YouTube&amp;#39;s recommendation algorithm in unwittingly promoting misinformation and conspiracy theories is not entirely understood. Yet, this can have dire real-world consequences, especially when pseudoscientific content is promoted to users at critical times, such as the COVID-19 pandemic. In this paper, we set out to characterize and detect pseudoscientific misinformation on YouTube. We collect 6.6K videos related to COVID-19, the Flat Earth theory, as well as the anti-vaccination and anti-mask movements. Using crowdsourcing, we annotate them as pseudoscience, legitimate science, or irrelevant and train a deep learning classifier to detect pseudoscientific videos with an accuracy of 0.79. We quantify user exposure to this content on various parts of the platform and how this exposure changes based on the user&amp;#39;s watch history. We find that YouTube suggests more pseudoscientific content regarding traditional pseudoscientific topics (e.g., flat earth, anti-vaccination) than for emerging ones (like COVID-19). At the same time, these recommendations are more common on the search results page than on a user&amp;#39;s homepage or when actively watching videos. Finally, we shed light on how a user&amp;#39;s watch history substantially affects the type of recommended videos.&lt;/p&gt;

&lt;p&gt;Preprint available&amp;nbsp;&lt;a href=""&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What do we offer in this software?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;We make&amp;nbsp;publicly available to the research community, as well as the open-source community, the following tools, and libraries:&lt;/p&gt;

	&lt;p&gt;The&amp;nbsp;codebase&amp;nbsp;of a Deep Learning Classifier for pseudoscientific videos detection on YouTube, and&amp;nbsp;examples&amp;nbsp;on how to train and test it;&lt;/p&gt;
	&lt;p&gt;A&amp;nbsp;library&amp;nbsp;that simplifies the usage of the trained classifier and implements all the required tasks for the classification of YouTube videos;&lt;/p&gt;
	&lt;p&gt;An&amp;nbsp;open-source library&amp;nbsp;that provides a unified framework for assessing the effects of personalization on YouTube video recommendations in multiple parts of the platform: a) the homepage; b) the search results page; and c) the video recommendations section (recommendations when watching videos).&lt;/p&gt;

&lt;p&gt;The codebase is also available on &lt;a href=""&gt;GitHub&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you make use of any modules available in this codebase in your work, please cite the following paper:&lt;/p&gt;

  title={"It is just a flu": Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations},
  author={Papadamou, Kostantinos and Zannettou, Savvas and Blackburn, Jeremy and De Cristofaro, Emiliano and Stringhini, Gianluca and Sirivianos, Michael},
  journal={arXiv preprint arXiv:2010.11638},

    <description descriptionType="Other">Acknowledgments: This project has received funding from the European Union's Horizon 2020 Research and Innovation program under the Marie Skłodowska-Curie ENCASE project (Grant Agreement No. 691025) and from the National Science Foundation under grant CNS-1942610.</description>
      <funderName>European Commission</funderName>
      <funderIdentifier funderIdentifierType="Crossref Funder ID">10.13039/100010661</funderIdentifier>
      <awardNumber awardURI="info:eu-repo/grantAgreement/EC/H2020/830927/">830927</awardNumber>
      <awardTitle>Cyber security cOmpeteNce fOr Research anD Innovation</awardTitle>
All versions This version
Views 7720
Downloads 30
Data volume 948.7 kB0 Bytes
Unique views 6019
Unique downloads 30


Cite as