Data for Policy
Published July 30, 2020 | Version v1
Conference paper Open

Comparing Measures of Internet Censorship: Analyzing the Tradeoffs between Expert Analysis and Remote Measurement

Creators

  • 1. Millennium Challenge Corporation

Description

As the internet increasingly becomes a leading tool for exchanging information, governments around the world sometimes seek methods to restrict citizens’ access. Two of the most common methods for restricting the internet are shutting down internet access entirely and filtering specific content. We compare the tradeoffs between measuring these phenomena using expert analysis (as measured by Freedom House and V-Dem) and remote measurement with manual oversight (as measured by Access Now and the OpenNet Initiative (ONI)). We find that remote measurement with manual oversight is less likely to include false positives, while expert analysis is less likely to include false negatives.

Notes

Included with the paper are the raw data files from Freedom House, V-Dem, Access Now, and ONI used in the analysis and the Stata do file used to compile and analyze these data.

Files

114_Fletcher.pdf

Files (14.3 MB)

Name Size Download all
md5:24c327e2960a424cbd13b25a19395aba
253.7 kB Preview Download
md5:1ed3b264280d153fb08181448dc54541
45.8 kB Download
md5:1a22818579d4a0ba886bed7da9dbc8e2
13.4 MB Download
md5:eb4ef502c1a8a9eacdeac34b236389b5
15.3 kB Download
md5:bb897266565be9b8b762edb70319a7b8
16.5 kB Download
md5:2ea58848161802b0cd62596df1fde244
19.3 kB Download
md5:ad55739a50de4bc8121f8e153f57834c
19.2 kB Download
md5:9e399a5e3f24279ca4c9c2f4261a98f1
17.7 kB Download
md5:d43aa3fc49c4262f4098d02643a6a180
137.2 kB Preview Download
md5:45a61b46b55ad755c974812b87d04178
9.5 kB Preview Download
md5:775042efb273197359a4c204790aeb8a
349.7 kB Download

Additional details

References

  • Access Now (2017). Shutdown Tracker Optimization Project. https://www.accessnow.org/cms/assets/uploads/2017/09/How-to-view-the-Access-Now-Internet-Shutdown-Tracker-2017.pdf
  • Al-Saqaf, W. (2015). Internet Censorship Circumvention Tools: Escaping the Control of the Syrian Regime. Media and Communication. 4(1), 39-50.
  • Amnesty International (2017). Cuba's Internet paradox: How controlled and censored internet risks Cuba's achievements in education. Amnesty International. https://www.amnesty.org/en/latest/news/2017/08/cubas-internet-paradox-how-controlled-and-censored-internet-risks-cubas-achievements-in-education/
  • Anthonio, F. (2020). A Shutdown taints Togo's 2020 presidential election: what happened and what's next. Access Now. https://www.accessnow.org/a-shutdown-taints-togos-2020-presidential-elections-what-happened-and-whats-next/
  • Bischoff, P. (2020). Internet Censorship 2020: A Global Map of Internet Restrictions. Comparitech. https://www.comparitech.com/blog/vpn-privacy/internet-censorship-map/
  • Carsten, P. (2014). Google's Gmail blocked in China. Reuters. https://www.reuters.com/article/us-google-china/googles-gmail-blocked-in-china-idUSKBN0K70BD20141229
  • Censored Planet (2020). Censored Planet Raw Data. https://censoredplanet.org/data/raw
  • CIPESA (2019). State of Internet Freedom in Africa 2019. Collaboration on International ICT Policy for East and Southern Africa. https://cipesa.org/?wpfb_dl=307
  • Clark, J., Faris, R. Jones, R.H., (2017a). Analyzing Accessibility of Wikipedia Projects Around the World. Berkman Klein Center for Internet & Society Research Publication.
  • Clark, J., Faris, R., Morrison-Westphal, R., Noman H., Tilton, C., & Zittrain, J. (2017b). The Shifting Landscape of Global Internet Censorship. Berkman Klein Center for Internet & Society Research Publication.
  • Cohen-Almagor, R. (2013). Internet History. In Luppicini, R. (Eds.), Moral, Ethical, and Social Dilemmas in the Age of Technology: Theories and Practice (pp. 19-39). IGI Global. http://doi:10.4018/978-1-4666-2931-8.ch002
  • Coppedge, M., Gerring, J., Knutsen, C. H., Lindberg S. I., Teorell, J., Altman, D., Bernhard, M., Fish, M. S., Glynn, A., Hicken, A., Luhrmann, A., Marquardt, K. L., McMann, K., Paxton, P., Pemstein, D., Seim, B., Sigman, R., Skaaning, S., Staton, J., Wilson, S., Cornell, A., Alizada, N., Gastaldi, L., Gjerløw, H., Hindle, G., Ilchenko, N., Maxwell, L., Mechkova, V., Medzihorsky, J., von Römer, J. Sundström, A., Tzelgov, E., Wang, Y., Wig, T., & Ziblatt, D. (2020). V-Dem Dataset v10. Varieties of Democracy (V-Dem) Project. https://doi.org/10.23696/vdemds20.
  • Corduneanu-Huci, C., & Hamilton, A. (2018). Selective Control: The Political Economy of Censorship. World Bank Group.
  • Crandall, J. R., Crete-Nishihata, & M., Knockel, J. (2015). Forgive Us our SYNs: Technical and Ethical Considerations for Measuring Internet Filtering. NS Ethics '15: Proceedings of the 2015 ACM SIGCOMM Workshop on Ethics in Networked Systems Research.
  • Dahan, N. (2019). Internet, interrupted: How network cuts are used to quell dissent in the Middle East. Middle East Eye. https://www.middleeasteye.net/news/internet-interrupted-how-middle-east-countries-use-network-restrictions-clamp-down-dissent
  • Dalek, J., Deibert, R., McKune, S., Gill, P., Senft, A., & Noor, N. (2015). Information Controls during Military Operations The case of Yemen during the 2015 political and armed conflict. University of Toronto. https://citizenlab.ca/2015/10/information-controls-military-operations-yemen/
  • Engagemedia & Sinar Project (2018). Internet Censorship Monitoring: Duterte's Drug War. https://sinarproject.org/digital-rights/updates/internet-censorship-monitoring-dutertes-drug-war
  • Faris, R. & Villeneuve, N. (2008). Measuring Global Internet Filtering in R. Deibert, J. Palfrey, R. Rohozinski, & J. Zittrain, (Eds.), Access Denied: The Practice and Policy of Global Internet Filtering, Cambridge: MIT Press. http://opennet.net/sites/opennet.net/files/Deibert_02_Ch01_005-028.pdf
  • Farivar, C., & Blankstein, A. (2019). Feds take down the 'world's largest dark web child porn marketplace.' NBC News. https://www.nbcnews.com/news/crime-courts/feds-take-down-world-s-largest-dark-web-child-porn-n1066511
  • Farrell, H. (2012). The Consequences of the Internet for Politics. Annual Review of Political Science, 15, 35-52.
  • Frantz, E., Kendall-Taylor, A., & Wright, J. (2020). Digital Repression in Autocracies. The Varieties of Democracy Institute.
  • Freedom House (2019). Freedom on the Net Research Methodology. https://freedomhouse.org/reports/freedom-net/freedom-net-research-methodology
  • Freyburg, T. & Garbe, L. (2018). Blocking the Bottleneck: Internet Shutdowns and Ownership at Election Times in Sub-Saharan Africa. International Journal of Communication, 12(2018), 3896-3916.
  • Gold, H. (2019). Information war escalates as Venezuela tries to contain uprising. CNN Business. https://www.cnn.com/2019/05/01/media/media-venezuela-information-war/index.html
  • Gopaldas, R. (2019). Digital Dictatorship versus Digital Democracy in Africa. SAIIA Policy Insights 75.
  • Gueorguiev, D., Shao, L., & Crabtree, C. (2017). Blurring the Lines: Rethinking Censorship Under Autocracy. 10.13140/RG.2.2.29037.08160.
  • Hall, R. (2019). Lebanon blocks Grindr in latest attack on LGBT+ community. The Independent. https://www.independent.co.uk/news/world/middle-east/grindr-lebanon-ban-lgbt-rights-dating-app-gay-a8933556.html
  • Howard, P., Agarwal, S.D., & Hussain, M. M. (2011). When Do States Disconnect Their Digital Networks? Regime Responses to the Political Uses of Social Media. The Communication Review, 14, 216-232.
  • Hoang, P. N., Doreen, S., Polychronakis, M., (2019). Measuring I2P Censorship at a Global Scale. In Proceedings of the 9th USENIX Workshop on Free and Open Communications on the Internet.
  • Immigration and Customs Enforcement (2018). Over a million websites seized in global operation. ICE Newsroom. https://www.ice.gov/news/releases/over-million-websites-seized-global-operation
  • Karatnycky, A., Piano, A., Puddington, A. (2003). Freedom in the World: The Annual Survey of Political Rights & Civil Liberties. Freedom House.
  • Kravets, D. (2012). Uncle Sam: If It Ends in .Com It's .Seizable. Wired.com. https://www.wired.com/2012/03/feds-seize-foreign-sites/
  • Lakshmana, K.V. (2018). A Coward's Political Weapon: Troll armies go on settling scores. Common Cause, 37(4), 27-29.
  • Leberknight, C.S., Chiang, M., Poor, H. V., Wong, F. (2012). A Taxonomy of Internet Censorship and Anti-Censorship.
  • Maréchal, N. (2017). Networked Authoritarianism and the Geopolitics of Information: Understanding Russian Internet Policy. Media and Communication, 5(1), 29-41.
  • McColm, R.B., Finn, J., Payne, D.W., Ryan, J.E., Sussman, L.R., Zarycky, G. (1991). Freedom in the World Political Rights & Civil Liberties. Freedom House.
  • McDevitt, D. (2017). Rwanda censors critical, independent media in targeted fashion. Open Technology Fund. https://www.opentech.fund/news/new-report-investigates-internet-censorship-during-rwandas-2017-presidential-election/
  • Mechkova, V., Daniel P., Brigitte S., & Steven W. (2020). Digital Society Project Dataset v2. Varieties of Democracy (V-Dem) Project
  • Millennium Challenge Corporation (2019). Guide to the MCC Indicators for Fiscal Year 2020. Millennium Challenge Corporation. https://www.mcc.gov/resources/doc/guide-to-the-indicators-fy-2020
  • Narayanan A, & Zevenbergen B. (2015). No Encore for Encore? Ethical questions for web-based censorship measurement. Technology Science. http://techscience.org/a/2015121501.
  • NetBlocks (2020). Internet cut in Ethiopia amid unrest following killing of singer. NetBlocks Mapping Net Freedom. https://netblocks.org/reports/internet-cut-in-ethiopia-amid-unrest-following-killing-of-singer-pA25Z28b
  • Niaki, A.A., Cho, S., Weinberg Z., Hoang, N.P., Razaghpanah, A., Christin, N., Gill, P. (2020). ICLab: A Global, Longitudinal Internet Censorship Measurement Platform. Proceedings of the 41st IEEE Symposium on Security and Privacy.
  • OONI (2020). OONI Data Explorer. https://explorer.ooni.org/
  • Parks, L. & Thompson R. (2020). The Slow Shutdown: Information and Internet Regulation in Tanzania From 2010 to 2018 and Impacts on Online Content Creators. International Journal of Communication, 14(2020), 1-21.
  • Pearce, P., Ensafi, R., Li, F., Feamster, N. & Paxson, V. (2018). "Toward Continual Measurement of Global Network-Level Censorship. IEEE Security & Privacy, 16(1), 24-33, 10.1109/MSP.2018.1331018.
  • Pemstein, D., Marquardt, K. L., Tzelgov, E., Wang, Y., Medzihorsky, J., Krusell, J., Miri, F., & von Römer, J. (2020). The V-Dem Measurement Model: Latent Variable Analysis for Cross-National and Cross-Temporal Expert-Coded Data. V-Dem Working Paper No. 21. 5th edition. University of Gothenburg: Varieties of Democracy Institute.
  • Puyosa, I. & Chaguaceda, A (2017). Cinco regímenes políticos en Latinoamérica, libertad de internet y mecanismos de control. RETOS, 7(14), 11-37.
  • Rahimi, N. & Gupta, B. (2020). A Study of the Landscape of Internet Censorship and Anti-Censorship in Middle East. EPiC Series in Computing: Proceeding of the 35th International Conference on Computers and Their Applications. 69, 60-68.
  • Raman, R. S., Stoll, A., Dalek, J., Sarabi, A., Ramesh, R. Scott, W., & Ensafi, R. (2020). Measuring the Deployment of Network Censorship Filters at Global Scale. Network and Distributed System Security (NDSS) Symposium 2020.
  • Raveendran, N., & Leberknight, C.S. (2018). Internet Censorship and Economic Impacts: A Case Study of Internet Outages in India. Proceedings of the Twenty-fourth Americas Conference on Information Systems.
  • Reno v. American Civil Liberties Union, 521, U.S. 811 (1997).
  • Reporters Without Borders (2020). World Press Freedom Index. https://rsf.org/en/ranking
  • Roberts, H., Zuckerman, E., & Palfrey, J. (2011). 2011 Circumvention Tool Evaluation. The Berkman Center for Internet & Society Research Publication Series.
  • Rydzak, J.A. (2018). A Total Eclipse of the Net: The Dynamics of Network Shutdowns and Collective Action Responses. University of Arizona.
  • Rydzak, J. (2019). Of Blackouts and Bandhs: The Strategy and Structure of Disconnected Protest in India. https://ssrn.com/abstract=3330413
  • Sagir, M. & Varlioglu, S. (2020). Explaining the Relationship between Internet and Democracy in Partly Free Countries Using Machine Learning Models. IT Research Symposium 2020.
  • Sayadi, E. & Taye, B. (2020). #KeepItOn: As Yemen's war goes online, internet shutdowns and censorship are hurting Yemenis. https://www.accessnow.org/keepiton-as-yemens-war-goes-online-internet-shutdowns-and-censorship-are-hurting-yemenis/
  • Selva, M. (2019). Reaching for the off switch: Internet shutdowns are growing as nations seek to control public access to information. Index on Censorship, 48(3), 19-22.
  • Shahbaz, A. & Funk, A. (2020). Freedom on the Net 2019: The Crisis of Social Media. Freedom House.
  • Sisario, B (2010). U.S. Shuts Down Web Sites in Piracy Crackdown. New York Times.
  • SK, C. (2020). Those Unspoken Thoughts A study of censorship and median freedom in Manipur, India. Open Observatory of Network Interference.
  • Sfakianakis, A., Athanasopoulos, E., Ioannidis, S. (2011). CensMon: A Web Censorship Monitor. In Proceedings of USENIX FOCI 2011.
  • Subramanian, R. (2012). The Growth of Global Internet Censorship and Circumvention: A Survey. Communication of the International Information Management Association.
  • Sutterlin, E. (2020). Flipping the Kill-Switch: Why Governments Shut Down the Internet. Undergraduate Honors Theses. Paper 1493.
  • Stepanova, E. (2011). The Role of Information Communication Technologies the "Arab Spring." PONARS Eurasia.
  • Taye, B. (2020). Targeted, Cut Off, and Left in the Dark The #KeepItOn Report on internet shutdowns in 2019. Access Now. https://www.accessnow.org/keepiton/
  • USAID (2019). FY 2020 USAID Journey to Self-Reliance Country Roadmap Methodology Guide. https://selfreliance.usaid.gov/
  • USAID (2020). Digital Strategy 2020-2024. United States Agency for International Development. https://www.usaid.gov/usaid-digital-strategy
  • VanderSloot, B., McDonald, A., Scott, W., Halderman, J.A., & Ensafi, R. (2018). Quack: Scalable Remote Measurement of Application-Layer Censorship. In Proceedings of the 27th USENIX Security Symposium.
  • Wagner, B., Gollatz, K., Calderaro, A. (2013). Common Narrative – Divergent Agendas: The Internet and Human Rights in Foreign Policy. Proceedings of the 1st International Conference on Internet Science.
  • Weaver, N., Sommer, R., & Paxson, V. (2009) Detecting Forged TCP Reset Packets. International Computer Science Institute.
  • Weinberg, Z. (2018). Toward Automated Worldwide Monitoring of Network-level Censorship. Carnegie Mellon University.
  • West, D.M. (2016). Internet Shutdowns cost countries $2.4 billion last year. Center for Technology Innovation at Brookings.
  • Woodhams, S. & Migliano, S. (2020). The Global Cost of Internet Shutdowns in 2019. Top10VPN.com.
  • Yadav, T. K., & Chakravarty, S. (2018). Trends and patterns of internet censorship in India. Indraprastha Institute of Information Technology Delhi.
  • Zeleke, T. A. (2019). The Quandary of Cyber Governance in Ethiopia. Journal of Public Policy and Administration, 3(1), 1-7.