WE1S Topic Model Interpretation Protocol

The WhatEvery1Says (WE1S) Project developed a topic-model interpretation protocol that declares standard instructions and observation steps for researchers using topic models — a transparent, documented, and understandable process for the interaction between machine learning and human interpretation. The goal of this protocol is not to assert a definitive topic-model interpretation process (because this will be different for different projects and materials), but to publish a paradigm that can be adapted, improved, and varied by others.

Our Topic Model Interpretation Protocol consists of a modular set of survey-like questionnaires with instructions, required observation methods and waypoints, and reporting methods (utilizing the principles of "grounded theory" iterative note-taking). Used in sequences, these protocols step a researcher through an interpretation and documentation process that results in a set of capturable notes that can be used to produce research reports. (See the included flowchart of modules in the protocol.)

We make our Interpretation Protocol available "as is" in its original Qualtrics survey formats (exported as QSF files for others who can import them into Qualtrics) as well as in adapted Word .docx formats (using customized versions of Word's "document properties" in each file to re-create the editable, repeated "running notes" in the original surveys). These files include instructions and references that are specific to the WE1S project and its materials. We hope that they can be forked, evolved, and adapted by other projects to evolve a consensus practice of open, reproducible digital humanities research.