AI-CRP: An AI-Cowriting Research Platform

This repository contains source code for an AI-Cowriting Research Platform (AI-CRP) that supports empirical studies of writers' interactions with large language models as shown in Williams-Ceci, Sterling, et al. "Bias in AI Autocomplete Suggestions Leads to Attitude Shift on Societal Issues." and Jakesch, Maurice, et al. "Co-writing with opinionated language models affects users’ views."

The repository consists of HTML and JS files for a writing app with an AI writing assistant, a set of serverless functions that generate writing suggestions and store participants' interaction data in the backend, as well as code snippets for integrating the experimental app into a Qualtrics survey. We also include instructions for serving the app through Firebase below.

Setting up the dependencies, the firebase project, and configuring the platform takes about 30 minutes. This software has been tested in MacOS 14.6.1 with npm v. 10.8.3, Node v. 22.9.0, and Firebase v. 13.20.2). For questions, contact Maurice Jakesch or one of the study authors.

An overview of the software architecture is provided in `architecture.png`. This software does not rely on preexisting data for the suggestions; instead, it generates text suggestions based on what users write. As a participant writes a their text, the text is sent to GPT with a request for completion on the backend. GPT generates a completion that takes into account instructions provided by the researcher. The suggestions are logged in the backend and delivered to the interface frontend, where they are shown dynamically. After the participant has written their response, the app records variables such as the total number of suggestions accepted by the user (e.g. “acceptedSuggestions“), stores them in a database, and returns them to the Qualtrics parent window. 

You should now see "Your Firebase project is ready!" with the project information. There will also be an warning about Cloud Firestore, which we will address next: To do so, view the new Google Platform project in the Firebase Console. Here, select Build > Firestore Database > Create database > Create to set up the Firestore database. Also, upgrade the project from the free Spark plan to the paid Blaze plan (bottom left or under Settings > Usage and billing) to use the Cloud Functions in the deployed version of the app (not needed for local testing).

Finally, return to your local repository, navigate to the functions directory, and run npm install to install the dependencies listed in functions/package.json. Open the file functions/index.js and insert the your OpenAI secret key in the authorization header at the top with the word "Bearer" before it. The line of code should look like this: "Authorization": "Bearer [YOUR API KEY]".

Finally, open the local server URL in your browser. If everything is set up correctly, you will see the writing interface and the assistant will start generating writing suggestions automatically, as shown in the screenshot.png file. If not, check your browser JS console and the output of your CLI command to see what's going on. Likely, you have not set the correct endpoints in /public/resources/assistant.js or you are missing the OpenAI authentication bearer in functions/index.js.

To show no suggestions, set group to "control". The remaining parameters can be left empty and are populated by the script when users submit their writing. The text participants wrote will be in the essay field, with text accepted from the AI underlined. The other parameters will also be populated by the script. For a more detailed analysis of interactions, take a look at the data in the Firestore, where every single user action and suggestions are saved and time-stamped. Note that the snippet will also hide the survey continuation button, as survey continuation is triggered directly by the web app.