Published August 8, 2024 | Version v3
Software Open

Replication package for "What You Need is What You Get: Theory of Mind for an LLM-Based Code Comprehension Assistant"

Creators

  • 1. ROR icon Radboud University Nijmegen

Description

ToMMY - web client and replication package

This is the replication package for our study "What You Need is What You Get: Theory of Mind for an LLM-Based Code Comprehension Assistant", accepted at the NIER track of the International Conference on Software Maintenance and Evolution (ICSME), 2024.

Files

The package contains the following items:
  • /analysis
    • data: processed dataset
    • clean-data.ipynb: data cleaning pipeline
    • simulate-conversations.ipynb: light-weight evaluation of ToMMY
    • user-study.ipynb: data analysis pipeline
  • /screenshots: screenshots of the different screens encountered in the web client
  •  /task
    • questions: quiz questions and answers for the code snippets
    • snippets: code snippets and licenses
    • design.ipynb: quiz question and answer creation pipeline
    • design.json: config file detailing the tasks, models (agents), and participant groups
  • /web
    • api: backend of the web client
    • client: frontend of the web client
    • shared: type definitions shared between the backend/frontend

 

The most noteworthy files are:
  • /README.md: a description of the package, and an installation guide fo the web client
  • /analysis/data: our dataset
  • /analysis/simulate-conversations.ipynb: contains a Python implementation of ToMMY and the control approach
  • /web/api/src/lib/agent.ts: contains a Javascript implementation of ToMMY and the control approach

Files

ToMMY.zip

Files (5.4 MB)

Name Size Download all
md5:c4bd0f8b97156bda3a969b49893e11be
5.4 MB Preview Download