COFFAIL: A Dataset of Successful and Anomalous Robot Skill Executions in the Context of Coffee Preparation

Dataset description

This dataset includes both successful and anomalous execution episodes of the Jessie robot (https://www.h-brs.de/en/a2s/robots) performing seven different skills in the context of preparing coffee. All episodes were collected by demonstration (either by executing hand-coded scripts that produce the desired behaviour or using kinesthetic teaching).

The included skills are:

1. Picking up a cup from a countertop
2. Moving a cup by pushing
3. Pouring from one cup into another (note that, for safety reasons, liquid was not used in these trials)
4. Placing a cup on a countertop
5. Picking up a teaspoon
6. Stirring
7. Placing a spoon in a sink

The dataset includes robot observations and actions from the execution episodes:

• Observations:
  • RGB images (of size 266x200 pixels) from a head camera and wrist cameras attached to each arm (for skills performed by a single arm, wrist images of the stationary arm are not collected)
  • Joint states (measured positions, velocities, and efforts)
  • End effector poses (with respect to the arm's base)
• Actions in the form of end-effector delta motions (x, y, z position changes and roll, pitch, yaw orientation changes)

Important note for correct action interpretation: The actions are given with respect to each manipulator's base frame, such that, considering the constraints imposed by the arm arrangement on Jessie, forward arm motion is actually along the negative x-axis of the manipulator frames (and not along the positive x, as would be expected); the sign of the y motions is also inverted accordingly.

Data format

The data is saved in a RLDS-like format, but in the form of MongoDB databases. In particular, the dataset includes 15 separate database dumps:

• Seven dumps corresponding to successful execution episodes (in total, 79 successful episodes over all skills)
• Seven dumps corresponding to episodes with execution anomalies (in total, 48 anomalous episodes over all skills)
• One dump of a database with anomaly annotations (anomalies are annotated using intervals, namely the annotations include entries for when each anomaly starts and when it ends, together with a comment on what the anomaly was)

In the databases corresponding to execution episodes:

• each collection corresponds to a separate episode
• each document corresponds to an execution step

Note that the recordings include idle states before and after each skill execution; these should be filtered out if the data is used for policy learning.

Usage examples

A Python package for working with the COFFAIL dataset is provided in the following repository: https://github.com/KEROL-project/coffail-utils/

Citing

If you find COFFAIL useful, please cite the following paper:

@inproceedings{coffail2026,
    author = {Mitrevski, Alex and Salunke, Ayush},
    title = {{COFFAIL: A Dataset of Successful and Anomalous Robot Skill Executions in the Context of Coffee Preparation}},
    booktitle = {2nd German Robotics Conference (GRC)},
    year = {2026},
    url = {https://arxiv.org/abs/2604.18236}
}