Artifact for "How does ROS-based Robot Software Evolve? An Empirical Study on Architectural Evolution"

Artifact: How does ROS-based Robot Software Evolve? An Empirical Study on Architectural Evolution

This artifact accompanies the paper "How does ROS-based Robot Software Evolve? An Empirical Study on Architectural Evolution." It contains the differential, cross-language architectural recovery tool, the mined dataset of extracted architectures and their evolution, and the analysis scripts that produce every table and figure in the paper.

Abstract

The Robot Operating System (ROS) is the de facto framework for building open-source robotic software systems. Developers architect their systems from reusable components, connecting them via configuration files and source code. These architectures evolve as developers add, remove, and modify components, connections, and configurations, and such changes can lead to architectural misconfigurations. Existing specification and analysis tools focus on snapshots in time, without considering how architectures evolve. To understand this evolution, we develop a differential, cross-language architectural recovery tool that extracts components, connections, and configurations from source code, and apply it to 294 repositories, comprising 1,728 packages across 9,914 releases, spanning ROS 1 and ROS 2 in C/C++, Python, and XML. We find that evolution diverges across levels: repositories remain addition-dominant while mature packages consolidate by removing connections and configurations, and changes co-occur across elements (e.g., component removals are associated with the removal of their connections and configurations).

What is included

• Recovery tool (src/evolution/) and its command-line interface (evolution-analysis), which detects changes to architectural elements directly from source-code diffs using 202 Tree-sitter rules (src/evolution/recovery/rules/).
• Mined dataset (evaluation/jsons/): one summary.json per repository (294 files), containing the recovered architecture and per-release changes.
• Analysis scripts (evaluation/): the pipeline and per-figure scripts that regenerate the paper's tables and figures.
• Artifact report (2025-evolution-artifact.pdf).

Installation

Prerequisites:

• Python 3.10, 3.11, 3.12, or 3.13.
• uv (Python package and environment manager).
• GitHub CLI (gh), installed and authenticated (gh auth login) — required for the mining steps only; the tool fetches each repository's releases through gh. Reproducing the tables and figures from the included dataset does not require it.

# Install dependencies from the pinned lockfile
uv sync --python 3.13

The mined dataset is included under evaluation/jsons/, with one summary.json per repository. Confirm it is present (should report 294):

find evaluation/jsons -name summary.json | wc -l

Usage

These steps validate the recovery tool on a single repository, then reproduce the paper's tables and figures from the included dataset.

Mining a single repository

Requires an authenticated GitHub CLI. Releases are fetched with gh, so run gh auth login first. Without it, the tool logs gh CLI not found (or an authentication error) and reports no releases.

git clone https://github.com/mgonzs13/yolo_ros /tmp/yolo_ros
uv run evolution-analysis --repo /tmp/yolo_ros --output output/

This writes output/yolo_ros/summary.json, recording the recovered architecture and the architectural changes per package for every release. Add --verbose for debug logging.

Reproducing the tables

cd evaluation
uv run python analysis_pipeline.py

This writes three files under evaluation/output/:

• data.tex — dataset macros (294 repositories, 1,728 packages, 9,914 releases, 238,847 changes).
• pkg_distro_distribution.tex — Table 1: packages by language, ROS version, and release status.
• subcategory_distribution.tex — Table 2: changes by sub-category, ROS version, and language.

Reproducing the figures

Run each script from the evaluation/ directory; PDFs are saved under evaluation/output/figures/.

cd evaluation
uv run python paper-repo-package-evolution-comparison.py   # Figure 3 (RQ1)
uv run python paper-changes-associations.py                # Figure 4 (RQ2)
uv run python paper-comparison-ros-distros.py              # Figure 5 (RQ3)
uv run python paper-components-systems-evolution.py        # Figure 6 (RQ3)

Reproducing the full dataset (optional)

Re-running the full mine clones all 294 repositories and diffs every release pair across 9,914 releases, and requires an authenticated gh. It is compute-heavy: in our environment it took about 3 days on 52 cores (~156 core-days), which extrapolates linearly to roughly 156 days (~5 months) on a single core. The output is exactly the dataset already included under evaluation/jsons/, so this step is only needed to rebuild it from scratch.

cd evaluation
uv run python dataset/mine_evol.py

Reusing on your own repositories

The recovery tool works on any local clone, not just the study dataset. To analyze your own repositories, clone each one and run evolution-analysis, collecting the summaries in an output directory of your choice:

for url in \
  https://github.com/your-org/your_ros_pkg \
  https://github.com/another-org/another_pkg; do
  name=$(basename "$url")
  git clone "$url" "/tmp/$name"
  uv run evolution-analysis --repo "/tmp/$name" --output my_summaries/
done

Each run writes a self-describing my_summaries/<repo>/summary.json (releases, packages, recovered architecture, and per-release changes) that you can analyze directly. To reuse the paper's analysis over your own set, place the summaries under evaluation/jsons/<owner>/<repo>/summary.json (the layout the pipeline expects) and re-run analysis_pipeline.py and the figure scripts. To extend detection to new APIs or languages, add rules to the per-language files under src/evolution/recovery/rules/; each rule pairs a Tree-sitter query with a category and change type, so no changes to the recovery engine are needed.

License

Apache-2.0. See the LICENSE file for details.

Citation

Please cite the accompanying paper. This artifact is archived on Zenodo with DOI 10.5281/zenodo.20691874.