Replication package for the research article entitled "From Sketches to Canonical Models: Benchmarking Vision–Language Models for UML Generation"

This repository contains the dataset and the output used for the paper entitled:  
"From Sketches to Canonical Models: Benchmarking Vision–Language Models for UML Generation"
Submitted to the International Conference on Evaluation and Assessment in Software Engineering (EASE), 2026 edition

Repository Structure

- informal_diagrams/ 

Contains the original informal diagrams extracted from academic papers. These were used as input for all tested language models.

- generated_diagrams/

Contains UML diagrams generated by four multimodal Large Language Models (LLMs):

• • ChatGPT 4o (OpenAI)
  • Claude 3.7 Sonnet (Anthropic)
  • Gemini 2.5 Pro (Google)
  • Llama 4 Maverick (Meta)

  Each file was generated from an informal diagram using a unified prompting approach and validated for PlantUML syntax.

- error_outputs/

Contains any failed or invalid output files that did not pass the PlantUML syntax validation step. These are useful for analyzing failure cases in UML generation.

- samples/ 

Contains some samples mentioned in the article, for a quick analysis.

Purpose

This dataset supports our evaluation of:
- Syntactic correctness of UML outputs
- Semantic alignment with the original informal input
- Model performance variation across five UML diagram types (Class, Activity, Use Case, State Machine, Sequence)