Specification Enrichment v28 (EISEJ submission) — Reference implementation and empirical evaluation
Authors/Creators
Description
v28 snapshot of the Specification Enrichment reference implementation + empirical evaluation, prepared for submission to e-Informatica Software Engineering Journal. Supersedes v1.0.0 / v2-precommit. The v28 manuscript restructures the empirical section so the prospectively-pinned visitor-kiosk multi-seed (§4.2) leads as the primary test of the structural coverage-expansion claim, with the retrospectively-seeded TodoMVC results demoted to supplementary illustration.
Contains: ~1,200 lines of TypeScript reference implementation (enricher, meta-prompt, pipeline, judge, harness); offline stub provider + live anthropic + ollama provider adapters; LLM-as-judge evaluator with position-bias guard; multi-seed harness (count-matched, enhanced, threshold sweep, visitor-kiosk); audit protocol; all result JSONs under results/; v28 manuscript and EISEJ cover letter.
Headline numbers (visitor-kiosk multi-seed, prospectively-pinned, claude-sonnet-4-6 at temperature 0, 3 seeds): Enriched-16 produces a coverage expansion in some implicit-failure-mode category that no count-matched or category-diversity-instructed baseline reaches — 3/3 seeds. Enriched-16 accepted-as-is rate is exactly 81.3% in every seed (range 0.0 pp); both baselines move 18.7 pp seed-to-seed. The enrichment stage itself is essentially deterministic at temperature 0 on this design.
Pre-registration: N/A for this artefact; the contribution is a pipeline pattern + reference implementation rather than a pre-registered empirical study.
Files
Files
(829.7 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:cb2dbd559d79a6ef7e0b5e10495ec1b5
|
829.7 kB | Download |
Additional details
Related works
- Is version of
- 10.5281/zenodo.20571863 (DOI)