Alloy4Fun Semantic Entries

This dataset groups together entries from the Alloy4Fun dataset according to their semantic and syntactic equivalence. Each entry is a student attempt to formalize a requirement (i.e., a challenge). Each challenge is a command that tests the equivalence of a student predicate against a hidden oracle. This grouping reduces ~51.000 entries into ~7.000 semantically distinct specifications (and ~17.000 syntactically distinct).

Semantic equivalence is tested with the Alloy Analyzer (with a default scope of 3 for top-level signatures).

Syntactic equivalence is tested after a simple normalization (variable anonymization and operator commutativity).

Each file original_cmd.json contains the information regarding a challenge represented by command labeled cmd of an Alloy4Fun exercise whose model has id original, namely:

• original: the ID of the original model in the Alloy4Fun database 
• cmd: the challenge command label
• pred: the challenge predicate label
• entry_count: the total number of entries for the challenge
• semantic_count: the number of semantically distinct entries
• syntactic_count: the number of syntactically distinct entries
• scope: the scope used to test equivalence
• groups: the actual semantic groups, containing:
  • entry_count: the total number of entries in the semantic group
  • syntactic_count: the number of syntactically distinct entries
  • correct: whether a correct solution to the challenge
  • elems: the actual syntactic groups, containing:
    • entry_count: the total number of entries in the syntactic group
    • code: a representative code submission

The following Alloy4Fun exercises where considered: