| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | AC | AD | AE | AF | AG | AH | AI | AJ | AK | AL | AM | AN | AO | AP | AQ | AR | AS | AT | AU | AV | AW | AX | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | GPT4 | Le Chat Mistral | Gemini Advanced | LLama 3 | ||||||||||||||||||||||||||||||||||||||||||||||
2 | Detect | Fix? | Explanation | Detect | Fix? | Explanation | Detect | Fix? | Explanation | Detect | Fix? | Explanation | GPT4 | Le Chat Mistral | Gemini Advanced | LLama 3 | ||||||||||||||||||||||||||||||||||
3 | id | Language | Macros | #Prod. | LOC | Type | Error | #CE | Code Element | Type of error | Location | Code Element | Type of error | Location | Code Element | Type of error | Location | Code Element | Type of error | Location | Error | Type | #NC | #Detect | Fix | Explanation | #Detect | Fix | Explanation | #Detect | Fix | Explanation | #Detect | Fix | Explanation | LLM | Detected Product Lines | Detected Products | Fix | Explanation | ||||||||||
4 | 1 | C | 2 | 4 | 9 | Semantic | Type not declared | 1 | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 0 | No | No | No | No | 0 | No | No | No | No | Type not declared | Semantic | 1 | 1 | Yes | Yes | 0 | No | No | 0 | No | No | 0 | No | No | GPT4 | 29/30 | 38/40 | 12/30 | 26/29 | ||
5 | 2 | C | 2 | 4 | 16 | Semantic | Function not defined; Function redefinition | 2 | 2 | Partially | Yes | Yes | Yes | 1 | No | Yes | Partially | Yes | 2 | No | Yes | Yes | Yes | 1 | No | Yes | Partially | Yes | Function not defined; Function redefinition | Semantic | 2 | 2 | Partially | Yes | 1 | No | No | 2 | No | Yes | 1 | No | No | Le Chat Mistral | 26/30 | 31/40 | 9/30 | 18/26 | ||
6 | 3 | C | 1 | 2 | 10 | Syntax | Mismatching quotes | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Mismatching quotes | Syntax | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | Gemini Advanced | 20/30 | 24/40 | 13/30 | 13/20 | ||
7 | 4 | C | 1 | 2 | 11 | Semantic | Function not defined | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | 1 | No | Yes | LLama 3 | 24/30 | 27/40 | 7/30 | 20/24 | ||
8 | 5 | C | 1 | 2 | 8 | Semantic | Variable not declared | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Variable not declared | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | Yes | Yes | |||||||
9 | 6 | C | 1 | 2 | 13 | Syntax | Mismatching brackets | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | No | No | Yes | 0 | No | No | No | No | 0 | No | No | No | No | Mismatching brackets | Syntax | 1 | 1 | Yes | Yes | 1 | No | No | 0 | No | No | 0 | No | No | |||||||
10 | 7 | C | 1 | 2 | 9 | Syntax | Missing semicolon | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 1 | Yes | Yes | Yes | Yes | Missing semicolon | Syntax | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 0 | No | No | 1 | Yes | Yes | |||||||
11 | 8 | C | 2 | 4 | 13 | Semantic | Type not declared | 1 | 1 | No | Yes | Yes | No | 0 | No | No | No | No | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | Type not declared | Semantic | 1 | 1 | No | No | 0 | No | No | 0 | No | No | 1 | No | Yes | |||||||
12 | 9 | C | 2 | 4 | 16 | Semantic | Variable not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Variable not declared | Semantic | 1 | 1 | No | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | No | Yes | |||||||
13 | 10 | C | 2 | 4 | 22 | Semantic | Function signature mismatch (2x) | 2 | 2 | No | Yes | Yes | Yes | 0 | No | No | No | No | 1 | Yes | Partially | Partially | Partially | 2 | No | Yes | Yes | Yes | Function signature mismatch (2x) | Semantic | 2 | 2 | No | Yes | 0 | No | No | 1 | Yes | No | 2 | No | Yes | |||||||
14 | 11 | C | 2 | 4 | 23 | Semantic | Function signature mismatch (2x); Variable not declared | 3 | 3 | No | Yes | Yes | Yes | 0 | Yes | No | No | No | 1 | No | Partially | Partially | Partially | 3 | No | Yes | Yes | Yes | Function signature mismatch (2x); Variable not declared | Semantic | 3 | 3 | No | Yes | 0 | Yes | No | 1 | No | No | 3 | No | Yes | |||||||
15 | 12 | C | 1 | 2 | 17 | Semantic | Label not declared | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | No | No | No | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | Label not declared | Semantic | 1 | 1 | Yes | Yes | 1 | No | No | 0 | No | No | 1 | No | Yes | |||||||
16 | 13 | C | 2 | 4 | 11 | Semantic | Type not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | No | Yes | No | Yes | 1 | No | Yes | No | Yes | 1 | No | Yes | Yes | Yes | Type not declared | Semantic | 1 | 1 | No | Yes | 1 | No | No | 1 | No | No | 1 | No | Yes | |||||||
17 | 14 | C | 1 | 2 | 5 | Syntax | Missing struct keyword | 1 | 1 | No | Yes | No | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | No | Yes | 1 | No | Yes | Yes | Yes | Missing struct keyword | Syntax | 1 | 1 | No | No | 1 | Yes | Yes | 1 | Yes | No | 1 | No | Yes | |||||||
18 | 15 | C | 1 | 2 | 10 | Semantic | Label not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | Label not declared | Semantic | 1 | 1 | No | Yes | 1 | Yes | Yes | 0 | No | No | 1 | No | Yes | |||||||
19 | 16 | C | 1 | 2 | 4 | Syntax | Invalid return type | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Partially | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Invalid return type | Syntax | 1 | 1 | Yes | Yes | 1 | Partially | Yes | 1 | Yes | Yes | 1 | No | Yes | |||||||
20 | 17 | C | 1 | 2 | 14 | Semantic | Continue statement outside a loop | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Continue statement outside a loop | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | No | Yes | |||||||
21 | 18 | C | 1 | 2 | 12 | Semantic | Address of bitfield requested | 1 | 1 | No | Yes | Yes | Yes | 1 | No | No | No | No | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Address of bitfield requested | Semantic | 1 | 1 | No | Yes | 1 | No | No | 1 | Yes | Yes | 1 | No | Yes | |||||||
22 | 19 | C | 3 | 5 | 32 | Semantic | Function not defined; Variable not declared | 2 | 1 | No | Partially | Partially | Partially | 2 | No | Yes | Yes | Yes | 1 | No | Partially | Partially | Partially | 1 | No | Partially | Partially | Yes | Function not defined; Variable not declared | Semantic | 2 | 1 | No | No | 2 | No | Yes | 1 | No | No | 1 | No | No | |||||||
23 | 20 | C | 3 | 5 | 27 | Semantic | Variable not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | No | No | No | No | 1 | Yes | Yes | No | Yes | 0 | No | No | No | No | Variable not declared | Semantic | 1 | 1 | No | Yes | 1 | No | No | 1 | Yes | No | 0 | No | No | |||||||
24 | 21 | C | 2 | 4 | 25 | Syntax | Invalid return type (3x) | 3 | 3 | No | Yes | Yes | Yes | 2 | No | No | No | No | 2 | Partially | Yes | Partially | Yes | 1 | Partially | Yes | Partially | Yes | Invalid return type (3x) | Syntax | 3 | 3 | No | Yes | 2 | No | No | 2 | Partially | No | 1 | Partially | No | |||||||
25 | 22 | C | 2 | 4 | 20 | Semantic | Function not defined | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | |||||||
26 | 23 | C | 2 | 4 | 19 | Semantic | Function not defined | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | No | Yes | |||||||
27 | 24 | C | 2 | 4 | 19 | Semantic | Function not defined | 1 | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | No | Yes | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | |||||||
28 | 25 | C | 5 | 8 | 33 | Semantic | Variable not declared (3x) | 3 | 3 | No | Yes | Yes | Yes | 3 | No | Yes | Yes | Yes | 3 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | Variable not declared (3x) | Semantic | 3 | 3 | No | Yes | 3 | No | Yes | 3 | Yes | Yes | 0 | No | No | |||||||
29 | 26 | C | 4 | 8 | 26 | Semantic | Function not defined (2x) | 2 | 2 | No | Yes | Yes | Yes | 2 | No | No | No | No | 0 | No | No | No | No | 1 | No | Yes | No | Partially | Function not defined (2x) | Semantic | 2 | 2 | No | Yes | 2 | No | No | 0 | No | No | 1 | No | No | |||||||
30 | 27 | C | 2 | 3 | 26 | Semantic | Type mismatch | 1 | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | 0 | No | No | No | No | 0 | No | No | No | No | Type mismatch | Semantic | 1 | 0 | No | No | 1 | No | Yes | 0 | No | No | 0 | No | No | |||||||
31 | 28 | C | 1 | 2 | 11 | Semantic | Function signature mismatch | 1 | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 1 | Yes | Yes | Yes | Yes | Function signature mismatch | Semantic | 1 | 1 | No | Yes | 1 | Yes | Yes | 0 | No | No | 1 | Yes | Yes | |||||||
32 | 29 | C | 2 | 4 | 22 | Semantic | Function redefinition | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Function redefinition | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | No | Yes | 1 | Yes | Yes | |||||||
33 | 30 | C | 3 | 2 | 21 | Semantic | Macro not defined | 1 | 1 | No | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 0 | No | No | No | No | 0 | No | No | No | No | Macro not defined | Semantic | 1 | 1 | No | Yes | 1 | No | Yes | 0 | No | No | 0 | No | No | |||||||
34 | ||||||||||||||||||||||||||||||||||||||||||||||||||
35 | ||||||||||||||||||||||||||||||||||||||||||||||||||
36 | ||||||||||||||||||||||||||||||||||||||||||||||||||
37 | Error | #Produtos não detectados | ||||||||||||||||||||||||||||||||||||||||||||||||
38 | Type not declared | 2 | ||||||||||||||||||||||||||||||||||||||||||||||||
39 | Function redefinition | 1 | ||||||||||||||||||||||||||||||||||||||||||||||||
40 | Function signature mismatch | 4 | ||||||||||||||||||||||||||||||||||||||||||||||||
41 | Variable not declared | 1 | ||||||||||||||||||||||||||||||||||||||||||||||||
42 | Invalid return type | 1 | ||||||||||||||||||||||||||||||||||||||||||||||||