| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | AC | AD | AE | AF | AG | AH | AI | AJ | AK | AL | AM | AN | AO | AP | AQ | AR | AS | AT | AU | AV | AW | AX | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | GPT4 | Le Chat Mistral | Gemini Advanced | LLama 3 | |||||||||||||||||||||||||||||||||||||||||||||||
2 | Detect | Fix? | Explanation | Detect | Fix? | Explanation | Detect | Fix? | Explanation | Detect | Fix? | Explanation | GPT4 | Le Chat Mistral | Gemini Advanced | LLama 3 | |||||||||||||||||||||||||||||||||||
3 | id | Language | Macros | #Prod. | LOC | Type | Error | #CE | Code Element | Type of error | Location | Code Element | Type of error | Location | Code Element | Type of error | Location | Code Element | Type of error | Location | Error | Type | #NC | #Detect | Fix | Explanation | #Detect | Fix | Explanation | #Detect | Fix | Explanation | #Detect | Fix | Explanation | LLM | Detected Product Lines | Detected Products | Fix | Explanation | |||||||||||
4 | 1 | C | 2 | 4 | 9 | Semantic | Type not declared | 1 | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 0 | No | No | No | No | 0 | No | No | No | No | Type not declared | Semantic | 1 | 1 | Yes | Yes | 0 | No | No | 0 | No | No | 0 | No | No | GPT4 | 29/30 | 38/40 | 12/30 | 26/29 | |||
5 | 2 | C | 2 | 4 | 16 | Semantic | Function not defined; Function redefinition | 2 | 2 | Partially | Yes | Yes | Yes | 1 | No | Yes | Partially | Yes | 2 | No | Yes | Yes | Yes | 1 | No | Yes | Partially | Yes | Function not defined; Function redefinition | Semantic | 2 | 2 | Partially | Yes | 1 | No | No | 2 | No | Yes | 1 | No | No | Le Chat Mistral | 26/30 | 31/40 | 9/30 | 18/26 | |||
6 | 3 | C | 1 | 2 | 10 | Syntax | Mismatching quotes | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Mismatching quotes | Syntax | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | Gemini Advanced | 20/30 | 24/40 | 13/30 | 13/20 | |||
7 | 4 | C | 1 | 2 | 11 | Semantic | Function not defined | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | 1 | No | Yes | LLama 3 | 24/30 | 27/40 | 7/30 | 20/24 | |||
8 | 5 | C | 1 | 2 | 8 | Semantic | Variable not declared | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Variable not declared | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | Yes | Yes | ||||||||
9 | 6 | C | 1 | 2 | 13 | Syntax | Mismatching brackets | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | No | No | Yes | 0 | No | No | No | No | 0 | No | No | No | No | Mismatching brackets | Syntax | 1 | 1 | Yes | Yes | 1 | No | No | 0 | No | No | 0 | No | No | ||||||||
10 | 7 | C | 1 | 2 | 9 | Syntax | Missing semicolon | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 1 | Yes | Yes | Yes | Yes | Missing semicolon | Syntax | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 0 | No | No | 1 | Yes | Yes | ||||||||
11 | 8 | C | 2 | 4 | 13 | Semantic | Type not declared | 1 | 1 | No | Yes | Yes | No | 0 | No | No | No | No | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | Type not declared | Semantic | 1 | 1 | No | No | 0 | No | No | 0 | No | No | 1 | No | Yes | ||||||||
12 | 9 | C | 2 | 4 | 16 | Semantic | Variable not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Variable not declared | Semantic | 1 | 1 | No | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | No | Yes | ||||||||
13 | 10 | C | 2 | 4 | 22 | Semantic | Function signature mismatch (2x) | 2 | 2 | No | Yes | Yes | Yes | 0 | No | No | No | No | 1 | Yes | Partially | Partially | Partially | 2 | No | Yes | Yes | Yes | Function signature mismatch (2x) | Semantic | 2 | 2 | No | Yes | 0 | No | No | 1 | Yes | No | 2 | No | Yes | ||||||||
14 | 11 | C | 2 | 4 | 23 | Semantic | Function signature mismatch (2x); Variable not declared | 3 | 3 | No | Yes | Yes | Yes | 0 | Yes | No | No | No | 1 | No | Partially | Partially | Partially | 3 | No | Yes | Yes | Yes | Function signature mismatch (2x); Variable not declared | Semantic | 3 | 3 | No | Yes | 0 | Yes | No | 1 | No | No | 3 | No | Yes | ||||||||
15 | 12 | C | 1 | 2 | 17 | Semantic | Label not declared | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | No | No | No | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | Label not declared | Semantic | 1 | 1 | Yes | Yes | 1 | No | No | 0 | No | No | 1 | No | Yes | ||||||||
16 | 13 | C | 2 | 4 | 11 | Semantic | Type not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | No | Yes | No | Yes | 1 | No | Yes | No | Yes | 1 | No | Yes | Yes | Yes | Type not declared | Semantic | 1 | 1 | No | Yes | 1 | No | No | 1 | No | No | 1 | No | Yes | ||||||||
17 | 14 | C | 1 | 2 | 5 | Syntax | Missing struct keyword | 1 | 1 | No | Yes | No | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | No | Yes | 1 | No | Yes | Yes | Yes | Missing struct keyword | Syntax | 1 | 1 | No | No | 1 | Yes | Yes | 1 | Yes | No | 1 | No | Yes | ||||||||
18 | 15 | C | 1 | 2 | 10 | Semantic | Label not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | Label not declared | Semantic | 1 | 1 | No | Yes | 1 | Yes | Yes | 0 | No | No | 1 | No | Yes | ||||||||
19 | 16 | C | 1 | 2 | 4 | Syntax | Invalid return type | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Partially | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Invalid return type | Syntax | 1 | 1 | Yes | Yes | 1 | Partially | Yes | 1 | Yes | Yes | 1 | No | Yes | ||||||||
20 | 17 | C | 1 | 2 | 14 | Semantic | Continue statement outside a loop | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Continue statement outside a loop | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | No | Yes | ||||||||
21 | 18 | C | 1 | 2 | 12 | Semantic | Address of bitfield requested | 1 | 1 | No | Yes | Yes | Yes | 1 | No | No | No | No | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Address of bitfield requested | Semantic | 1 | 1 | No | Yes | 1 | No | No | 1 | Yes | Yes | 1 | No | Yes | ||||||||
22 | 19 | C | 3 | 5 | 32 | Semantic | Function not defined; Variable not declared | 2 | 1 | No | Partially | Partially | Partially | 2 | No | Yes | Yes | Yes | 1 | No | Partially | Partially | Partially | 1 | No | Partially | Partially | Yes | Function not defined; Variable not declared | Semantic | 2 | 1 | No | No | 2 | No | Yes | 1 | No | No | 1 | No | No | ||||||||
23 | 20 | C | 3 | 5 | 27 | Semantic | Variable not declared | 1 | 1 | No | Yes | Yes | Yes | 1 | No | No | No | No | 1 | Yes | Yes | No | Yes | 0 | No | No | No | No | Variable not declared | Semantic | 1 | 1 | No | Yes | 1 | No | No | 1 | Yes | No | 0 | No | No | ||||||||
24 | 21 | C | 2 | 4 | 25 | Syntax | Invalid return type (3x) | 3 | 3 | No | Yes | Yes | Yes | 2 | No | No | No | No | 2 | Partially | Yes | Partially | Yes | 1 | Partially | Yes | Partially | Yes | Invalid return type (3x) | Syntax | 3 | 3 | No | Yes | 2 | No | No | 2 | Partially | No | 1 | Partially | No | ||||||||
25 | 22 | C | 2 | 4 | 20 | Semantic | Function not defined | 1 | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | Yes | Yes | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | ||||||||
26 | 23 | C | 2 | 4 | 19 | Semantic | Function not defined | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | Yes | Yes | 1 | No | Yes | ||||||||
27 | 24 | C | 2 | 4 | 19 | Semantic | Function not defined | 1 | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Function not defined | Semantic | 1 | 1 | No | Yes | 1 | Yes | Yes | 1 | Yes | Yes | 1 | Yes | Yes | ||||||||
28 | 25 | C | 5 | 8 | 33 | Semantic | Variable not declared (3x) | 3 | 3 | No | Yes | Yes | Yes | 3 | No | Yes | Yes | Yes | 3 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | Variable not declared (3x) | Semantic | 3 | 3 | No | Yes | 3 | No | Yes | 3 | Yes | Yes | 0 | No | No | ||||||||
29 | 26 | C | 4 | 8 | 26 | Semantic | Function not defined (2x) | 2 | 2 | No | Yes | Yes | Yes | 2 | No | No | No | No | 0 | No | No | No | No | 1 | No | Yes | No | Partially | Function not defined (2x) | Semantic | 2 | 2 | No | Yes | 2 | No | No | 0 | No | No | 1 | No | No | ||||||||
30 | 27 | C | 2 | 3 | 26 | Semantic | Type mismatch | 1 | 0 | No | No | No | No | 1 | No | Yes | Yes | Yes | 0 | No | No | No | No | 0 | No | No | No | No | Type mismatch | Semantic | 1 | 0 | No | No | 1 | No | Yes | 0 | No | No | 0 | No | No | ||||||||
31 | 28 | C | 1 | 2 | 11 | Semantic | Function signature mismatch | 1 | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | 0 | No | No | No | No | 1 | Yes | Yes | Yes | Yes | Function signature mismatch | Semantic | 1 | 1 | No | Yes | 1 | Yes | Yes | 0 | No | No | 1 | Yes | Yes | ||||||||
32 | 29 | C | 2 | 4 | 22 | Semantic | Function redefinition | 1 | 1 | Yes | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 1 | Yes | Yes | Yes | Yes | Function redefinition | Semantic | 1 | 1 | Yes | Yes | 1 | No | Yes | 1 | No | Yes | 1 | Yes | Yes | ||||||||
33 | 30 | C | 3 | 2 | 21 | Semantic | Macro not defined | 1 | 1 | No | Yes | Yes | Yes | 1 | No | Yes | Yes | Yes | 0 | No | No | No | No | 0 | No | No | No | No | Macro not defined | Semantic | 1 | 1 | No | Yes | 1 | No | Yes | 0 | No | No | 0 | No | No | ||||||||
34 | |||||||||||||||||||||||||||||||||||||||||||||||||||
35 | |||||||||||||||||||||||||||||||||||||||||||||||||||
36 | |||||||||||||||||||||||||||||||||||||||||||||||||||
37 | Error | #Produtos não detectados | |||||||||||||||||||||||||||||||||||||||||||||||||
38 | Type not declared | 2 | |||||||||||||||||||||||||||||||||||||||||||||||||
39 | Function redefinition | 1 | |||||||||||||||||||||||||||||||||||||||||||||||||
40 | Function signature mismatch | 4 | |||||||||||||||||||||||||||||||||||||||||||||||||
41 | Variable not declared | 1 | |||||||||||||||||||||||||||||||||||||||||||||||||
42 | Invalid return type | 1 | |||||||||||||||||||||||||||||||||||||||||||||||||