Symptoms

Every bug report of our dataset consists of a short description that contains information about how the bug is triggered along with the compiler’s expected and actual behavior. We carefully examined the differences between the compiler’s expected and actual behavior, and grouped these differences into categories.

A symptom indicates how a user understands that something is going wrong in the compiler. We found five categories of symptoms

  • Unexpected Compile-Time Error

  • Internal Compiler Error

  • Unexpected Runtime Behavior

  • Misleading Report

  • Compilation Performance Issue

Below, we discuss each symptom category in detail.

Unexpected Compile-Time Error

A bug involving this symptom manifests itself when the compiler rejects a well-typed program, producing an informative error message to the developer. Such errors may frustrate developers, leaving them with the impression that their programs are indeed incorrect.

Example

A buggy compiler does not compiler (although it should) the following correct Kotlin program, and produces the erroneous report: “5: error: inferred type is Any, but String was expected”

1 fun foo() =
2    ""
3 
4 fun test() {
5   val x: String = foo() // error here
6 }

Internal Compiler Error

Internal compiler errors manifest themselves when the compiler terminates its execution abnormally. This symptom differs from unexpected compile-time error, because the compiler is unable to yield a normal diagnostic message, or even generate target code. Internal compiler errors are clear indications that something is not working well in the compiler.

Example

The Java compiler crashes with the following stack trace when trying to compile an input program

java.lang.NullPointerException: Cannot read field "sym" because "this.lvar[2]" is null
at jdk.compiler/com.sun.tools.javac.jvm.Code.emitop0(Code.java:574)
at jdk.compiler/com.sun.tools.javac.jvm.Gen.visitTypeTest(Gen.java:2233)
...

Unexpected Runtime Behavior

A bug related to an unexpected runtime behavior manifests itself when running the executable generated by the compiler. This involves the successful compilation of a given source program and the generation of a faulty executable that in turn, may lead to errors and wrong outcomes.

There are two reasons why a compiler may generate incorrect executables. First, a compiler bug can break the soundness of the type system. Hence, the compiler accepts an invalid program which it should have rejected. Such bugs are important, because they defeat the safety offered by type systems in statically-typed languages. Second, the compiler may perform wrong static linking between methods and objects (e.g., it chooses the wrong overloaded method to call).

Example

The compiler of Groovy mistakenly accepts the following type-incorrect program (notice the type error at line 3), and produces the corresponding JAR file.

1 class Test {
2   public static void main(String[] args) {
3     Integer[] arr = new Integer[] {1, "foo"}; // should report error here
4     Integer x = arr[1]; // ClassCastException at runtime
5   }
6 }

However, when running the generated JAR file, JVM produces a ClassCastException when performing the assignment at line 4 as follows

Exception in thread "main" org.codehaus.groovy.runtime.typehandling.GroovyCastException:
Cannot cast object 'str' with class 'java.lang.String' to class 'java.lang.Integer'

Misleading Report

Misleading reports appear when for a given program, the compiler emits a false warning or a false error message. False warnings and error messages may be misleading because they suggest ineffective fixes (e.g., warning about an unsafe cast, but the cast is actually safe). Furthermore, spurious messages can hide other program errors (e.g., the compiler reports a type mis-match error instead of an uninitialized variable error). Unlike unexpected compile-time error, in case of misleading report, the compiler correctly accepts (or rejects), a valid (or invalid) program. However, it does so by producing wrong diagnostic messages.

Example

Consider the following Kotlin program.

1 interface X<T> {
2   inner enum class C : X<T>
3 }

This program triggers a bug with a misleading report symptom, because the compiler produces two contradictory error messages:

  • error (2, 3): Modifier ’inner’ is not applicable to enum class”

  • error (2, 26): Expression is inaccessible from a nested class ‘C’, use ‘inner’ keyword to make the class inner”.

Compilation Performance Issue

Bugs related to this symptom cause noticeable degradations in compilation performance. The impact of such bugs is the waste of developers’ time and resources, because the compiler requires much time or memory to compile even the simplest fragment of code, and in many cases,compilation never terminates.

Example

The compiler takes 5 minutes to compile a program consisting of 10 lines of code.