Exploring Input Parsers with Fast Failure Feedback

If we have a program for which neither code nor traces nor input samples nor input specifications are available, how can we generate test inputs? In this paper, we show that this is possible if the program provides failure feedback--that is, information on whether an input is valid or not, and whether the input can be considered a prefix of a valid input (i.e. incomplete) or not. Such feedback is available out of the box (or very easy to implement) for numerous input processors such as microprocessors, compilers, user interfaces, and off-the-shelf parsers. Given this feedback, we show that it is possible to quickly generate any input that will be accepted by the program, by systematically enumerating and testing input continuations. We have implemented our technique in a prototype called DECODER. In its evaluation on five subjects, DECODER is effective and efficient even in comparison to instrumentation-based competition.