@ Martin Kellogg, August 22 2016

This file describes the archive of the tarballs used in the N-Prog
scalability experiment. This experiment will appear in our paper
"Combining Bug Detection and Test Case Generation" which we're
submitting to ICSE '17.

This archive contains 214 single bug scenarios and 8 multi-bug
scenarios. The 8 multi-bug scenarios are:

4 scenarios with three bugs:
  potion-123.tar.gz
  potion-456.tar.gz
  potion-789.tar.gz
  potion-131415.tar.gz

3 scenarios with five bugs:
  potion-12345.tar.gz
  potion-678910.tar.gz
  potion-1112131415.tar.gz

1 scenario with fifteen bugs:
  potion-all.tar.gz

All of the rest of the scenarios have just one bug. Each scenario
has been modified in two ways: first, it has been setup to interact
with GenProg and its toolchain (see the GenProg documentation for
information about this process). Second, I have modified the
negative test cases. In the original version of each of these
programs, each negative test checks if a particular result
(the correct answer) is achieved, and fails otherwise. For this
experiment, we needed to know if a particular result (the buggy
output) was NOT achieved, to test for divergence. To do this,
the negative tests have all been modified: their expected output
has been replaced with the output of the buggy version.

The scenarios are named according to their GenProg scenario
name, with -instr appended to the name, with the exception
of potion. All the potion scenarios don't have the -instr
suffix.

The results reported in the paper come from running a
collection of scripts---which eventually became N-Prog---on
each of these tarballs in an Amazon AWS instance, using the
GenProg 2012 AMI. We don't recommend trying to run these
on any other platform---it's a pain and often doesn't work
because of dependencies, etc.