Questions

• We're planning on extracting information from BugSwarm pairs, but the failing commits of these pairs are only a subset of all commits that fail CI builds. Jeremy has seen failing builds with no corresponding passing build (e.g., the maintainer just accepts a pull request that is failing). What is the density of the failing commits that have a corresponding passing test among all failing commits? Are their properties different? If this is the case, can we predict if a bug can be fixed by APR?

• Regressions vs. TDD vs. bad tests?

• Does the stack trace provide useful information for localising the bug or developer fix? (I.e., should we try to use stack trace information to guide APR.)

• How quickly was the bug fixed by the developer? Could APR have fixed the bug within that window of time?

• Was the bug reported on the issue tracker? Did was the user that reported the bug the one who fixed it?

• How many bugs are the result of refucktoring or API migrations? There is reason to believe that we could tackle that subset of bugs quite effectively, but is that subset large enough to warrant our research efforts?

Repairnator Questions:

• For those failing builds, what do the successive commits look like? What does progress to the fix look like?