BugsJS: a Benchmark of JavaScript Bugs

Abstract

JavaScript is a popular programming language that is also error-prone due to its asynchronous, dynamic, and loosely-typed nature. Hence, in recent years many techniques have been proposed for analyzing and testing JavaScript applications. However, our survey of the literature in this area reveals that the proposed techniques are often evaluated on different datasets of programs and bugs. The lack of a commonly used benchmark limits the ability to perform fair and unbiased comparisons for assessing the efficacy of new techniques. To fill this gap, we propose BugsJS, a benchmark of 453 real, manually-validated JavaScript bugs from 10 popular JavaScript server-side pro- grams, comprising 444k LOC in total. Each bug is accompanied by its bug report, the test cases that detect it, as well as the patch that fixes it. BugsJS features a rich interface for accessing the faulty and fixed versions of the programs and executing the corresponding test cases, which facilitates conducting highly- reproducible empirical studies and comparisons of JavaScript analysis and testing tools.

Bugs Benchmark JavaScript Node.JS ICST

BibTex

@inproceedings{Gyimesi:ICST:2019:BugsJS,
    author={Gyimesi, Péter and Vancsics, Béla and Stocco, Andrea and Mazinanian, Davood and Beszédes, Árpád and Ferenc, Rudolf and Mesbah, Ali},
    title={{BugsJS: a Benchmark of JavaScript Bugs}},
    booktitle={{Proceedings of the 12th International Conference on Software Testing, Verification, and Validation (ICST)}},
    series = {ICST 2019},
    year = {2019},
    location = {Xian, China},
    numpages = {12}
}