Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
cff-version: 1.2.0
message: "Please cite this software using the metadata from 'preferred-citation'."
title: "RedBench: A Benchmark for Test Case Reduction"
abstract: This repository contains the RedBench benchmark for the evaluation of test case reduction techniques.
authors:
- family-names: Kreutzer
given-names: Patrick
affiliation: "Programming Systems Group, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Germany"
- family-names: Kunze
given-names: Tom
affiliation: "Programming Systems Group, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Germany"
- family-names: Philippsen
given-names: Michael
affiliation: "Programming Systems Group, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Germany"
orcid: "https://orcid.org/0000-0002-3202-2904"
license: MIT
repository-code: "https://github.com/FAU-Inf2/RedBench"
preferred-citation:
authors:
- family-names: Kreutzer
given-names: Patrick
affiliation: "Programming Systems Group, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Germany"
- family-names: Kunze
given-names: Tom
affiliation: "Programming Systems Group, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Germany"
- family-names: Philippsen
given-names: Michael
affiliation: "Programming Systems Group, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Germany"
orcid: "https://orcid.org/0000-0002-3202-2904"
title: "Test Case Reduction: A Framework, Benchmark, and Comparative Study"
type: conference-paper
abstract: "Given a program that triggers a bug in a compiler (or other kind of language processor), the goal of
test case reduction is to cut away all code that is irrelevant for the bug, i.e., to generate a smaller program
that still induces the bug. Research has proposed several language-agnostic reduction techniques that automatically
reduce bug-inducing programs in arbitrary programming languages, but there is no large-scale, conclusive evaluation
of these algorithms yet. Furthermore, the development of new algorithms is hampered by the unavailability of
comparable implementations of previous techniques and of diverse test programs that trigger different bugs in real
compilers. To close these gaps and to foster future research in this area, this paper makes three contributions:
(1) A framework that includes efficient, fine-tuned implementations of 6 state-of-the-art reducers,
(2) a diverse benchmark that comprises 321 fuzzer-generated programs in two programming languages that trigger 110
different bugs in real compilers, and (3) a comparative study that builds upon our framework and benchmark and
compares the reduction techniques w.r.t. their effectiveness and efficiency. Our results show that there is no
reduction technique yet that performs best across all test cases and languages. Our framework and benchmark are
available online and we provide the necessary scripts and tools to replicate our study."
doi: "10.1109/ICSME52107.2021.00012"
conference:
name: "ICSME'21: International Conference on Software Maintenance and Evolution"
city: Virtual
country: LU
date-published: "2021-09-27"
year: 2021
month: 9
start: 58
end: 69
issn: "2576-3148"