It should be possible to give replay patterns in case a benchmarks failed, together with a post mortem of the failed benchmark execution (eg files with stdin and stdout). It's otherwise a source of stress when closing in on deadlines with large evaluation in the pipe. In other words, one failed benchmark shouldn't compromise an entire campaign.
It should be possible to give replay patterns in case a benchmarks failed, together with a post mortem of the failed benchmark execution (eg files with stdin and stdout). It's otherwise a source of stress when closing in on deadlines with large evaluation in the pipe. In other words, one failed benchmark shouldn't compromise an entire campaign.