Skip to content

Split -f flag into separate options: include results from failures vs exitcode handling #285

@smarr

Description

@smarr

Currently, the -f both means:

  1. that we include data from failing runs, which can be caused either by benchmarks aborting or because the validation of results fails or output indicates other issues

  2. that the rebench exit code indicates no failure

We may want to separate these concerns out.

It might be useful to have a --dont-fail option, just to keep the exit code 0.
Where --faulty is only about including data from runs that indicate some issue.

Requested by @OctaveLarose

Metadata

Metadata

Assignees

No one assigned

    Labels

    featuregood first issuea relatively simple task to familiarize oneself with the code base, and the contribution processhelp wantedWe'd like help with this issue, and are happy to provide guidance

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions