Skip to content

Handles errors in before and after in the worst possible way. #44

@LukeShu

Description

@LukeShu

I have a test plan that creates a GPG key in before. I had upgraded GPG since the last time I'd run GPG, so the running gpg-agent didn't match /bin/gpg. This caused an error whenever it tried to run GPG. That is to say, I have a before function that started erroring for reasons unrelated to my test suite or the code being tested.

roundup handled this error in before by aborting, pretending that any pending tests or test plans did not exist.

This meant that the test plan with the failing before showed up as having 0 tests. I, being an alert developer, knew that I had no test plans with 0 tests, and that roundup must be wrong.

Upon hacking roundup to deal with this in a better way, I discovered that I had several other tests that had been failing for much longer; I had about 20 tests that roundup had silently been skipping for many months (as a different before/after had been failing)!

This is a critical bug for a testing framework. Title is only a slight exaggeration (rm -rf on somewhere important would be worse).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions