FailureReport's exit() causes exception backtrace to not get output

[cgriego]

Given a project using --failure-threshold and an uncaught exception being raised (one outside of a test/spec), some/all of the tests/specs will fail to execute, resulting in threshold not being met. At this point, the FailureReport formatter will kick in, notice the coverage failure, and call exit(1), to ensure an error exit code, which raises a SystemExit. The SystemExit causes the original exception's backtrace to not get output.

This is affecting my team's ability to diagnose test failures that happen as a result of an initialization failure (database connection problem, configuration problem, etc.) because the underlying exception isn't visible, but instead we simply see a 0% coverage error.

I'm unsure what would be the best solution to this would be, but here are two options that seemed to work when tried. One is not to raise SystemExit when an exception has already been raised using exit(1) if $__rcov_exit_exception.nil?. The exit code will still be non-zero because of the original exception. Another is to register the SystemExit as an after_exit hook using after_exit { exit(1) }.

[abedra]

This is an interesting case. I agree that this is an issue. The initial reason for introducing the nonzero exit is for CI builds that look for non zero exits to trigger a failure in the build. I will take a closer look here and see what the best course of action is. I don't think the after_exit hook is the best option because that has the potential to be overridden or ignored. The conditional exit looks like it might be appropriate though.