This might be specific to the way notifications are working in Ubuntu.
This change:
66aafde
results in run_tests() always returning true, hence :all_after_pass no longer functions.
I had a look through the tests and because $? is stubbed all over the place I can't find a trivial way of putting in an accurate test without refactoring.