Add --attempts cli flag and test method

[ozziexsh]

What:

• Bug Fix
• New Feature

Description:

This feature is mostly needed due to the addition of the browser testing API. I would argue in normal testing scenarios you would want to design tests that aren't flakey, however with the browser a lot of stuff is out of your control. This PR adds a new --attempts=X cli flag as well as an ->attempts(X) test method, which will try to run the tests up to X times, marking it as passed the first time it succeeds. If the tries exceeds X then the test is marked as failed.

Something that would be nice that I haven't added yet is some sort of output at the end of the test run which lets you know that a test was attempted a few times. I've created a class to keep track of the tests that were reattempted, but I haven't done anything with the tracked tests yet as I'm not sure the best way to add to the final test output

Related:

• [Bug]: AwaitableWebpage::click times out when the next page is slow to load #1511

[ozziexsh]

Converting this to a draft to get feedback first. It's working with normal testing suites but as I'm testing it more with the Browser plugin it doesn't seem to work with it just yet

[nunomaduro]

can you check how this method is being called on frameworks like vitest or jest?

[ozziexsh]

vitest passes retry as an option to the test itself

https://vitest.dev/api/#test-api-reference

test('some test', {retry: 3}, () => {...});

jest uses jest.retryTrimes(3) as a method call at the top of the test file:

https://jestjs.io/docs/jest-object#jestretrytimesnumretries-options

jest.retryTimes(3);

test('some test', () => {...});

I went with attempts since --retry is already a cli flag used for something else.

Also I just tested it out with the browser plugin again in a more controlled environment and I was wrong, it's working fine with it. If this is something you're interested in adding would you be able to give me some pointers on printing to the final test output? It'd be nice to be warned that reattempts were used