Conversation
Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>
… for more robust testing
…tly used in webdriverv8 Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>
Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>
…on and interestital spec
…plemenetary data tests are executing before page is rendered Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>
| // directory is where your package.json resides, so `wdio` will be called from there. | ||
| // | ||
| specs: ["./spec/**/*.js"], | ||
| specs, |
There was a problem hiding this comment.
We now define Specs at the top, depedning on what the MODE variable equals too. So we can sepearte the SDS Specs (flakey) from the rest of the Specs
| booleanFlag, | ||
| }); | ||
| this.url(`/session?token=${token}`); | ||
| await this.url(`/session?token=${encodeURIComponent(token)}`); |
There was a problem hiding this comment.
await to prevent any race conditions - flakiness
encodeURIcomponent for an added layer of security
| npm run test_functional_main | ||
|
|
||
| test-functional-sds: generate-pages | ||
| EQ_FUNCTIONAL_TEST_MAX_INSTANCES=1 npm run test_functional_sds |
There was a problem hiding this comment.
Flakiness was persisting when Instances was higher than 1
|
Just added a few comments explaining reasons for certain changes |
liamtoozer
left a comment
liamtoozer
left a comment
There was a problem hiding this comment.
Changes look great and just wanted to call out great PR description as well 👍
I think this is more to do with the bump to wdio v9, but I wonder if some of the new flakiness we're seeing more of in Actions now is because of this. I think we might need to spend a bit of extra time to prevent introducing extra flakiness for some of these tests, because I'm seeing a few different failures that weren't cropping up as often with v8. Appreciate that you're trying to tackle this already though 👍 Not sure what the solution is yet but we might need to park this and spawn off an investigation into some of the newer intermittent failures?
Also, just a very minor suggestion, I think we might benefit from calling getHTML() from inside a central wrapper/helper function, and defaulting with the { prettify: false } object there instead. Otherwise, passing the object in multiple place might be a maintenance overhead if we need to refactor in future? Might help keep some of the assertions to look a bit cleaner?
| beforeEach(async () => { | ||
| await browser.openQuestionnaire("test_mutually_exclusive.json"); | ||
| await browser.pause(100); | ||
|
|
There was a problem hiding this comment.
Very, very (very) minor: Not sure if the linter might have done this, but there are now blank lines where the previous browser.pause() was? I think there are a few files like this that might need to be updated too
There was a problem hiding this comment.
Blank lines have been removed 🎉
Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>
4 participants
What is the context of this PR?
This PR upgrades all WDIO packages to
v9.20.0and updates our functional tests to comply with the new upgrades.Changes made
getHTML()no longer takes a boolean flag. It now accepts an options object. The default options:({ includeSelectorTag: true, prettify: true })were causing differences in the returned HTML, which led to assertion failures (formatting changes and missing alt attributes on logo tags). We have to explicitly set these values to false.newWindow()now needs to explicitly have a URL value. In v8,newWindow()implicitly openedabout:blankwhen no URL was provided. In v9 this implicit default was removed, so we now explicitly pass 'about:blank' when opening a new window.SDS Tests in GA are failing when in the main test, but work when separated into their own job with "MAX_INSTANCE=1". Also the very first SDS launch always hits the “Sorry, there is a problem with this service” page. So a new helper function -
openQuestionnaireWithRetry(). I think we need an investigation on why SDS is failing like this in GA only.Chrome options also updated in an attempt to make tests more consistent.
Babel dependencies have also been upgraded to latest versions
How to review
Links
Checklist