Conversation
✅ Deploy Preview for superhero-trendminer-har2sa45a90zw62 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for superhero-c9df18 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for celadon-youtiao-c85d92 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app ❌ Failed Tests: |
Andreea167
changed the title
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app ❌ Failed Tests: |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app ❌ Failed Tests: |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
✅ E2E Tests PASSEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
2 similar comments
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
✅ E2E Tests PASSEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
❌ E2E Tests FAILEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
✅ E2E Tests PASSEDTested URL: https://deploy-preview-133--superhero-c9df18.netlify.app |
Andreea167
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| id: failed_tests | ||
| working-directory: keypair-testing | ||
| run: | | ||
| if [ -f "test-output.log" ]; then |
There was a problem hiding this comment.
Bug: Failed Tests Not Reported in PR Comments
The E2E test workflow attempts to extract failed test names from test-output.log, but the Playwright command doesn't save its output to this file. As a result, failed tests are never reported in the PR comment.
| timeout-minutes: 20 | ||
|
|
||
| steps: | ||
| - name: Wait for Netlify Preview |
There was a problem hiding this comment.
This is not obvious that e2e tests should depend on deployment. Maybe it would be faster to build the app here or get the built app from the previous step? Doing so will definitely be more reliable since it won't depend on Netlify.
| env: | ||
| SUPERHERO_COM_BASE_URL: ${{ steps.netlify.outputs.url }} | ||
| run: | | ||
| SUPERHERO_COM_BASE_URL=$SUPERHERO_COM_BASE_URL npx playwright test tests_superhero_react/superhero-react.spec.ts --retries=1 --config=./playwright.config.ts |
There was a problem hiding this comment.
The problem with this approach is that in some cases, the test fails not because the app is broken, but because it is not compatible with the app changes. Then the developer needs to adjust the failing test, but since it is in another repository, he needs to do another PR to the test suite 🙈 The best would be to keep tests in the same repository as the app.
There was a problem hiding this comment.
Do you need --retries=1 because of flaky tests?
| retention-days: 30 | ||
| if-no-files-found: warn | ||
|
|
||
| - name: Comment PR with test results |
There was a problem hiding this comment.
This step is not necessary as GitHub already displays the CI status at the end of the PR and for every commit. It also appears more compact than the comments.
| - name: Clone keypair-testing repo | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: aeternity/keypair-testing |
There was a problem hiding this comment.
Makes sense to specify the commit of "keypair-testing" explicitly, to keep the relation between repositories. Otherwise, e2e tests won't work for old commits in "superhero" because "keypair-testing" get changed.
3 participants
No description provided.