Skip to content

Codecov Test Analytics demo#18

Open
sfanahata wants to merge 3 commits intomainfrom
test-analytics-demo
Open

Codecov Test Analytics demo#18
sfanahata wants to merge 3 commits intomainfrom
test-analytics-demo

Conversation

@sfanahata
Copy link
Copy Markdown
Owner

This PR demonstrates Codecov's Test Analytics features with:

🎯 Intentionally Failing Tests

  • 10 definitely failing tests to show failure detection
  • Various failure types: math errors, DOM issues, async timeouts, exceptions
  • Clear stack traces for easy debugging

🎲 Flaky Tests

  • 8 flaky tests that fail randomly (20-50% of the time)
  • Timing-based failures, network simulation, race conditions
  • Environment-dependent failures

📊 Test Analytics Features Demonstrated

  • Failed Test Detection: Clear identification of which tests fail and why
  • Stack Trace Analysis: Detailed error information in PR comments
  • Flaky Test Identification: Distinguishes between real failures and flaky tests
  • Test Suite Performance: Shows slow/failing tests for prioritization

🔧 Additional Changes

  • Add activity count display in activities page header
  • Enhanced test coverage reporting
  • JUnit test results for better analytics

This setup will trigger Codecov's Test Analytics to show: ✅ Failed test detection with stack traces
✅ Flaky test identification and separation
✅ Test suite performance metrics
✅ Actionable PR comments for developers

Perfect for demonstrating the value of Test Analytics! 🎉

This PR demonstrates Codecov's Test Analytics features with:

## 🎯 Intentionally Failing Tests
- 10 definitely failing tests to show failure detection
- Various failure types: math errors, DOM issues, async timeouts, exceptions
- Clear stack traces for easy debugging

## 🎲 Flaky Tests
- 8 flaky tests that fail randomly (20-50% of the time)
- Timing-based failures, network simulation, race conditions
- Environment-dependent failures

## 📊 Test Analytics Features Demonstrated
- **Failed Test Detection**: Clear identification of which tests fail and why
- **Stack Trace Analysis**: Detailed error information in PR comments
- **Flaky Test Identification**: Distinguishes between real failures and flaky tests
- **Test Suite Performance**: Shows slow/failing tests for prioritization

## 🔧 Additional Changes
- Add activity count display in activities page header
- Enhanced test coverage reporting
- JUnit test results for better analytics

This setup will trigger Codecov's Test Analytics to show:
✅ Failed test detection with stack traces
✅ Flaky test identification and separation
✅ Test suite performance metrics
✅ Actionable PR comments for developers

Perfect for demonstrating the value of Test Analytics! 🎉
@vercel
Copy link
Copy Markdown

vercel bot commented Sep 12, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
fitfest Ready Ready Preview Comment Sep 12, 2025 7:25pm

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 12, 2025

❌ 19 Tests Failed:

Tests completed Failed Passed Skipped
39 19 20 0
View the top 3 failed test(s) by shortest run time
Definitely Failing Tests for Test Analytics Demo should fail due to exception handling
Stack Traces | 0s run time
TypeError: Cannot read properties of null (reading 'nonExistentProperty')
    at Object.nonExistentProperty (.../src/tests/DefinitelyFailingTest.test.tsx:41:16)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to regex mismatch
Stack Traces | 0s run time
Error: expect(received).toBe(expected) // Object.is equality

Expected: true
Received: false
    at Object.toBe (.../src/tests/DefinitelyFailingTest.test.tsx:47:36)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Test Analytics Demo - Intentionally Problematic Tests should fail due to exception
Stack Traces | 0s run time
TypeError: Cannot read properties of null (reading 'someProperty')
    at Object.someProperty (.../src/tests/TestAnalyticsDemo.test.tsx:76:16)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to object property check
Stack Traces | 0.001s run time
Error: expect(received).toHaveProperty(path)

Expected path: "email"
Received path: []

Received value: {"age": 30, "name": "John"}
    at Object.toHaveProperty (.../src/tests/DefinitelyFailingTest.test.tsx:29:17)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to promise rejection
Stack Traces | 0.001s run time
Error: expect(received).resolves.toBe()

Received promise rejected instead of resolved
Rejected to value: [Error: Intentional failure]
    at expect (.../expect/build/index.js:113:15)
    at Object.expect (.../src/tests/DefinitelyFailingTest.test.tsx:57:11)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to type assertion
Stack Traces | 0.001s run time
Error: expect(received).toBe(expected) // Object.is equality

Expected: "42"
Received: 42
    at Object.toBe (.../src/tests/DefinitelyFailingTest.test.tsx:52:19)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Test Analytics Demo - Intentionally Problematic Tests should be flaky due to environment dependencies
Stack Traces | 0.001s run time
Error: expect(received).toBeLessThan(expected)

Expected: < 12
Received:   19
    at Object.toBeLessThan (.../src/tests/TestAnalyticsDemo.test.tsx:61:25)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to array comparison
Stack Traces | 0.002s run time
Error: expect(received).toEqual(expected) // deep equality

- Expected  - 1
+ Received  + 0

  Array [
    1,
    2,
    3,
-   4,
  ]
    at Object.toEqual (.../src/tests/DefinitelyFailingTest.test.tsx:24:19)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Test Analytics Demo - Intentionally Problematic Tests should fail due to incorrect assertion
Stack Traces | 0.002s run time
Error: expect(received).toBeGreaterThan(expected)

Expected: > 1
Received:   0.36789284511285425
    at Object.toBeGreaterThan (.../src/tests/TestAnalyticsDemo.test.tsx:20:20)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to incorrect math
Stack Traces | 0.003s run time
Error: expect(received).toBe(expected) // Object.is equality

Expected: 3
Received: 2
    at Object.toBe (.../src/tests/DefinitelyFailingTest.test.tsx:8:19)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to wrong string comparison
Stack Traces | 0.003s run time
Error: expect(received).toBe(expected) // Object.is equality

Expected: "Goodbye, World!"
Received: "Hello, World!"
    at Object.toBe (.../src/tests/DefinitelyFailingTest.test.tsx:13:21)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Definitely Failing Tests for Test Analytics Demo should fail due to missing DOM element
Stack Traces | 0.006s run time
TestingLibraryElementError: Unable to find an element with the text: Click Me Not. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

Ignored nodes: comments, script, style
<body>
  <div>
    <button
      class="bg-fitfest-deep text-white px-4 py-2 rounded hover:bg-fitfest-bright dark:bg-fitfest-bright dark:hover:bg-fitfest-deep transition-colors disabled:opacity-50 shadow-sm "
    >
      Click Me
    </button>
  </div>
</body>
    at Object.getElementError (.../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/config.js:37:19)
    at .../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/query-helpers.js:76:38
    at .../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/query-helpers.js:52:17
    at .../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/query-helpers.js:95:19
    at Object.getByText (.../src/tests/DefinitelyFailingTest.test.tsx:19:19)
    at Promise.then.completed (.../fitfest/app/node_modules/jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../fitfest/app/node_modules/jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../fitfest/app/node_modules/jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../fitfest/app/node_modules/jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../fitfest/app/node_modules/jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../fitfest/app/node_modules/jest-circus/build/run.js:121:9)
    at run (.../fitfest/app/node_modules/jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../fitfest/app/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../fitfest/app/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../fitfest/app/node_modules/jest-runner/build/runTest.js:367:16)
    at runTest (.../fitfest/app/node_modules/jest-runner/build/runTest.js:444:34)
    at Object.worker (.../fitfest/app/node_modules/jest-runner/build/testWorker.js:106:12)
Flaky Test Examples for Codecov Test Analytics should sometimes fail due to timing issues
Stack Traces | 0.006s run time
Error: expect(received).toBeGreaterThan(expected)

Expected: > 0.8
Received:   0.3934368637685368
    at Object.toBeGreaterThan (.../src/tests/FlakyTestExample.test.tsx:9:20)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Test Analytics Demo - Intentionally Problematic Tests should fail consistently to demonstrate test failure detection
Stack Traces | 0.006s run time
Error: expect(received).toBe(expected) // Object.is equality

Expected: 5
Received: 4
    at Object.toBe (.../src/tests/TestAnalyticsDemo.test.tsx:8:19)
    at Promise.then.completed (.../jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../jest-circus/build/run.js:121:9)
    at run (.../jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Test Analytics Demo - Intentionally Problematic Tests should be flaky due to network simulation
Stack Traces | 0.007s run time
Error: expect(received).toBeUndefined()

Received: [Error: Simulated network failure]
    at Object.toBeUndefined (.../src/tests/TestAnalyticsDemo.test.tsx:53:21)
Test Analytics Demo - Intentionally Problematic Tests should fail due to missing element
Stack Traces | 0.037s run time
TestingLibraryElementError: Unable to find an element with the text: Non-existent button text. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

Ignored nodes: comments, script, style
<body>
  <div>
    <button
      class="bg-fitfest-deep text-white px-4 py-2 rounded hover:bg-fitfest-bright dark:bg-fitfest-bright dark:hover:bg-fitfest-deep transition-colors disabled:opacity-50 shadow-sm "
    >
      Test Button
    </button>
  </div>
</body>
    at Object.getElementError (.../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/config.js:37:19)
    at .../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/query-helpers.js:76:38
    at .../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/query-helpers.js:52:17
    at .../fitfest/app/node_modules/@testing-library/react/node_modules/@.../dom/dist/query-helpers.js:95:19
    at Object.getByText (.../src/tests/TestAnalyticsDemo.test.tsx:14:19)
    at Promise.then.completed (.../fitfest/app/node_modules/jest-circus/build/utils.js:298:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../fitfest/app/node_modules/jest-circus/build/utils.js:231:10)
    at _callCircusTest (.../fitfest/app/node_modules/jest-circus/build/run.js:316:40)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at _runTest (.../fitfest/app/node_modules/jest-circus/build/run.js:252:3)
    at _runTestsForDescribeBlock (.../fitfest/app/node_modules/jest-circus/build/run.js:126:9)
    at _runTestsForDescribeBlock (.../fitfest/app/node_modules/jest-circus/build/run.js:121:9)
    at run (.../fitfest/app/node_modules/jest-circus/build/run.js:71:3)
    at runAndTransformResultsToJestFormat (.../fitfest/app/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)
    at jestAdapter (.../fitfest/app/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:79:19)
    at runTestInternal (.../fitfest/app/node_modules/jest-runner/build/runTest.js:367:16)
    at runTest (.../fitfest/app/node_modules/jest-runner/build/runTest.js:444:34)
    at Object.worker (.../fitfest/app/node_modules/jest-runner/build/testWorker.js:106:12)
Test Analytics Demo - Intentionally Problematic Tests should be flaky due to timing issues
Stack Traces | 0.041s run time
Error: expect(received).toBeGreaterThan(expected)

Expected: > 100
Received:   39
    at Object.toBeGreaterThan (.../src/tests/TestAnalyticsDemo.test.tsx:30:33)
Definitely Failing Tests for Test Analytics Demo should fail due to async operation timeout
Stack Traces | 0.101s run time
Error: thrown: "Exceeded timeout of 100 ms for a test.
Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."
    at it (.../src/tests/DefinitelyFailingTest.test.tsx:32:3)
    at _dispatchDescribe (.../jest-circus/build/index.js:91:26)
    at describe (.../jest-circus/build/index.js:55:5)
    at Object.describe (.../src/tests/DefinitelyFailingTest.test.tsx:5:1)
    at Runtime._execModule (.../jest-runtime/build/index.js:1439:24)
    at Runtime._loadModule (.../jest-runtime/build/index.js:1022:12)
    at Runtime.requireModule (.../jest-runtime/build/index.js:882:12)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:77:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)
Test Analytics Demo - Intentionally Problematic Tests should fail due to async timeout
Stack Traces | 1s run time
Error: thrown: "Exceeded timeout of 1000 ms for a test.
Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."
    at it (.../src/tests/TestAnalyticsDemo.test.tsx:64:3)
    at _dispatchDescribe (.../jest-circus/build/index.js:91:26)
    at describe (.../jest-circus/build/index.js:55:5)
    at Object.describe (.../src/tests/TestAnalyticsDemo.test.tsx:5:1)
    at Runtime._execModule (.../jest-runtime/build/index.js:1439:24)
    at Runtime._loadModule (.../jest-runtime/build/index.js:1022:12)
    at Runtime.requireModule (.../jest-runtime/build/index.js:882:12)
    at jestAdapter (.../build/legacy-code-todo-rewrite/jestAdapter.js:77:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at runTestInternal (.../jest-runner/build/runTest.js:367:16)
    at runTest (.../jest-runner/build/runTest.js:444:34)
    at Object.worker (.../jest-runner/build/testWorker.js:106:12)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

- Remove continue-on-error to allow test failures to fail the build
- Combine test coverage and JUnit output generation
- Enable Test Analytics to detect failing tests

This should now properly trigger Codecov Test Analytics when tests fail.
- Use codecov/test-results-action@v1 for Test Analytics (as per docs)
- Generate JUnit XML with proper Jest configuration
- Use JEST_JUNIT_CLASSNAME and --reporters=jest-junit
- Follow exact setup from Codecov Test Analytics documentation

This should now properly trigger Test Analytics with failing tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant