Skip to content

TypeError: Cannot read properties of undefined (reading 'pool') when rendering errors with vitest 4.x #362

@tuntisz

Description

@tuntisz

Description

When using evalite with vitest 4.x, if an eval test throws an error, the error rendering crashes with:

TypeError: Cannot read properties of undefined (reading 'pool')

This masks the actual underlying error, making debugging impossible.

Root Cause

In node_modules/vitest/dist/chunks/index.456_DGfR.js:2983:

if (options.task?.file.pool === "browser" && project.browser)

The optional chaining on task doesn't protect against task.file being undefined. When task exists but task.file is undefined, accessing .pool throws.

This is triggered from evalite's reporter in renderErrorsSummary (node_modules/evalite/dist/reporter/rendering.js:75).

Stack Trace

TypeError: Cannot read properties of undefined (reading 'pool')
    at Object.parseErrorStacktrace (file:///node_modules/vitest/dist/chunks/index.456_DGfR.js:2983:26)
    at printErrorInner (file:///node_modules/vitest/dist/chunks/index.456_DGfR.js:3015:25)
    at printErrorInner (file:///node_modules/vitest/dist/chunks/index.456_DGfR.js:3064:3)
    at printError (file:///node_modules/vitest/dist/chunks/index.456_DGfR.js:2970:9)
    at Logger.printError (file:///node_modules/vitest/dist/chunks/cli-api.C7sYjHmQ.js:6551:3)
    at renderErrorsSummary (file:///node_modules/evalite/dist/reporter/rendering.js:75:24)
    at EvaliteReporter.onTestRunEnd (file:///node_modules/evalite/dist/reporter.js:134:9)

Reproduction Steps

  1. Create a minimal evalite setup with vitest 4.x:
{
  "devDependencies": {
    "@modelcontextprotocol/sdk": "^1.24.3",
    "evalite": "0.19.0",
    "vitest": "4.0.15"
  }
}
  1. Create an eval that makes an HTTP request that fails (e.g., MCP client connecting to a server that isn't running):
// evals/example.eval.ts
import { evalite } from 'evalite'
import { Client } from '@modelcontextprotocol/sdk/client/index.js'
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js'

evalite('mcp-eval', {
  data: async () => [{ input: 'test' }],
  task: async () => {
    // This will throw: TypeError: fetch failed
    // Caused by: Error: connect ECONNREFUSED 127.0.0.1:443
    const client = new Client({ name: 'test', version: '1.0.0' })
    const transport = new StreamableHTTPClientTransport(
      new URL('https://localhost/mcp')
    )
    await client.connect(transport)
    return 'done'
  },
  scorers: [],
})
  1. Run npx evalite

  2. Instead of seeing the actual error:

TypeError: fetch failed
 ❯ StreamableHTTPClientTransport.send node_modules/@modelcontextprotocol/sdk/dist/esm/client/streamableHttp.js:315:30

Caused by: Error: connect ECONNREFUSED 127.0.0.1:443
Serialized Error: { errno: -61, code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 443 }

You get the unhelpful:

TypeError: Cannot read properties of undefined (reading 'pool')

Environment

  • evalite: 0.19.0
  • vitest: 4.0.15
  • Node: 24.x

Expected Behavior

Evalite should properly display the actual error from the failing eval task (the ECONNREFUSED error), not crash while trying to render it.

Suggested Fix

When calling vitest's printError / parseErrorStacktrace, ensure the task object has the required file property, or pass undefined for the task if file information is not available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions