Skip to content

Add Harness Evolver to Benchmark/Evaluator#135

Open
raphaelchristi wants to merge 1 commit intoJenqyang:mainfrom
raphaelchristi:add-harness-evolver
Open

Add Harness Evolver to Benchmark/Evaluator#135
raphaelchristi wants to merge 1 commit intoJenqyang:mainfrom
raphaelchristi:add-harness-evolver

Conversation

@raphaelchristi
Copy link
Copy Markdown

Summary

Adds Harness Evolver to the Benchmark/Evaluator section.

What it is

LangSmith-native autonomous agent optimization. Multi-agent proposers evolve prompts, routing, tools, and architecture in isolated git worktrees with LLM-as-judge evaluation and regression guards.

  • npm: npx harness-evolver@latest

@Jenqyang
Copy link
Copy Markdown
Owner

Decision: Not ready to merge yet.

Reason:

  1. The candidate may fit, but the current line does not clearly justify why this belongs in Benchmark/Evaluator rather than a tooling/optimization bucket.
  2. The README makes LangSmith a central runtime dependency, so under the updated standard we need that commercial dependency boundary to be framed more clearly.
  3. The wording should be tightened to a more neutral technical description.

Next step: Please clarify the section fit, make the LangSmith dependency explicit in neutral terms, and tighten the line, then we can re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants