Nudge synthesis agent to make use of search tools

[rajivnathan]

Nudge synthesis agent to make use of search tools to help with getting the latest context

Summary by CodeRabbit

• New Features

  • Synthesis agent can now perform internet searches to incorporate up-to-date external information into its outputs.

• Tests

  • Updated test expectations to validate inclusion of the internet search capability within the synthesis agent's generated instructions.

[coderabbitai]

Warning

Rate limit exceeded

@rajivnathan has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 8 minutes and 30 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

Walkthrough

The synthesis agent's custom instructions were extended to instruct the agent to use an internet search tool to gather the latest information related to alerts; unit and end-to-end expected prompt fixtures were updated to require the new "internet search" concept in the instructions.

Changes

Cohort / File(s)	Summary
Synthesis Agent Implementation backend/tarsy/agents/synthesis_agent.py	Augmented custom_instructions to instruct the agent to use an internet search tool to gather up-to-date information related to the original alert/issue while preserving existing synthesis and evaluation guidance.
Unit Test(s) — Synthesis Agent Config backend/tests/unit/config/test_synthesis_agent_config.py	Updated assertions to expect the additional key concept "internet search" in the SynthesisAgent custom_instructions, expanding the validated concept set.
E2E Expected Prompts / Fixtures backend/tests/e2e/expected_parallel_conversations.py	Added the explicit instruction to "Use the internet search tool to gather the latest information..." in three system prompt blocks to align expected conversations with the new external information-gathering step.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant Client as Client
participant Synth as Synthesis Agent
participant Search as Internet Search Tool
participant KB as Internal Knowledge Base
participant Investigator as Parallel Investigator Agents

Client->>Synth: Submit alert / issue
Synth->>KB: Retrieve existing context & evidence
Synth->>Investigator: Spawn parallel investigations (synthesize & evaluate)
Synth->>Search: Use internet search tool for latest info
Search-->>Synth: Return search results
Synth->>Investigator: Provide updated external context
Investigator-->>Synth: Collate findings & synthesize conclusions
Synth-->>Client: Return final synthesized report

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I hopped into prompts with a curious twitch,
Told the agent, "Go search—fetch the freshest stitch!"
Old facts now refreshed, the threads realigned,
Synthesis sings clearer, with internet aligned. 🥕🔍

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Nudge synthesis agent to make use of search tools' directly and clearly summarizes the main change: modifying the synthesis agent to use internet search tools for gathering current information.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

backend/tarsy/agents/synthesis_agent.py (1)

52-84: ⚠️ Potential issue | 🟠 Major

mcp_servers() returns an empty list, but custom_instructions() instructs the agent to use "the internet search tool".

The agent is configured to have no MCP servers (line 62 returns []; line 57 states "doesn't need tools"), yet the final instruction (line 84) tells the LLM to "Use the internet search tool." Per the coding guidelines, agents are restricted to their assigned MCP server subset during tool validation—the agent won't have access to any search tool at runtime, making this instruction unexecutable and likely to cause LLM hallucinations or confusion.

Either:

1. Add a search-capable MCP server to mcp_servers() and update the docstring, or
2. Remove the internet search instruction since synthesis is pure analysis of parallel investigation results.

🧹 Nitpick comments (1)

backend/tests/unit/config/test_synthesis_agent_config.py (1)

117-128: Test validates "internet search" concept but doesn't verify the agent can actually perform searches.

The test confirms the instructions mention "internet search," but given that mcp_servers() returns [], there's no test verifying the agent actually has access to a search tool. Consider adding a test that validates the agent's mcp_servers() includes a search server if the intent is for the agent to use one — otherwise this test is enshrining a broken contract.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.71%. Comparing base (74723cf) to head (1f8773c).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #271      +/-   ##
==========================================
- Coverage   83.74%   83.71%   -0.04%     
==========================================
  Files         143      143              
  Lines       13543    13543              
==========================================
- Hits        11342    11337       -5     
- Misses       2201     2206       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.