Mcp agent os optimization v2 16156909190644574428#6
Open
AGI-Corporation wants to merge 8 commits intomainfrom
Open
Mcp agent os optimization v2 16156909190644574428#6AGI-Corporation wants to merge 8 commits intomainfrom
AGI-Corporation wants to merge 8 commits intomainfrom
Conversation
This commit completes the integration of Mistral AI into the Agent OS framework: - **Mistral AI Provider**: Added a new provider for Mistral AI in the common AI framework, supporting chat completions and native tool-calling. - **Agent OS Defaults**: Updated the 'Optimal Workflow Agent' to use 'mistral-large-latest' as the default model. - **CactusRoute Integration**: Fully integrated adaptive routing layers (Difficulty, Repair, Semantic Validation, Deterministic Extraction) into the MCP Server tool execution flow. - **Guido Rule Engine**: Enhanced the virtual tool service with complex rule validation (nested paths, 'contains', negation). - **OpenAPI Tooling**: Added a prototype for importing OpenAPI specs as MCP tools. - **Discovery**: Ensured NANDA-compliant discovery and agent optimization via enhanced metadata and naming conventions. Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
This comprehensive update fully integrates the requested research-backed frameworks into the Activepieces MCP ecosystem, creating a robust "Agent OS": - **CactusRoute (Adaptive Routing)**: Implemented all 7 layers of the framework. Added `estimateDifficulty` (Layer 1), `repairOutput` (Layer 3), `semanticValidate` (Layer 4), and a full regex-based `deterministicExtract` fallback (Layer 7) in `cactus-utils.ts`. - **Guido (Rule Engine)**: Significantly enhanced the virtual tool validation logic to support nested object paths (e.g., 'Section.Key'), condition/target negation via `not: true`, and the `contains` state for flexible array and string matching. - **Mistral AI Integration**: Added a native Mistral AI provider supporting chat and tool-calling. Set 'mistral-large-latest' as the default for optimized agentic workflows. - **OpenAPI Tooling & Persistence**: Functionalized the OpenAPI import workflow. Added `VirtualToolEntity` and a database migration to persist blended and imported tools. Integrated virtual tools into the MCP server discovery and execution cycle. - **NANDA Protocol**: Updated the discovery manifest with explicit governance and compliance flags for AI-powered pieces. - **Security & Branding**: Enforced ownership checks across all MCP endpoints and polished the documentation with the new "Four Pillars of Agent OS" structure. - **Agent Optimization**: Refined tool naming conventions and metadata passing (aiDescription, examples) to maximize LLM tool-calling accuracy. Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
…nal Tools This milestone commit delivers the complete "Activepieces Agent OS" ecosystem: - **Branded Documentation**: Created a new `Agent OS` documentation hub in `docs/agent-os/` including: - `about.md`: Vision of the trillion-agent future. - `architecture.md`: Technical deep-dive with Mermaid diagrams for CactusRoute and NANDA flows. - `user-guide.md`: Step-by-step instructions for Virtual Tools and Guido rules. - **Enhanced Framework Manifesto**: Updated `AGENT_FRAMEWORK.md` with high-level architecture diagrams and the "Four Pillars" structure. - **Functional Virtual Tools**: - Implemented the `VirtualToolEntity` and database migrations for persistence. - Integrated the **Guido Rule Engine** (SET, CONTAINS, NOT, nested paths) into the tool execution cycle. - Functionalized the **OpenAPI Import** workflow to fetch, parse, and save tools from any specification URL. - **Robust Cactus Execution**: Completed the **7-Layer Adaptive Routing** logic in `cactus-utils.ts` with comprehensive regex-based semantic extraction (Location, Person, Message, etc.). - **Mistral Deep Integration**: Fully functional Mistral provider with native tool-calling and optimized default settings for Agentic workflows. - **NANDA Governance**: Updated manifest generation with compliance flags (GDPR_COMPLIANT_AI) for proper agent discovery. - **Enterprise Security**: Enforced ownership and multi-tenancy checks across all new MCP and Virtual Tool endpoints. Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
… Architecture Diagrams This commit marks the completion of the "Activepieces Agent OS" transformation: - **Functional Agentic Stack**: - **Cactus Adaptive Layer**: Fully implemented 7-layer routing with difficulty estimation, neuro-symbolic repair, and deterministic regex extraction. - **Guido Governance**: Integrated the rule engine into the tool execution cycle, enforcing safety via SET, CONTAINS, and NOT logic. - **OpenAPI Tooling**: Functional import workflow that persists OpenAPI schemas as usable MCP tools. - **Mistral Native**: Dedicated provider for Mistral AI with optimized tool-calling performance. - **System Architecture & Design**: - Added comprehensive Mermaid diagrams to `README.md` and `docs/` illustrating the multi-layer execution stack and NANDA discovery sequence. - Established the "Four Pillars" of Agent OS in the framework manifesto. - **Standardized Discovery**: - Implemented the NANDA protocol via `/.well-known/agent.json` for federated agent discovery and capability negotiation. - **Enterprise Grade**: - Enforced strict project-based ownership checks for all new MCP and Virtual Tool entities. - Added basic SSRF protection to URL-based imports. - **Documentation Hub**: - Created a complete documentation suite in `docs/agent-os/` covering vision, architecture, and a branded user guide. - **Consolidated Rebranding**: - Migrated 'AI Agent' capabilities into a unified 'Agent OS' piece for a seamless user experience. Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
📝 WalkthroughWalkthroughThis pull request rebrands the Agent OS framework to Route.X across documentation and package metadata, introduces Mistral AI provider integration, implements VirtualTool entity and service layer for managing blended and OpenAPI-imported tools, adds evaluation actions (RAG, hallucination detection, benchmarking), expands semantic extraction patterns, and enhances MCP server with new endpoints for tool creation and OpenAPI import workflows. Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client/Flow
participant MCPCtrl as MCP Controller
participant VTService as VirtualTool Service
participant OpenAPI as OpenAPI Spec (HTTP)
participant DB as Database
Client->>MCPCtrl: POST /:id/openapi-import (URL)
MCPCtrl->>MCPCtrl: Validate MCP & URL protocol
MCPCtrl->>OpenAPI: HTTP GET OpenAPI spec
OpenAPI-->>MCPCtrl: OpenAPI JSON response
MCPCtrl->>VTService: createToolsFromOpenApi(spec)
VTService->>VTService: Parse spec, build tool objects
VTService-->>MCPCtrl: Tool definitions array
MCPCtrl->>VTService: create() for each tool
VTService->>DB: INSERT virtual_tool records
DB-->>VTService: Persisted with id, timestamps
VTService-->>MCPCtrl: Tool creation results
MCPCtrl-->>Client: IMPORT_COMPLETED (toolsCount, tools[])
sequenceDiagram
participant Client as Client/Workflow
participant MCPServer as MCP Server
participant VTService as VirtualTool Service
participant GuidoEngine as Guido Rule Engine
participant ToolExec as Tool/HTTP Executor
participant Mistral as Mistral AI Client
Client->>MCPServer: Call virtual tool (params)
MCPServer->>VTService: validateBlendedData(params, ruleSets)
VTService->>GuidoEngine: Apply rule validation (SET, CONTAINS, etc.)
GuidoEngine-->>VTService: Validation result
alt Validation Passes
alt OpenAPI Tool
MCPServer->>ToolExec: HTTP request (method, url, params)
ToolExec-->>MCPServer: Response body
else Base Action
MCPServer->>ToolExec: Execute base actions
ToolExec-->>MCPServer: Execution result
end
MCPServer->>MCPServer: Check evaluation trigger
opt Auto-Evaluate
MCPServer->>Mistral: Chat completion (system prompt, output)
Mistral-->>MCPServer: Evaluation JSON (relevance, groundedness)
end
else Validation Fails
MCPServer-->>Client: Error response
end
MCPServer-->>Client: Tool result (JSON code block)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ❌ 3❌ Failed checks (2 warnings, 1 inconclusive)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2fbd7a6160
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| export const mcpServerController: FastifyPluginAsyncTypebox = async (app) => { | ||
|
|
||
| app.addHook('preHandler', entitiesMustBeOwnedByCurrentProject) |
There was a problem hiding this comment.
entitiesMustBeOwnedByCurrentProject is implemented as a pre-serialization hook (request, reply, payload, done), but this commit also registers it as a preHandler. Fastify preHandler hooks receive only (request, reply, done), so done is passed into the payload slot and the final done() call is undefined, causing runtime failures before route handlers execute for this controller.
Useful? React with 👍 / 👎.
Comment on lines
+3
to
+4
| export class AddVirtualToolTable1745000000000 implements MigrationInterface { | ||
| name = 'AddVirtualToolTable1745000000000' |
There was a problem hiding this comment.
This migration class is added but never referenced by the configured migration lists (a repo-wide search for AddVirtualToolTable1745000000000 only finds this file), so environments running with synchronize=false will not create virtual_tool and runtime queries against it will fail with relation/table-not-found errors.
Useful? React with 👍 / 👎.
Comment on lines
+67
to
+71
| return { | ||
| content: [{ | ||
| type: 'text', | ||
| text: `✅ Executed ${vt.metadata.method} ${vt.metadata.url}\n\n` + | ||
| `\`\`\`json\n${JSON.stringify({ params }, null, 2)}\n\`\`\``, |
There was a problem hiding this comment.
For OPENAPI virtual tools, the handler returns a success message and echoes params but never performs an HTTP request to vt.metadata.url, so imported tools report Executed ... even though no external API call happens. This silently drops expected side effects and can mislead users into thinking operations completed.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 11
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/server/api/src/app/mcp/virtual-tool-service.ts (1)
97-100:⚠️ Potential issue | 🟠 MajorFilter path item keys to HTTP methods before generating tools.
Current iteration treats every key under a path as an operation. Non-method keys can produce bogus tools and invalid metadata.
🐛 Suggested fix
+ const supportedMethods = new Set(['get', 'post', 'put', 'patch', 'delete', 'head', 'options', 'trace']) for (const [path, methods] of Object.entries(paths)) { for (const [method, operation] of Object.entries(methods as any)) { + if (!supportedMethods.has(method.toLowerCase())) { + continue + } const op = operation as any const name = op.operationId || `${method}_${path.replace(/\//g, '_')}`Also applies to: 124-133
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/virtual-tool-service.ts` around lines 97 - 100, The loop that iterates Object.entries(paths) and then Object.entries(methods) (variables paths, method, operation, op, name) treats every path item key as an HTTP operation; filter the keys to only standard HTTP methods (e.g., GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD) before creating tools and computing operationId/name to avoid generating bogus tools and metadata. Update the first loop (where name is built from op.operationId or `${method}_${path.replace(/\//g, '_')}`) and the similar loop later (lines around the second path iteration) to skip non-method keys by checking method.toLowerCase() is in the allowed-methods set before using operation/op to generate tools/metadata.
🧹 Nitpick comments (5)
docs/agent-os/user-guide.md (1)
41-44: Add an explicit “token is secret” warning in Discovery section.This section mentions sharing and registration, but it should clearly state that the Discovery Token is sensitive credential material.
📝 Suggested doc tweak
Every Agent OS project has a unique **Discovery Token**. +> ⚠️ Treat this token like a secret credential. Anyone with it can invoke your MCP endpoint. - You can find your token in the **MCP Settings** page. - Use this token to register your agent in external indexes or to share it with your team. - If your token is compromised, use the **Rotate Token** button to instantly invalidate the old one.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/agent-os/user-guide.md` around lines 41 - 44, Add an explicit security warning to the Discovery Token section: state that the Discovery Token is a sensitive secret (treat like a password/API key), must not be posted in public channels or shared casually, should be stored in a secure secrets manager, and rotated immediately if believed compromised using the "Rotate Token" button referenced in the text; place this warning directly after the sentence that says "Every Agent OS project has a unique Discovery Token" and make it visually prominent (e.g., a caution callout) near references to "MCP Settings" and "Rotate Token".docs/agent-os/architecture.md (1)
23-27: Minor readability polish for the layer bullets.The four bullets read repetitive due repeated “Layer X” lead-in; slight rewording would improve scanability.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/agent-os/architecture.md` around lines 23 - 27, Reword the four bullets to remove the repetitive "Layer X" lead-ins by making each line start with a concise action or purpose followed by the layer label in parentheses (e.g., "Assess query difficulty (Layer 1): ..." or "Format corrections (Layer 3): ..."); update the entries for "Layer 1", "Layer 3", "Layer 4", and "Layer 7" so they each begin with a short verb phrase describing the responsibility and then include the existing explanatory text—this improves scanability while preserving the original meanings.docs/agent-os/about.md (1)
3-3: Qualify absolute performance/research claims with scope or citations.Phrases like “world’s first,” “99% reliability,” and named research references should include benchmark scope/citations or be softened to avoid overclaiming.
Also applies to: 28-28
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/agent-os/about.md` at line 3, The bold absolute claims in the opening sentence (e.g., "world's first", "99% reliability", and implied research backing around "Activepieces Agent OS") must be qualified or supported: replace "world's first" with a softer phrase like "one of the first" or "a research-backed", change "99% reliability" to a scoped claim such as "up to 99% reliability in internal benchmarks" or "99% reliability in X benchmark" and add citation links or footnotes to the research/benchmarks referenced; update the same phrasing at the other occurrence (line with duplicate claim) so both are consistent and include either a citation list or clear scope for the metric.packages/server/api/src/app/database/migration/postgres/1745000000000-AddVirtualToolTable.ts (1)
25-31: Align index name with entity schema to avoid migration drift.The migration creates
idx_virtual_tool_mcp_id, whilepackages/server/api/src/app/mcp/virtual-tool-entity.tsdeclaresvirtual_tool_mcp_id. Keeping names aligned avoids duplicate-index churn in future generated migrations.♻️ Suggested adjustment
- CREATE INDEX "idx_virtual_tool_mcp_id" ON "virtual_tool" ("mcpId") + CREATE INDEX "virtual_tool_mcp_id" ON "virtual_tool" ("mcpId")- DROP INDEX "idx_virtual_tool_mcp_id" + DROP INDEX "virtual_tool_mcp_id"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/database/migration/postgres/1745000000000-AddVirtualToolTable.ts` around lines 25 - 31, The migration's index name is inconsistent with the entity: update the index name used in the AddVirtualToolTable migration so it matches the entity's declared index "virtual_tool_mcp_id" (change both the CREATE INDEX in the up method and the DROP INDEX in the down method to use "virtual_tool_mcp_id"), ensuring the same quoted identifier is used in both QueryRunner.query calls to prevent future migration drift.packages/server/api/src/app/mcp/virtual-tool-service.ts (1)
12-13: Consider replacinganywith typed DTOs for virtual tool persistence/import.Typed inputs/outputs here would make malformed OpenAPI-derived tool shapes fail fast at compile time instead of at runtime.
Also applies to: 92-93, 102-107
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/virtual-tool-service.ts` around lines 12 - 13, The create(data: any) signature (and other functions in this file that accept untyped any) should be changed to accept a properly typed DTO so malformed OpenAPI-derived tool shapes fail at compile time: define a VirtualToolDto (or more granular DTOs) matching your persistence shape, update the method signature from create(data: any) to create(data: VirtualToolDto), and replace any direct use of untyped properties with the typed fields; ensure repo().save is passed the correctly typed entity (or map DTO -> Entity) and add runtime validation with class-validator/class-transformer or a schema check if needed for extra safety.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/pieces/community/common/src/lib/ai/providers/mistral/index.ts`:
- Around line 63-76: The JSON.parse on toolCall.function.arguments can throw and
should be guarded; wrap parsing of toolCall.function.arguments in a try/catch
(inside the response mapping where choices and call are constructed — look for
toolCall, response.body.choices and AIChatRole.ASSISTANT) and on parse failure
return a safe fallback (e.g., null or the raw string) for the arguments and
optionally log/debug the parse error, ensuring a malformed tool-call does not
break the entire response handling path.
In `@packages/server/api/src/app/ai/cactus-utils.ts`:
- Around line 77-82: LOCATION_RE and PERSON_RE are too strict (only Title Case)
and the other patterns greedily over-capture when quotes are absent; update
LOCATION_RE and PERSON_RE to be case-insensitive and accept lowercase/multi-word
names (e.g., switch to a case-insensitive class like [A-Za-z]+ or Unicode \p{L}+
with the u and i flags on LOCATION_RE and PERSON_RE), and make MESSAGE_RE,
TITLE_RE, SONG_RE, QUERY_RE either require quotes for multi-word values or
change their captures to non-greedy with a terminating lookahead to common
separators (for example use a non-greedy capture and a lookahead like
(?=\s+(?:and|then|also|for|with|to|$)) so they stop at the next keyword or
end-of-string); update the six constants (LOCATION_RE, PERSON_RE, MESSAGE_RE,
TITLE_RE, SONG_RE, QUERY_RE) accordingly.
In `@packages/server/api/src/app/mcp/mcp-server-controller.ts`:
- Around line 57-67: The POST handlers for '/:id/blended-tools' (using
CreateBlendedToolRequest and virtualToolService.create) and
'/:id/openapi-import' trust req.params.id without verifying the caller owns that
MCP; add the existing ownership assertion used elsewhere (e.g.,
assertMcpOwnership(req, mcpId) or the project/mcp ownership helper used in other
controllers) immediately after extracting const mcpId = req.params.id and before
any call to virtualToolService.create or fetching/parsing remote specs so the
request is rejected if the MCP does not belong to the caller's project.
- Around line 75-83: Replace the weak startsWith check with robust URL
validation: parse the incoming url with the URL constructor, enforce protocol
=== 'http:' || protocol === 'https:', reject any URLs containing credentials
(username/password), and normalize/validate hostname by resolving it
(dns.lookup) and checking the resolved IP(s) against deny-ranges (localhost,
127.0.0.0/8, ::1, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, link-local, etc.)
or an explicit allowlist; also ensure no automatic following of redirects by
configuring httpClient.sendRequest to disable redirects or validate Location
headers before following. Update the code around the url variable and the
httpClient.sendRequest call (and HttpMethod.GET usage) so any disallowed URL is
rejected before calling sendRequest.
- Around line 57-68: The POST handler for blended tool creation currently
returns a value with the default 200 status because it only uses the (req)
parameter; update the handler signature to (req, reply) and set the response
status to 201 before returning the created resource (e.g., call
reply.code(201).send(...)) in the route defined as
app.post('/:id/blended-tools', CreateBlendedToolRequest, async (req) => { ... })
and make the same change to the other matching handler referenced (the similar
POST route around the second occurrence).
- Around line 11-12: Remove the incorrect preHandler hook registration: the
function entitiesMustBeOwnedByCurrentProject is typed as a
preSerializationHookHandler and must only be registered with
app.addHook('preSerialization', entitiesMustBeOwnedByCurrentProject); delete the
line that registers it with app.addHook('preHandler', ...) so the handler is
only attached as preSerialization to avoid the runtime signature mismatch.
In `@packages/server/api/src/app/mcp/mcp-server.ts`:
- Around line 53-55: The duplicate-checking logic uses the wrong key: you
register tools with a truncated/prefixed name via server.tool(...) but check and
store duplicates using action.name and raw vt.name, allowing collisions. Fix by
computing the actual registration key (e.g., registeredName = vt.name.slice(0,
MAX_TOOL_NAME_LENGTH) plus any prefix used) and use that single registeredName
for the duplicate check, for storing in any maps/sets, and when calling
server.tool; update the same logic at the other spots referenced (lines around
84-85 and 89-99) so action.name or raw vt.name are not used for uniqueness tests
or storage—always use the computed registeredName consistently.
- Around line 65-74: The OPENAPI branch in mcp-server.ts currently returns a
mock success; replace the mock with actual HTTP call logic: detect OPENAPI via
vt.metadata.type and use vt.metadata.method and vt.metadata.url plus the params
payload to perform a fetch/axios request, await the response, parse JSON (or
text) and include the real response body and HTTP status in the returned
content, and handle errors by returning a content item with an error message and
status; update any function handling (e.g., the caller that expects the returned
shape) to propagate response data instead of the hardcoded JSON string.
In `@packages/server/api/src/app/mcp/nanda-manifest-service.ts`:
- Around line 67-70: The compliance array currently derives flags by
heuristically matching pieceName substrings (in the enabledPieces logic) which
is fragile; update the logic in nanda-manifest-service.ts to read an explicit
vetted metadata/policy field on each piece (e.g., piece.metadata.complianceFlags
or piece.policyFlags) instead of using pieceName.includes, collect flags
deterministically from those fields (falling back to empty if missing),
deduplicate the resulting list, and use that deduped list for the compliance
property so HIPAA/GDPR/GDPR_COMPLIANT_AI are only set when explicitly declared.
In `@packages/server/api/src/app/mcp/virtual-tool-entity.ts`:
- Line 12: Update the documentation example for blended_tool_config in
optimal-workflow.ts to use the API's strict field names so examples match the
schema: change any example objects like { 'piece': 'gmail', 'action':
'send_email' } to { 'pieceName': 'gmail', 'actionName': 'send_email' } so they
align with the mcp-server-controller.ts schema and the type referenced in
virtual-tool-entity.ts (baseActions / blended_tool_config).
In `@README.md`:
- Line 68: Update the README text that currently reads "open source MCP toolkit"
to use a hyphenated compound adjective: change it to "open-source MCP toolkit"
so the phrase before "MCP toolkit" is grammatically correct; locate the string
"open source MCP toolkit" in the README and replace it with "open-source MCP
toolkit".
---
Outside diff comments:
In `@packages/server/api/src/app/mcp/virtual-tool-service.ts`:
- Around line 97-100: The loop that iterates Object.entries(paths) and then
Object.entries(methods) (variables paths, method, operation, op, name) treats
every path item key as an HTTP operation; filter the keys to only standard HTTP
methods (e.g., GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD) before creating
tools and computing operationId/name to avoid generating bogus tools and
metadata. Update the first loop (where name is built from op.operationId or
`${method}_${path.replace(/\//g, '_')}`) and the similar loop later (lines
around the second path iteration) to skip non-method keys by checking
method.toLowerCase() is in the allowed-methods set before using operation/op to
generate tools/metadata.
---
Nitpick comments:
In `@docs/agent-os/about.md`:
- Line 3: The bold absolute claims in the opening sentence (e.g., "world's
first", "99% reliability", and implied research backing around "Activepieces
Agent OS") must be qualified or supported: replace "world's first" with a softer
phrase like "one of the first" or "a research-backed", change "99% reliability"
to a scoped claim such as "up to 99% reliability in internal benchmarks" or "99%
reliability in X benchmark" and add citation links or footnotes to the
research/benchmarks referenced; update the same phrasing at the other occurrence
(line with duplicate claim) so both are consistent and include either a citation
list or clear scope for the metric.
In `@docs/agent-os/architecture.md`:
- Around line 23-27: Reword the four bullets to remove the repetitive "Layer X"
lead-ins by making each line start with a concise action or purpose followed by
the layer label in parentheses (e.g., "Assess query difficulty (Layer 1): ..."
or "Format corrections (Layer 3): ..."); update the entries for "Layer 1",
"Layer 3", "Layer 4", and "Layer 7" so they each begin with a short verb phrase
describing the responsibility and then include the existing explanatory
text—this improves scanability while preserving the original meanings.
In `@docs/agent-os/user-guide.md`:
- Around line 41-44: Add an explicit security warning to the Discovery Token
section: state that the Discovery Token is a sensitive secret (treat like a
password/API key), must not be posted in public channels or shared casually,
should be stored in a secure secrets manager, and rotated immediately if
believed compromised using the "Rotate Token" button referenced in the text;
place this warning directly after the sentence that says "Every Agent OS project
has a unique Discovery Token" and make it visually prominent (e.g., a caution
callout) near references to "MCP Settings" and "Rotate Token".
In
`@packages/server/api/src/app/database/migration/postgres/1745000000000-AddVirtualToolTable.ts`:
- Around line 25-31: The migration's index name is inconsistent with the entity:
update the index name used in the AddVirtualToolTable migration so it matches
the entity's declared index "virtual_tool_mcp_id" (change both the CREATE INDEX
in the up method and the DROP INDEX in the down method to use
"virtual_tool_mcp_id"), ensuring the same quoted identifier is used in both
QueryRunner.query calls to prevent future migration drift.
In `@packages/server/api/src/app/mcp/virtual-tool-service.ts`:
- Around line 12-13: The create(data: any) signature (and other functions in
this file that accept untyped any) should be changed to accept a properly typed
DTO so malformed OpenAPI-derived tool shapes fail at compile time: define a
VirtualToolDto (or more granular DTOs) matching your persistence shape, update
the method signature from create(data: any) to create(data: VirtualToolDto), and
replace any direct use of untyped properties with the typed fields; ensure
repo().save is passed the correctly typed entity (or map DTO -> Entity) and add
runtime validation with class-validator/class-transformer or a schema check if
needed for extra safety.
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (17)
AGENT_FRAMEWORK.mdREADME.mddocs/agent-os/about.mddocs/agent-os/architecture.mddocs/agent-os/user-guide.mddocs/mint.jsonpackages/pieces/community/ai-agent/src/lib/actions/optimal-workflow.tspackages/pieces/community/common/src/lib/ai/providers/index.tspackages/pieces/community/common/src/lib/ai/providers/mistral/index.tspackages/server/api/src/app/ai/cactus-utils.tspackages/server/api/src/app/database/database-connection.tspackages/server/api/src/app/database/migration/postgres/1745000000000-AddVirtualToolTable.tspackages/server/api/src/app/mcp/mcp-server-controller.tspackages/server/api/src/app/mcp/mcp-server.tspackages/server/api/src/app/mcp/nanda-manifest-service.tspackages/server/api/src/app/mcp/virtual-tool-entity.tspackages/server/api/src/app/mcp/virtual-tool-service.ts
Comment on lines
+63
to
+76
| const toolCall = response.body.choices[0].message.tool_calls?.[0]; | ||
|
|
||
| return { | ||
| choices: response.body.choices.map((choice: any) => ({ | ||
| role: AIChatRole.ASSISTANT, | ||
| content: choice.message.content || '', | ||
| })), | ||
| call: toolCall | ||
| ? { | ||
| id: toolCall.id, | ||
| function: { | ||
| name: toolCall.function.name, | ||
| arguments: JSON.parse(toolCall.function.arguments), | ||
| }, |
There was a problem hiding this comment.
Guard JSON.parse for tool-call arguments to avoid hard failures.
JSON.parse can throw if arguments are malformed. A single malformed model output currently breaks the whole response handling path.
🛡️ Suggested fix
- const toolCall = response.body.choices[0].message.tool_calls?.[0];
+ const toolCall = response.body.choices[0].message.tool_calls?.[0];
+ let parsedArguments: unknown = {}
+ if (toolCall?.function?.arguments) {
+ try {
+ parsedArguments = JSON.parse(toolCall.function.arguments)
+ } catch {
+ parsedArguments = toolCall.function.arguments
+ }
+ }
return {
choices: response.body.choices.map((choice: any) => ({
role: AIChatRole.ASSISTANT,
content: choice.message.content || '',
})),
call: toolCall
? {
id: toolCall.id,
function: {
name: toolCall.function.name,
- arguments: JSON.parse(toolCall.function.arguments),
+ arguments: parsedArguments,
},
}
: null,🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/pieces/community/common/src/lib/ai/providers/mistral/index.ts`
around lines 63 - 76, The JSON.parse on toolCall.function.arguments can throw
and should be guarded; wrap parsing of toolCall.function.arguments in a
try/catch (inside the response mapping where choices and call are constructed —
look for toolCall, response.body.choices and AIChatRole.ASSISTANT) and on parse
failure return a safe fallback (e.g., null or the raw string) for the arguments
and optionally log/debug the parse error, ensuring a malformed tool-call does
not break the entire response handling path.
Comment on lines
+77
to
+82
| const LOCATION_RE = /(?:in|at|for)\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)/; | ||
| const PERSON_RE = /(?:to|send|contact)\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)/; | ||
| const MESSAGE_RE = /(?:saying|message|body|text)\s+["']?([^"']+)["']?/; | ||
| const TITLE_RE = /(?:title|note|task|reminder)\s+["']?([^"']+)["']?/; | ||
| const SONG_RE = /(?:play|song|track)\s+["']?([^"']+)["']?/; | ||
| const QUERY_RE = /(?:search|find|query)\s+["']?([^"']+)["']?/; |
There was a problem hiding this comment.
Case sensitivity and greedy capture issues in new regex patterns.
Two problems:
-
LOCATION_REandPERSON_RErequire title case ([A-Z][a-z]+), so inputs like"send john a message"or"weather in paris"won't match. The existingTIME_REuses theiflag for case insensitivity. -
MESSAGE_RE,TITLE_RE,SONG_RE,QUERY_REover-capture when quotes are absent. The pattern[^"']+greedily consumes everything to end-of-string. For example,"message hello world then set alarm"extracts"hello world then set alarm"instead of just"hello world".
🐛 Proposed fix for regex patterns
-const LOCATION_RE = /(?:in|at|for)\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)/;
-const PERSON_RE = /(?:to|send|contact)\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)/;
-const MESSAGE_RE = /(?:saying|message|body|text)\s+["']?([^"']+)["']?/;
-const TITLE_RE = /(?:title|note|task|reminder)\s+["']?([^"']+)["']?/;
-const SONG_RE = /(?:play|song|track)\s+["']?([^"']+)["']?/;
-const QUERY_RE = /(?:search|find|query)\s+["']?([^"']+)["']?/;
+const LOCATION_RE = /(?:in|at|for)\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)/i;
+const PERSON_RE = /(?:to|send|contact)\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)/i;
+const MESSAGE_RE = /(?:saying|message|body|text)\s+(?:["']([^"']+)["']|(\S+(?:\s+\S+)*))/;
+const TITLE_RE = /(?:title|note|task|reminder)\s+(?:["']([^"']+)["']|(\S+(?:\s+\S+)*))/;
+const SONG_RE = /(?:play|song|track)\s+(?:["']([^"']+)["']|(\S+(?:\s+\S+)*))/;
+const QUERY_RE = /(?:search|find|query)\s+(?:["']([^"']+)["']|(\S+(?:\s+\S+)*))/;Note: The greedy capture fix is complex because determining boundaries without quotes is inherently ambiguous. Consider requiring quotes for multi-word values, or using a more sophisticated approach (e.g., stopping at known keywords like and, then, also).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/ai/cactus-utils.ts` around lines 77 - 82,
LOCATION_RE and PERSON_RE are too strict (only Title Case) and the other
patterns greedily over-capture when quotes are absent; update LOCATION_RE and
PERSON_RE to be case-insensitive and accept lowercase/multi-word names (e.g.,
switch to a case-insensitive class like [A-Za-z]+ or Unicode \p{L}+ with the u
and i flags on LOCATION_RE and PERSON_RE), and make MESSAGE_RE, TITLE_RE,
SONG_RE, QUERY_RE either require quotes for multi-word values or change their
captures to non-greedy with a terminating lookahead to common separators (for
example use a non-greedy capture and a lookahead like
(?=\s+(?:and|then|also|for|with|to|$)) so they stop at the next keyword or
end-of-string); update the six constants (LOCATION_RE, PERSON_RE, MESSAGE_RE,
TITLE_RE, SONG_RE, QUERY_RE) accordingly.
Comment on lines
+11
to
12
| app.addHook('preHandler', entitiesMustBeOwnedByCurrentProject) | ||
| app.addHook('preSerialization', entitiesMustBeOwnedByCurrentProject) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Hook registrations in MCP controller ==="
rg -n "addHook" packages/server/api/src/app/mcp/mcp-server-controller.ts
echo
echo "=== Hook function signature in authorization module ==="
rg -n "export const entitiesMustBeOwnedByCurrentProject" -A 20 packages/server/api/src/app/authentication/authorization.tsRepository: AGI-Corporation/mcpmyid
Length of output: 1250
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== All registrations of entitiesMustBeOwnedByCurrentProject ==="
rg -n "entitiesMustBeOwnedByCurrentProject" --type ts
echo
echo "=== Check if entitiesMustBeOwnedByCurrentProject is imported elsewhere ==="
rg -n "import.*entitiesMustBeOwnedByCurrentProject" --type tsRepository: AGI-Corporation/mcpmyid
Length of output: 6938
Remove preHandler hook registration on line 11—function is typed for preSerialization only.
entitiesMustBeOwnedByCurrentProject is defined as preSerializationHookHandler with signature (request, _response, payload, done). Registering it as preHandler causes a runtime error: the function receives (request, reply, done) but expects the payload parameter at position 3. All other usages in the codebase correctly register this function as preSerialization. Line 11 should be removed; line 12 is correct.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/mcp-server-controller.ts` around lines 11 -
12, Remove the incorrect preHandler hook registration: the function
entitiesMustBeOwnedByCurrentProject is typed as a preSerializationHookHandler
and must only be registered with app.addHook('preSerialization',
entitiesMustBeOwnedByCurrentProject); delete the line that registers it with
app.addHook('preHandler', ...) so the handler is only attached as
preSerialization to avoid the runtime signature mismatch.
Comment on lines
57
to
+67
| app.post('/:id/blended-tools', CreateBlendedToolRequest, async (req) => { | ||
| // Implementation for saving user-created blended tools | ||
| return { | ||
| status: 'CREATED', | ||
| name: req.body.name, | ||
| id: apId(), | ||
| } | ||
| const mcpId = req.params.id | ||
| const { name, description, baseActions, ruleSets } = req.body | ||
|
|
||
| return virtualToolService(req.log).create({ | ||
| mcpId, | ||
| name, | ||
| description, | ||
| baseActions, | ||
| ruleSets: ruleSets ?? [], | ||
| }) |
There was a problem hiding this comment.
Missing request-time ownership checks on :id mutation endpoints.
Both endpoints trust req.params.id without validating it belongs to the caller’s project. This allows cross-project virtual tool creation/import if an MCP ID is known.
🔒 Suggested fix pattern
app.post('/:id/blended-tools', CreateBlendedToolRequest, async (req) => {
const mcpId = req.params.id
+ const mcp = await mcpService(req.log).getOrThrow({ mcpId })
+ if (req.principal.type !== PrincipalType.SERVICE && mcp.projectId !== req.principal.projectId) {
+ throw new Error('Not authorized to modify this MCP')
+ }
const { name, description, baseActions, ruleSets } = req.body
return virtualToolService(req.log).create({
mcpId,
name,
description,
baseActions,
ruleSets: ruleSets ?? [],
})
})Apply the same ownership assertion to /:id/openapi-import before fetching/parsing remote specs.
Also applies to: 71-73
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/mcp-server-controller.ts` around lines 57 -
67, The POST handlers for '/:id/blended-tools' (using CreateBlendedToolRequest
and virtualToolService.create) and '/:id/openapi-import' trust req.params.id
without verifying the caller owns that MCP; add the existing ownership assertion
used elsewhere (e.g., assertMcpOwnership(req, mcpId) or the project/mcp
ownership helper used in other controllers) immediately after extracting const
mcpId = req.params.id and before any call to virtualToolService.create or
fetching/parsing remote specs so the request is rejected if the MCP does not
belong to the caller's project.
Comment on lines
57
to
68
| app.post('/:id/blended-tools', CreateBlendedToolRequest, async (req) => { | ||
| // Implementation for saving user-created blended tools | ||
| return { | ||
| status: 'CREATED', | ||
| name: req.body.name, | ||
| id: apId(), | ||
| } | ||
| const mcpId = req.params.id | ||
| const { name, description, baseActions, ruleSets } = req.body | ||
|
|
||
| return virtualToolService(req.log).create({ | ||
| mcpId, | ||
| name, | ||
| description, | ||
| baseActions, | ||
| ruleSets: ruleSets ?? [], | ||
| }) | ||
| }) |
There was a problem hiding this comment.
Response status code mismatches declared schema for blended tool creation.
Route schema declares 201 Created, but the handler returns default 200 OK.
✅ Suggested fix
- app.post('/:id/blended-tools', CreateBlendedToolRequest, async (req) => {
+ app.post('/:id/blended-tools', CreateBlendedToolRequest, async (req, reply) => {
...
- return virtualToolService(req.log).create({
+ const created = await virtualToolService(req.log).create({
...
})
+ return reply.status(StatusCodes.CREATED).send(created)
})Also applies to: 166-167
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/mcp-server-controller.ts` around lines 57 -
68, The POST handler for blended tool creation currently returns a value with
the default 200 status because it only uses the (req) parameter; update the
handler signature to (req, reply) and set the response status to 201 before
returning the created resource (e.g., call reply.code(201).send(...)) in the
route defined as app.post('/:id/blended-tools', CreateBlendedToolRequest, async
(req) => { ... }) and make the same change to the other matching handler
referenced (the similar POST route around the second occurrence).
Comment on lines
+53
to
+55
| server.tool( | ||
| vt.name.slice(0, MAX_TOOL_NAME_LENGTH), | ||
| vt.description, |
There was a problem hiding this comment.
Tool de-duplication checks the wrong key, so name collisions can slip through.
You register by truncated/tool-prefixed names, but check duplicates using action.name and store raw vt.name. This can still register duplicate tool names and break tool registration at runtime.
🐛 Suggested fix
for (const vt of virtualTools) {
+ const virtualToolName = vt.name.slice(0, MAX_TOOL_NAME_LENGTH)
+ if (uniqueActions.has(virtualToolName)) {
+ logger.warn({ virtualToolName }, '[McpServer] Duplicate virtual tool name, skipping registration')
+ continue
+ }
server.tool(
- vt.name.slice(0, MAX_TOOL_NAME_LENGTH),
+ virtualToolName,
vt.description,
Object.fromEntries(
Object.entries(vt.props || {}).map(([key, prop]: [string, any]) =>
[key, piecePropertyToZod(prop)],
),
),
async (params) => {
...
}
)
- uniqueActions.add(vt.name)
+ uniqueActions.add(virtualToolName)
}
pieces.flatMap(piece => {
return Object.values(piece.actions).map(action => {
- if (uniqueActions.has(action.name)) {
- return
- }
-
const shortPieceName = piece.name.replace('@activepieces/piece-', '')
const actionName = `${shortPieceName}_${action.name}`.slice(0, MAX_TOOL_NAME_LENGTH)
+ if (uniqueActions.has(actionName)) {
+ return
+ }
uniqueActions.add(actionName)Also applies to: 84-85, 89-99
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/mcp-server.ts` around lines 53 - 55, The
duplicate-checking logic uses the wrong key: you register tools with a
truncated/prefixed name via server.tool(...) but check and store duplicates
using action.name and raw vt.name, allowing collisions. Fix by computing the
actual registration key (e.g., registeredName = vt.name.slice(0,
MAX_TOOL_NAME_LENGTH) plus any prefix used) and use that single registeredName
for the duplicate check, for storing in any maps/sets, and when calling
server.tool; update the same logic at the other spots referenced (lines around
84-85 and 89-99) so action.name or raw vt.name are not used for uniqueness tests
or storage—always use the computed registeredName consistently.
Comment on lines
+67
to
+70
| compliance: [ | ||
| ...(enabledPieces.some(p => p.pieceName.includes('fhir')) ? ['HIPAA', 'GDPR_HEALTH'] : []), | ||
| ...(enabledPieces.some(p => p.pieceName.includes('mistral') || p.pieceName.includes('ai-agent')) ? ['GDPR_COMPLIANT_AI'] : []), | ||
| ], |
There was a problem hiding this comment.
Compliance classification is currently heuristic and can overstate legal posture.
Inferring HIPAA/GDPR flags from pieceName.includes(...) is fragile and may emit incorrect compliance claims. This should come from explicit vetted metadata/policy flags, not string matching.
🛠️ Safer interim pattern (deterministic + deduplicated)
+ const normalizedPieceNames = new Set(
+ enabledPieces.map((p) => p.pieceName.toLowerCase()),
+ )
+ const compliance = new Set<string>()
+ if (normalizedPieceNames.has('fhir')) {
+ compliance.add('HIPAA')
+ compliance.add('GDPR_HEALTH')
+ }
+ if (normalizedPieceNames.has('mistral') || normalizedPieceNames.has('ai-agent')) {
+ compliance.add('GDPR_COMPLIANT_AI')
+ }
+
return {
@@
governance: {
data_retention: 'ZERO_RETENTION_PREFERENCE',
human_in_loop: 'REQUIRED_FOR_SENSITIVE',
- compliance: [
- ...(enabledPieces.some(p => p.pieceName.includes('fhir')) ? ['HIPAA', 'GDPR_HEALTH'] : []),
- ...(enabledPieces.some(p => p.pieceName.includes('mistral') || p.pieceName.includes('ai-agent')) ? ['GDPR_COMPLIANT_AI'] : []),
- ],
+ compliance: Array.from(compliance),
},🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/nanda-manifest-service.ts` around lines 67 -
70, The compliance array currently derives flags by heuristically matching
pieceName substrings (in the enabledPieces logic) which is fragile; update the
logic in nanda-manifest-service.ts to read an explicit vetted metadata/policy
field on each piece (e.g., piece.metadata.complianceFlags or piece.policyFlags)
instead of using pieceName.includes, collect flags deterministically from those
fields (falling back to empty if missing), deduplicate the resulting list, and
use that deduped list for the compliance property so
HIPAA/GDPR/GDPR_COMPLIANT_AI are only set when explicitly declared.
| mcpId: ApId | ||
| name: string | ||
| description: string | ||
| baseActions: { pieceName: string, actionName: string }[] |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Step 1: Locate and inspect virtual-tool-entity.ts ==="
fd 'virtual-tool-entity\.ts$' -t f
echo
echo "=== Step 2: Search for baseActions usage in service and controller ==="
fd 'virtual-tool-service\.ts$' -t f -x cat -n
fd 'mcp-server-controller\.ts$' -t f -x wc -l
echo
echo "=== Step 3: Search for baseActions across codebase ==="
rg -n 'baseActions' -t ts --max-count 20
echo
echo "=== Step 4: Search for pieceName and actionName usage ==="
rg -n '(pieceName|actionName)' -t ts --max-count 20
echo
echo "=== Step 5: Search for piece and action keys in examples ==="
rg -n '"(piece|action)"' -t ts -A 2 -B 2 | head -50Repository: AGI-Corporation/mcpmyid
Length of output: 50379
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== virtual-tool-entity.ts (full file) ==="
cat -n packages/server/api/src/app/mcp/virtual-tool-entity.ts
echo
echo "=== mcp-server-controller.ts schema around line 159 ==="
sed -n '50,170p' packages/server/api/src/app/mcp/mcp-server-controller.ts | cat -n
echo
echo "=== Search for JSON examples with piece/action (short form) ==="
rg -A 3 -B 3 '"piece"\s*:' packages/server/api/src/app/mcp/ --type ts 2>/dev/null || echo "No matches found"
echo
echo "=== Check blended_tool_config or similar user guide examples ==="
fd -t f -e md -e ts | xargs rg -l 'optimal.*workflow|blended.*tool|user.*guide' 2>/dev/null | head -5 || echo "No user guides found"Repository: AGI-Corporation/mcpmyid
Length of output: 6774
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Check optimal-workflow.ts for piece/action keys ==="
cat packages/pieces/community/ai-agent/src/lib/actions/optimal-workflow.ts | grep -A 5 -B 5 'piece\|action' | head -40Repository: AGI-Corporation/mcpmyid
Length of output: 1831
Fix documentation example in blended_tool_config property to match API schema.
The property description in optimal-workflow.ts shows [{ 'piece': 'gmail', 'action': 'send_email' }], but the API schema at mcp-server-controller.ts expects { pieceName, actionName }. Update the example to [{ 'pieceName': 'gmail', 'actionName': 'send_email' }] to match the strict schema validation.
While Fastify's schema validation will reject mismatched payloads before persistence (preventing silent failures), users following the example will receive validation errors. Align documentation with implementation to avoid user confusion.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/virtual-tool-entity.ts` at line 12, Update
the documentation example for blended_tool_config in optimal-workflow.ts to use
the API's strict field names so examples match the schema: change any example
objects like { 'piece': 'gmail', 'action': 'send_email' } to { 'pieceName':
'gmail', 'actionName': 'send_email' } so they align with the
mcp-server-controller.ts schema and the type referenced in
virtual-tool-entity.ts (baseActions / blended_tool_config).
|
|
||
| - **🤖 Agent OS Pillar**: Built on research-backed adaptive routing (CactusRoute) and decentralized discovery (NANDA). | ||
| - **🤖 Agent OS Pillar**: Built on research-backed adaptive routing (**CactusRoute**), logic-based validation (**Guido**), and decentralized discovery (**NANDA**). | ||
| - **🛠️ Largest open source MCP toolkit**: All our pieces (280+) are available as MCP that you can use with LLMs on Claude Desktop, Cursor or Windsurf. |
There was a problem hiding this comment.
Use a hyphenated compound adjective (open-source).
Line 68 should use open-source before “MCP toolkit” for correct grammar and clearer copy.
🧰 Tools
🪛 LanguageTool
[grammar] ~68-~68: Use a hyphen to join words.
Context: ...covery (NANDA). - 🛠️ Largest open source MCP toolkit: All our pieces (28...
(QB_NEW_EN_HYPHEN)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@README.md` at line 68, Update the README text that currently reads "open
source MCP toolkit" to use a hyphenated compound adjective: change it to
"open-source MCP toolkit" so the phrase before "MCP toolkit" is grammatically
correct; locate the string "open source MCP toolkit" in the README and replace
it with "open-source MCP toolkit".
… Hub This final major update completes the "Activepieces Agent OS" ecosystem with advanced Mistral-powered evaluation and a comprehensive documentation suite: - **Mistral RAG Evaluation (LLM as a Judge)**: - Added a new `evaluate_rag` action to the 'Agent OS' piece, implementing Mistral's structured RAG evaluation cookbook for Context Relevance, Answer Relevance, and Groundedness. - Integrated the evaluation layer into the system architecture and vision. - **Functional Virtual Tool Execution**: - Replaced placeholder logic with actual HTTP execution for OpenAPI-imported tools using `httpClient`. - Added a `props` field to `VirtualToolEntity` to persist PiecePropertyMap schemas, allowing AI agents to discover and use virtual tool parameters correctly. - **Enhanced Security & SSRF Protection**: - Implemented robust URL validation and host blacklisting (localhost, internal metadata IPs) for the OpenAPI import endpoint to mitigate SSRF risks. - **Comprehensive Documentation Hub**: - Created an `Agent OS` section in `docs/` featuring `about.md`, `architecture.md`, and `user-guide.md`. - Added technical Mermaid diagrams illustrating the 7-layer Cactus pipeline, the NANDA discovery sequence, and the Mistral evaluation loop. - Polished the repository `README.md` and `AGENT_FRAMEWORK.md` with branded architecture diagrams and user-centric guides. - **Protocol Interoperability**: - Finalized the NANDA protocol implementation via `/.well-known/agent.json` and capability broadcasting with AI-specific governance flags. - **Consolidated Rebranding**: - Successfully rebranded the 'MCP' piece to 'Agent OS', consolidating all agentic features (Adaptive Routing, Rule Engine, Evaluation) into a single unified capability. Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (2)
README.md (1)
68-68:⚠️ Potential issue | 🟡 MinorFix the hyphenation issue (previously flagged).
The text still contains "open source MCP toolkit" instead of "open-source MCP toolkit". This grammar issue was flagged in a previous review but has not been addressed.
📝 Suggested fix
-- **🛠️ Largest open source MCP toolkit**: All our pieces (280+) are available as MCP that you can use with LLMs on Claude Desktop, Cursor or Windsurf. +- **🛠️ Largest open-source MCP toolkit**: All our pieces (280+) are available as MCP that you can use with LLMs on Claude Desktop, Cursor or Windsurf.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@README.md` at line 68, Update the README text to fix the hyphenation by changing the phrase "open source MCP toolkit" to "open-source MCP toolkit" wherever it appears (specifically the string "**🛠️ Largest open source MCP toolkit**" in the diff), ensuring the hyphenated compound adjective is used consistently throughout the file.packages/server/api/src/app/mcp/mcp-server.ts (1)
54-56:⚠️ Potential issue | 🟠 MajorUse a single computed tool key for all uniqueness checks and registration.
The current flow mixes keys (
vt.name.slice(...), rawvt.name,action.name, andactionName). This can still allow collisions after truncation/prefixing and break runtime registration.Suggested fix
for (const vt of virtualTools) { + const virtualToolName = vt.name.slice(0, MAX_TOOL_NAME_LENGTH) + if (uniqueActions.has(virtualToolName)) { + logger.warn({ virtualToolName }, '[McpServer] Duplicate virtual tool name, skipping registration') + continue + } server.tool( - vt.name.slice(0, MAX_TOOL_NAME_LENGTH), + virtualToolName, vt.description, Object.fromEntries( Object.entries(vt.props || {}).map(([key, prop]: [string, any]) => [key, piecePropertyToZod(prop)], ), ), async (params) => { // ... } ) - uniqueActions.add(vt.name) + uniqueActions.add(virtualToolName) } pieces.flatMap(piece => { return Object.values(piece.actions).map(action => { - if (uniqueActions.has(action.name)) { - return - } - const shortPieceName = piece.name.replace('@activepieces/piece-', '') const actionName = `${shortPieceName}_${action.name}`.slice(0, MAX_TOOL_NAME_LENGTH) + if (uniqueActions.has(actionName)) { + return + } uniqueActions.add(actionName)Also applies to: 94-95, 99-109
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/mcp-server.ts` around lines 54 - 56, Compute a single canonical tool key once (e.g., using vt.name sliced to MAX_TOOL_NAME_LENGTH plus any prefix) and use that same key for all uniqueness checks and for server.tool registration instead of mixing vt.name.slice(...), raw vt.name, action.name, and actionName; update the registration calls around server.tool, and the checks that reference action.name/actionName to use that single computed key so truncation/prefixing is applied consistently across the code paths (also apply the same change to the other occurrences mentioned near the later registration blocks).
🧹 Nitpick comments (2)
packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts (1)
56-65: Consider defining a JSON schema in the prompt for parseable output.The current implementation returns free-form text evaluation. For downstream processing (e.g., aggregating scores, automated decisions), consider instructing the model to output JSON with explicit fields for each score and reasoning. This aligns with the TODO comment on lines 56-57.
💡 Example structured output prompt addition
Groundedness: How faithful is the generated answer to the retrieved context?`; + +Respond in the following JSON format: +{ + "context_relevance": { "reasoning": "...", "score": 0-3 }, + "answer_relevance": { "reasoning": "...", "score": 0-3 }, + "groundedness": { "reasoning": "...", "score": 0-3 } +}`;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts` around lines 56 - 65, Update the chat prompt used in evaluate-rag.ts (the ai.chat.text call that sends systemPrompt and userContent) to require machine-parsable JSON output: modify systemPrompt/userContent to include a JSON schema (explicit fields like overallScore, accuracyScore, relevanceScore, reasoning) and an instruction to return only valid JSON, then parse the response string into an object and validate the expected fields before downstream use (i.e., after the response from ai.chat.text stored in response). Ensure the parsing/validation handles errors and falls back or logs a clear message if output is not valid JSON.README.md (1)
110-110: Consider softening the superlative marketing claim.The phrase "world's first unified 'Agentic Operating System'" is a bold claim that may be difficult to verify and could quickly become outdated as the space evolves.
Consider a more measured statement like: "a unified 'Agentic Operating System'" or "one of the first unified 'Agentic Operating System' implementations."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@README.md` at line 110, Replace the unverified superlative in the README sentence that currently reads "Activepieces Agent OS is the world's first unified 'Agentic Operating System'..." with a softened claim such as "a unified 'Agentic Operating System'" or "one of the first unified 'Agentic Operating System' implementations" to avoid an absolute, hard-to-verify statement; update the README.md sentence accordingly wherever the phrase appears.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/pieces/community/mcp/src/index.ts`:
- Line 11: The logoUrl value in the module (logoUrl:
"https://cdn.activepieces.com/pieces/ai-agent.svg") points to a missing asset;
upload the ai-agent.svg file to the CDN at /pieces/ai-agent.svg (or update the
URL to the correct CDN location) so the reference in
packages/pieces/community/mcp/src/index.ts resolves without a 404; after
uploading, verify the file is publicly accessible and the URL loads correctly in
the browser.
In `@packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts`:
- Around line 67-71: The return accesses response.choices[0].content without
checking for an empty or missing choices array; update the evaluate-rag logic to
defensively verify response.choices exists and has length > 0 before accessing
index 0 (e.g., via a guard around response.choices or using a local variable
choice = response.choices?.[0]); if no choice is present, return a safe fallback
(e.g., evaluation: null or a message and adjust status to a non-EVALUATED state)
so that the code in this file (evaluate-rag) won’t throw a TypeError when the
Mistral response is empty or malformed.
In `@packages/server/api/src/app/mcp/mcp-server.ts`:
- Around line 69-83: The OPENAPI branch that calls httpClient.sendRequest (when
vt.metadata?.type === 'OPENAPI') needs a local try/catch around the await
httpClient.sendRequest(...) to handle thrown errors; catch the error and return
a structured MCP error payload instead of letting the exception bubble.
Specifically, wrap the existing call in try { const response = await
httpClient.sendRequest(...) ... return success content } catch (err) { return an
MCP-compatible error content object (e.g., content with type:'error' or an error
text field) including the vt.name and error message }, so callers of the OPENAPI
tool always receive a controlled response from the code path that handles
vt.metadata.type === 'OPENAPI'.
- Around line 73-75: The current call is duplicating the entire params object
into the URL by setting queryParams: params while also sending body:
params['body']; update the call in mcp-server.ts so queryParams only includes
the actual query fields (exclude the 'body' key) instead of passing params
wholesale. Concretely, build a filtered query object from params that omits
'body' (or pick only known query keys) and use that for queryParams, leaving
body: params['body'] unchanged; locate the caller that sets queryParams: params
and replace it with the filtered query object.
---
Duplicate comments:
In `@packages/server/api/src/app/mcp/mcp-server.ts`:
- Around line 54-56: Compute a single canonical tool key once (e.g., using
vt.name sliced to MAX_TOOL_NAME_LENGTH plus any prefix) and use that same key
for all uniqueness checks and for server.tool registration instead of mixing
vt.name.slice(...), raw vt.name, action.name, and actionName; update the
registration calls around server.tool, and the checks that reference
action.name/actionName to use that single computed key so truncation/prefixing
is applied consistently across the code paths (also apply the same change to the
other occurrences mentioned near the later registration blocks).
In `@README.md`:
- Line 68: Update the README text to fix the hyphenation by changing the phrase
"open source MCP toolkit" to "open-source MCP toolkit" wherever it appears
(specifically the string "**🛠️ Largest open source MCP toolkit**" in the diff),
ensuring the hyphenated compound adjective is used consistently throughout the
file.
---
Nitpick comments:
In `@packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts`:
- Around line 56-65: Update the chat prompt used in evaluate-rag.ts (the
ai.chat.text call that sends systemPrompt and userContent) to require
machine-parsable JSON output: modify systemPrompt/userContent to include a JSON
schema (explicit fields like overallScore, accuracyScore, relevanceScore,
reasoning) and an instruction to return only valid JSON, then parse the response
string into an object and validate the expected fields before downstream use
(i.e., after the response from ai.chat.text stored in response). Ensure the
parsing/validation handles errors and falls back or logs a clear message if
output is not valid JSON.
In `@README.md`:
- Line 110: Replace the unverified superlative in the README sentence that
currently reads "Activepieces Agent OS is the world's first unified 'Agentic
Operating System'..." with a softened claim such as "a unified 'Agentic
Operating System'" or "one of the first unified 'Agentic Operating System'
implementations" to avoid an absolute, hard-to-verify statement; update the
README.md sentence accordingly wherever the phrase appears.
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
AGENT_FRAMEWORK.mdREADME.mddocs/agent-os/about.mddocs/agent-os/architecture.mdpackages/pieces/community/mcp/src/index.tspackages/pieces/community/mcp/src/lib/actions/evaluate-rag.tspackages/server/api/src/app/mcp/mcp-server-controller.tspackages/server/api/src/app/mcp/mcp-server.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- docs/agent-os/about.md
- packages/server/api/src/app/mcp/mcp-server-controller.ts
Comment on lines
+67
to
+71
| return { | ||
| evaluation: response.choices[0].content, | ||
| status: "EVALUATED", | ||
| judge: "Mistral AI" | ||
| }; |
There was a problem hiding this comment.
Missing null check on response.choices[0] may cause runtime error.
If the Mistral API returns an empty choices array or the response structure differs unexpectedly, accessing response.choices[0].content will throw a TypeError. Add a defensive check before accessing the array element.
🛡️ Proposed fix to add defensive check
+ const evaluationContent = response.choices?.[0]?.content;
+ if (!evaluationContent) {
+ throw new Error('Mistral API returned no evaluation content');
+ }
+
return {
- evaluation: response.choices[0].content,
+ evaluation: evaluationContent,
status: "EVALUATED",
judge: "Mistral AI"
};📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| return { | |
| evaluation: response.choices[0].content, | |
| status: "EVALUATED", | |
| judge: "Mistral AI" | |
| }; | |
| const evaluationContent = response.choices?.[0]?.content; | |
| if (!evaluationContent) { | |
| throw new Error('Mistral API returned no evaluation content'); | |
| } | |
| return { | |
| evaluation: evaluationContent, | |
| status: "EVALUATED", | |
| judge: "Mistral AI" | |
| }; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts` around lines
67 - 71, The return accesses response.choices[0].content without checking for an
empty or missing choices array; update the evaluate-rag logic to defensively
verify response.choices exists and has length > 0 before accessing index 0
(e.g., via a guard around response.choices or using a local variable choice =
response.choices?.[0]); if no choice is present, return a safe fallback (e.g.,
evaluation: null or a message and adjust status to a non-EVALUATED state) so
that the code in this file (evaluate-rag) won’t throw a TypeError when the
Mistral response is empty or malformed.
Comment on lines
+69
to
+83
| if (vt.metadata?.type === 'OPENAPI') { | ||
| const response = await httpClient.sendRequest({ | ||
| method: vt.metadata.method as HttpMethod, | ||
| url: vt.metadata.url, | ||
| queryParams: params, // Simplified mapping | ||
| body: params['body'], | ||
| }) | ||
|
|
||
| return { | ||
| content: [{ | ||
| type: 'text', | ||
| text: `✅ Successfully executed ${vt.name}\n\n` + | ||
| `\`\`\`json\n${JSON.stringify(response.body, null, 2)}\n\`\`\``, | ||
| }] | ||
| } |
There was a problem hiding this comment.
Handle OPENAPI request failures and return structured MCP error content.
If httpClient.sendRequest throws, the tool call will fail without a controlled response. Add local try/catch and return an explicit error payload.
Suggested fix
if (vt.metadata?.type === 'OPENAPI') {
- const response = await httpClient.sendRequest({
- method: vt.metadata.method as HttpMethod,
- url: vt.metadata.url,
- queryParams: params, // Simplified mapping
- body: params['body'],
- })
-
- return {
- content: [{
- type: 'text',
- text: `✅ Successfully executed ${vt.name}\n\n` +
- `\`\`\`json\n${JSON.stringify(response.body, null, 2)}\n\`\`\``,
- }]
- }
+ try {
+ const response = await httpClient.sendRequest({
+ method: vt.metadata.method as HttpMethod,
+ url: vt.metadata.url,
+ queryParams: params,
+ body: params['body'],
+ })
+ return {
+ content: [{
+ type: 'text',
+ text: `✅ Successfully executed ${vt.name}\n\n` +
+ `\`\`\`json\n${JSON.stringify(response.body, null, 2)}\n\`\`\``,
+ }],
+ }
+ }
+ catch (error) {
+ logger.warn({ error, virtualTool: vt.name }, '[McpServer] OPENAPI virtual tool execution failed')
+ return {
+ content: [{
+ type: 'text',
+ text: `❌ Error executing ${vt.name}\n\n\`\`\`\n${(error as Error)?.message || 'Unknown error'}\n\`\`\``,
+ }],
+ }
+ }
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if (vt.metadata?.type === 'OPENAPI') { | |
| const response = await httpClient.sendRequest({ | |
| method: vt.metadata.method as HttpMethod, | |
| url: vt.metadata.url, | |
| queryParams: params, // Simplified mapping | |
| body: params['body'], | |
| }) | |
| return { | |
| content: [{ | |
| type: 'text', | |
| text: `✅ Successfully executed ${vt.name}\n\n` + | |
| `\`\`\`json\n${JSON.stringify(response.body, null, 2)}\n\`\`\``, | |
| }] | |
| } | |
| if (vt.metadata?.type === 'OPENAPI') { | |
| try { | |
| const response = await httpClient.sendRequest({ | |
| method: vt.metadata.method as HttpMethod, | |
| url: vt.metadata.url, | |
| queryParams: params, | |
| body: params['body'], | |
| }) | |
| return { | |
| content: [{ | |
| type: 'text', | |
| text: `✅ Successfully executed ${vt.name}\n\n` + | |
| `\`\`\`json\n${JSON.stringify(response.body, null, 2)}\n\`\`\``, | |
| }], | |
| } | |
| } | |
| catch (error) { | |
| logger.warn({ error, virtualTool: vt.name }, '[McpServer] OPENAPI virtual tool execution failed') | |
| return { | |
| content: [{ | |
| type: 'text', | |
| text: `❌ Error executing ${vt.name}\n\n\`\`\`\n${(error as Error)?.message || 'Unknown error'}\n\`\`\``, | |
| }], | |
| } | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/mcp-server.ts` around lines 69 - 83, The
OPENAPI branch that calls httpClient.sendRequest (when vt.metadata?.type ===
'OPENAPI') needs a local try/catch around the await httpClient.sendRequest(...)
to handle thrown errors; catch the error and return a structured MCP error
payload instead of letting the exception bubble. Specifically, wrap the existing
call in try { const response = await httpClient.sendRequest(...) ... return
success content } catch (err) { return an MCP-compatible error content object
(e.g., content with type:'error' or an error text field) including the vt.name
and error message }, so callers of the OPENAPI tool always receive a controlled
response from the code path that handles vt.metadata.type === 'OPENAPI'.
Comment on lines
+73
to
+75
| queryParams: params, // Simplified mapping | ||
| body: params['body'], | ||
| }) |
There was a problem hiding this comment.
Avoid duplicating payload into query string for OPENAPI calls.
At Line 73, all params are sent as query params while Line 74 also sends body. This can leak body data into URLs/logs and distort request semantics.
Suggested fix
+ const { body, ...queryParams } = params as Record<string, unknown>
const response = await httpClient.sendRequest({
method: vt.metadata.method as HttpMethod,
url: vt.metadata.url,
- queryParams: params, // Simplified mapping
- body: params['body'],
+ queryParams,
+ body,
})🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/mcp-server.ts` around lines 73 - 75, The
current call is duplicating the entire params object into the URL by setting
queryParams: params while also sending body: params['body']; update the call in
mcp-server.ts so queryParams only includes the actual query fields (exclude the
'body' key) instead of passing params wholesale. Concretely, build a filtered
query object from params that omits 'body' (or pick only known query keys) and
use that for queryParams, leaving body: params['body'] unchanged; locate the
caller that sets queryParams: params and replace it with the filtered query
object.
This commit finalizes the "LLM as a Judge" integration with specialized Mistral actions and pipeline optimizations:
- **Mistral Evaluation Actions**:
- Enhanced `evaluate_rag` to use Mistral's structured JSON output mode, ensuring precise scores for Context Relevance, Answer Relevance, and Groundedness.
- Added a dedicated `evaluate_hallucination` action in the 'Agent OS' piece to detect factual inconsistencies against source-of-truth contexts.
- **Execution Pipeline**:
- Updated `mcp-server.ts` Zod schemas to include an optional `query` parameter, enabling AI agents to pass natural language intent for the Cactus adaptive routing and extraction layers.
- Functionalized the evaluation loop (Layer 6: Observation) with log-level triggers for RAG-related tool calls.
- **Mistral AI Provider Enhancements**:
- Enforced `response_format: { type: 'text' }` for standard chat completions and added support for the `pixtral-large-latest` multimodal model.
- **Enterprise Security**:
- Completed the functional OpenAPI import workflow with persistent PiecePropertyMap schemas and robust SSRF protection.
- **Documentation**:
- Updated `docs/agent-os/user-guide.md` with detailed evaluation metrics and hallucination detection procedures.
- **Agent OS Consolidation**:
- Rebranded the community piece to 'Agent OS', providing a unified interface for Cactus optimization, Guido governance, and Mistral evaluation.
Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
This commit completes the "Activepieces Agent OS" toolset with specialized benchmark and visualization capabilities:
- **CactusRoute Benchmark**:
- Added a new `cactus_benchmark` action to the 'Agent OS' piece.
- Allows users to simulate and verify the framework's research-backed performance (F1 score, rescue rate) against standard datasets.
- **Advanced Visualization**:
- Added detailed Mermaid diagrams to `docs/agent-os/user-guide.md` illustrating the Guido Governance flow and the Mistral Hallucination Detection loop.
- Enhanced the `README.md` with a "Research & Framework Integration" section linking to the core research repositories (CactusRoute, Guido, NANDA).
- **Tool Logic Finalization**:
- Correctly wired the `evaluate_rag` action to use the native Mistral proxy with `response_format: { type: 'json_object' }`.
- Added multimodal support for `pixtral-large-latest` in the common AI framework.
- **Enterprise Governance**:
- Ensured all virtual tools and adaptive execution paths are fully discoverable via the NANDA discovery protocol.
Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
This final major update rebrands the entire agentic ecosystem to **Route.X** and completes the deep integration of research-backed protocols: - **Rebranding**: - Renamed "Agent OS", "MCP.My.ID", and "Cognition Hub" to **Route.X** across all documentation, code, and piece metadata. - Updated file paths and directories to use the `route-x` naming convention. - **Mistral RAG Evaluation**: - Finalized the `evaluate_rag` action with structured JSON output enforcement. - Implemented the `evaluate_hallucination` action for precise factual verification. - Added support for the `pixtral-large-latest` multimodal model. - **Cactus Adaptive Routing**: - Successfully integrated all 7 layers of the CactusRoute framework, including Difficulty Estimation and Deterministic Fallback. - Added a `cactus_benchmark` action to simulate and verify pipeline performance. - **Guido Governance**: - Enforced logic-based rules (SET, CONTAINS, NOT) in the tool execution cycle for both standard pieces and Virtual Tools. - **Technical Documentation**: - Created a dedicated `ARCHITECTURE.md` with comprehensive Mermaid diagrams illustrating the discovery, governance, and execution phases. - Updated the `README.md` and `docs/` with branded architecture sections and user guides. - **Enterprise Security**: - Enforced project-based ownership checks and implemented robust SSRF protection for OpenAPI imports. - **Discovery (NANDA)**: - Implemented the NANDA protocol via `/.well-known/agent.json` for decentralized capability discovery. Co-authored-by: AGI-Corporation <186229839+AGI-Corporation@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/pieces/community/route-x/README.md (1)
28-28:⚠️ Potential issue | 🟡 MinorStale default value in documentation.
The table documents
Model Namedefault asgpt-4o, but the code inoptimal-workflow.tsnow defaults tomistral-large-latest. Update the documentation to reflect the new default.📝 Suggested fix
-| **Model Name** | The LLM to use for orchestration (default: gpt-4o). | +| **Model Name** | The LLM to use for orchestration (default: mistral-large-latest). |🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/pieces/community/route-x/README.md` at line 28, Update the README table to reflect the actual default model used by the code: change the "Model Name" default from "gpt-4o" to "mistral-large-latest" so it matches the default set in optimal-workflow.ts; locate the "Model Name" row in packages/pieces/community/route-x/README.md and replace the stale default string with "mistral-large-latest" to keep docs and the optimal-workflow.ts implementation consistent.
♻️ Duplicate comments (8)
packages/server/api/src/app/mcp/mcp-server.ts (2)
54-56:⚠️ Potential issue | 🟠 MajorUse the actual registered tool name as the dedupe key.
Registration uses truncated/prefixed names, but dedupe uses raw
vt.name/action.name, so collisions can still slip through.Also applies to: 107-107, 112-123
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/mcp-server.ts` around lines 54 - 56, The dedupe logic uses raw vt.name/action.name while registration calls server.tool with a truncated/prefixed name (vt.name.slice(0, MAX_TOOL_NAME_LENGTH)), which allows collisions; change the code to compute the actual registeredName once (the same truncated/prefixed value used in server.tool) and use that registeredName everywhere (both when calling server.tool and as the key in the dedupe map/lookup for action registration and the code around lines where action.name is used) so dedupe and registration use the identical identifier (refer to MAX_TOOL_NAME_LENGTH, server.tool, vt.name, and action.name).
72-77:⚠️ Potential issue | 🟠 MajorOPENAPI execution should not duplicate body into query and should handle request failures locally.
Line 75 currently sends all params as query params while Line 76 also sends
body; this can leak payload fields into URLs. Also wrap outbound call with localtry/catchto return controlled MCP error content.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/mcp-server.ts` around lines 72 - 77, The outbound HTTP call in mcp-server.ts currently passes the whole params object as queryParams while also sending params['body'], which can leak body fields into the URL and lacks local error handling; update the httpClient.sendRequest invocation (where vt.metadata.method and vt.metadata.url are used) to build queryParams by excluding the body key (e.g., construct queryParams from params minus 'body') and pass params['body'] only as the request body, and wrap the sendRequest call in a try/catch that converts any failure into a controlled MCP error response (map the caught error into the MCP error shape your callers expect and return or throw that structured error instead of letting the raw exception propagate).packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts (1)
75-75:⚠️ Potential issue | 🟠 MajorGuard nested response access before using
content.Line 75 can throw on empty/malformed responses, which bypasses your
try/catchfallback path.Suggested fix
- const content = response.body.choices[0].message.content; + const content = response.body?.choices?.[0]?.message?.content; + if (typeof content !== 'string' || content.length === 0) { + return { + raw_evaluation: null, + status: "EVALUATION_UNAVAILABLE", + judge: "Mistral AI" + }; + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts` at line 75, The direct access to response.body.choices[0].message.content can throw on empty/malformed responses; update the code in evaluate-rag.ts to validate the nested structure before using it (check response and response.body exist, ensure Array.isArray(response.body.choices) and choices.length > 0, and that choices[0].message and message.content are defined) or use optional chaining on response.body?.choices?.[0]?.message?.content and handle the undefined case by either throwing a clear error or assigning a safe default so downstream logic in the function (e.g., where content is consumed) doesn't crash.packages/server/api/src/app/mcp/mcp-server-controller.ts (4)
57-68:⚠️ Potential issue | 🟡 MinorReturn
201 Createdfor blended-tool creation to match schema.The route schema declares
StatusCodes.CREATEDbut the handler currently returns default200.Also applies to: 171-172
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/mcp-server-controller.ts` around lines 57 - 68, The POST handler for '/:id/blended-tools' (the app.post(..., CreateBlendedToolRequest, async (req) => ...) function) currently returns the created tool with a default 200; update the handler to return HTTP 201 Created to match the route schema by setting the response status to StatusCodes.CREATED (or using the framework's response helper, e.g., reply.status(201) / h.response(...).code(201)) when returning the result of virtualToolService(...).create; make the same change for the other blended-tool creation handler referenced (the handler around lines 171-172) so both endpoints return 201 on success.
57-68:⚠️ Potential issue | 🔴 CriticalAdd request-time ownership checks before mutating MCP-scoped resources.
These handlers trust
req.params.idwithout asserting it belongs to the caller’s project. Line 71 fetches MCP but never verifies ownership.Suggested fix pattern
app.post('/:id/blended-tools', CreateBlendedToolRequest, async (req) => { const mcpId = req.params.id + const mcp = await mcpService(req.log).getOrThrow({ mcpId }) + if (req.principal.type !== PrincipalType.SERVICE && mcp.projectId !== req.principal.projectId) { + throw new Error('Not authorized to modify this MCP') + } const { name, description, baseActions, ruleSets } = req.body return virtualToolService(req.log).create({Also applies to: 71-73
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/mcp-server-controller.ts` around lines 57 - 68, The handler for CreateBlendedToolRequest uses req.params.id (mcpId) without verifying the MCP belongs to the caller’s project; before calling virtualToolService(...).create, fetch the MCP by that id (via your existing MCP lookup/service used elsewhere) and assert its project ownership matches the caller (e.g., compare mcp.projectId to req.auth/projectId/context value). If it does not match, return/throw a 403 forbidden error; only then proceed to call virtualToolService(req.log).create with the validated mcpId. Ensure the check runs in this route (the function handling POST '/:id/blended-tools') so no MCP-scoped mutation happens without ownership verification.
11-12:⚠️ Potential issue | 🔴 CriticalRemove the
preHandlerregistration for this authorization hook.Line 11 registers a
preSerialization-style handler aspreHandler; the callback signature does not match and can crash request handling.#!/bin/bash set -euo pipefail echo "=== Hook registration in controller ===" rg -n "addHook\\('preHandler'|addHook\\('preSerialization'" packages/server/api/src/app/mcp/mcp-server-controller.ts -n -C2 echo echo "=== Authorization hook signature ===" rg -n "export const entitiesMustBeOwnedByCurrentProject" packages/server/api/src/app/authentication/authorization.ts -n -A20🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/mcp-server-controller.ts` around lines 11 - 12, The hook entitiesMustBeOwnedByCurrentProject is a preSerialization-style handler but is incorrectly registered as a preHandler; remove the app.addHook('preHandler', entitiesMustBeOwnedByCurrentProject) registration in the MCP controller and keep only app.addHook('preSerialization', entitiesMustBeOwnedByCurrentProject) so the callback signature matches the lifecycle hook (look for the app.addHook calls in the mcp-server-controller and the exported entitiesMustBeOwnedByCurrentProject in authentication/authorization to confirm behavior).
75-83:⚠️ Potential issue | 🔴 CriticalStrengthen SSRF defenses for OpenAPI import URL validation.
Current checks are bypassable (private IP variants, DNS rebinding, redirects). This endpoint still needs robust hostname/IP resolution and redirect-safe fetch policy.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/server/api/src/app/mcp/mcp-server-controller.ts` around lines 75 - 83, The current SSRF checks (forbiddenHosts, parsedUrl) are insufficient; update URL validation to resolve parsedUrl.hostname to all IP addresses using dns.promises.lookup(..., { all: true }) and reject if any resolved address is in loopback (127.0.0.0/8, ::1), private (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), link-local (169.254.0.0/16, fe80::/10) or unspecified ranges using net.isIP and CIDR checks, and also reject if parsedUrl.hostname is already a literal IP in those ranges; when fetching the URL, set fetch(..., { redirect: 'manual' }) or otherwise prevent automatic redirects and after any allowed redirect resolve and re-check the destination IPs (use response.headers.get('location') only for manual redirect handling) so final target IPs are verified before returning content; keep existing forbiddenHosts check as an early fast-fail but rely on DNS resolution + CIDR checks and use symbols parsedUrl, forbiddenHosts, and the fetch call in mcp-server-controller to locate code to change.README.md (1)
68-68:⚠️ Potential issue | 🟡 MinorUse a hyphenated compound adjective (
open-source).Line 68 should use
open-sourcebefore "MCP toolkit" for correct grammar.✏️ Proposed fix
-- **🛠️ Largest open source MCP toolkit**: All our pieces (280+) are available as MCP that you can use with LLMs on Claude Desktop, Cursor or Windsurf. +- **🛠️ Largest open-source MCP toolkit**: All our pieces (280+) are available as MCP that you can use with LLMs on Claude Desktop, Cursor or Windsurf.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@README.md` at line 68, Update the sentence that begins "🛠️ Largest open source MCP toolkit" to use the hyphenated compound adjective "open-source" (i.e., replace "Largest open source MCP toolkit" with "Largest open-source MCP toolkit") so the README grammar is correct.
🧹 Nitpick comments (2)
packages/pieces/community/mcp/src/lib/actions/cactus-benchmark.ts (1)
23-42: Benchmark returns static data regardless of dataset selection.The
runfunction returns hardcoded metrics regardless of which dataset is selected. Thedatasetprop value is only echoed back in the response but doesn't influence the benchmark results. This could mislead users into believing actual benchmarking occurred.Consider either:
- Implementing actual dataset-specific benchmark logic, or
- Making the placeholder nature more explicit in the description and response.
📝 Suggested clarification
export const cactusBenchmark = createAction({ name: 'cactus_benchmark', displayName: 'CactusRoute Benchmark', - description: 'Run a simulated benchmark of the 7-layer Cactus pipeline against 30 standard cases.', + description: 'Returns reference benchmark metrics from CactusRoute research. Note: This is a demonstration action that returns pre-computed reference values.', props: {And update the verdict:
status: "COMPLETED", - verdict: "Framework is operating within research-backed performance boundaries." + verdict: "Reference metrics from CactusRoute research paper. Actual performance may vary." };🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/pieces/community/mcp/src/lib/actions/cactus-benchmark.ts` around lines 23 - 42, The run function currently returns static metrics (metrics, status, verdict) and only echoes context.propsValue.dataset, so update the logic in run to either compute dataset-specific benchmark outputs or explicitly mark the response as a placeholder: inspect context.propsValue.dataset and branch to produce different metrics (or seeded, synthetic values based on dataset id) and adjust the metrics object keys (f1_score, on_device_ratio, avg_latency_ms, rescue_rate) accordingly; alternatively, set a clear placeholder flag and change verdict/status to indicate "SIMULATED" and include a human-readable message explaining results are synthetic. Ensure you modify the run function, the returned metrics object, and the verdict/status fields (and any descriptive text) so the dataset selection meaningfully affects the response or the placeholder nature is explicit.ARCHITECTURE.md (1)
1-82: Consider consolidating or differentiating the architecture documentation.Both
ARCHITECTURE.md(root level) anddocs/route-x/architecture.mdcontain similar content describing the Route.X layers, pillars, and components. This duplication may lead to maintenance challenges and inconsistencies over time.Consider one of the following approaches:
- Consolidate: Keep one comprehensive architecture document and remove or redirect the other
- Differentiate: Clearly distinguish their purposes (e.g., root-level as executive summary, docs/route-x/ as detailed technical spec)
- Cross-reference: Add clear cross-references between the documents to indicate their relationship
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ARCHITECTURE.md` around lines 1 - 82, There are duplicate architecture docs; decide whether to consolidate into a single source or differentiate their scopes, then apply the change: either merge content from docs/route-x/architecture.md into ARCHITECTURE.md and remove or redirect the doc, or make ARCHITECTURE.md an executive summary and move detailed sections (pillars, Phase diagrams, System Components like mcp-server.ts, cactus-utils.ts, virtual-tool-service.ts, nanda-manifest-service.ts, evaluate-rag.ts) into docs/route-x/architecture.md and add clear cross-references at the top of both files indicating canonical source and intended audience; update any README links to point to the chosen canonical file.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@AGENT_FRAMEWORK.md`:
- Around line 70-72: Update the broken examples link by replacing the string
"examples/route-x/" with "examples/agent-os/" in the AGENT_FRAMEWORK.md
Resources section (the line that currently reads "- **[Sample
Templates](examples/route-x/)**") and make the same replacement inside
docs/route-x/quickstart.md to point both references to the correct examples
directory.
In `@docs/route-x/about.md`:
- Around line 13-28: The heading "## The Four Pillars" is inconsistent with five
listed items; update the heading or list so they match: either rename the
heading to "## The Five Pillars" or remove/merge one of the numbered sections
(for example combine "### 🌬️ 4. Multi-Model Optimization (Mistral & Beyond)"
and "### ⚖️ 5. LLM as a Judge (Evaluation Layer)") and renumber the remaining
sections to keep the sequence consistent throughout the file.
In `@packages/pieces/community/mcp/src/lib/actions/evaluate-hallucination.ts`:
- Line 67: The assignment const content =
response.body.choices[0].message.content is unsafe when choices is missing or
empty; update the extraction in evaluate-hallucination to defensively obtain
content by checking response.body?.choices?.length > 0 and
response.body.choices[0].message?.content first, fallback to the existing
JSON-parse path (or a safe default like an empty string) if that check fails,
and ensure any downstream code that uses content handles the fallback
accordingly.
In `@packages/pieces/community/route-x/src/lib/actions/optimal-workflow.ts`:
- Around line 13-17: The model Property.ShortText (named "model" with
defaultValue "mistral-large-latest") is unused in the run function and the
README still claims a default of "gpt-4o"; either wire the "model" prop into
execution (read the model value from inputs/config and pass it to the function
or client call that selects the LLM) or make the Property.ShortText optional and
add an inline comment on its intended future use; additionally update README.md
Configuration to match the chosen behavior and defaultValue so code and docs
stay consistent.
In `@packages/server/api/src/app/mcp/mcp-server.ts`:
- Around line 61-62: The code uses the Zod symbol z (e.g., z.string() in the
array schema and later around line 133) but never imports it, causing a compile
error; fix by adding the Zod import (import { z } from 'zod') at the top of the
file so references to z, z.string(), etc. in mcp-server.ts resolve correctly.
---
Outside diff comments:
In `@packages/pieces/community/route-x/README.md`:
- Line 28: Update the README table to reflect the actual default model used by
the code: change the "Model Name" default from "gpt-4o" to
"mistral-large-latest" so it matches the default set in optimal-workflow.ts;
locate the "Model Name" row in packages/pieces/community/route-x/README.md and
replace the stale default string with "mistral-large-latest" to keep docs and
the optimal-workflow.ts implementation consistent.
---
Duplicate comments:
In `@packages/pieces/community/mcp/src/lib/actions/evaluate-rag.ts`:
- Line 75: The direct access to response.body.choices[0].message.content can
throw on empty/malformed responses; update the code in evaluate-rag.ts to
validate the nested structure before using it (check response and response.body
exist, ensure Array.isArray(response.body.choices) and choices.length > 0, and
that choices[0].message and message.content are defined) or use optional
chaining on response.body?.choices?.[0]?.message?.content and handle the
undefined case by either throwing a clear error or assigning a safe default so
downstream logic in the function (e.g., where content is consumed) doesn't
crash.
In `@packages/server/api/src/app/mcp/mcp-server-controller.ts`:
- Around line 57-68: The POST handler for '/:id/blended-tools' (the
app.post(..., CreateBlendedToolRequest, async (req) => ...) function) currently
returns the created tool with a default 200; update the handler to return HTTP
201 Created to match the route schema by setting the response status to
StatusCodes.CREATED (or using the framework's response helper, e.g.,
reply.status(201) / h.response(...).code(201)) when returning the result of
virtualToolService(...).create; make the same change for the other blended-tool
creation handler referenced (the handler around lines 171-172) so both endpoints
return 201 on success.
- Around line 57-68: The handler for CreateBlendedToolRequest uses req.params.id
(mcpId) without verifying the MCP belongs to the caller’s project; before
calling virtualToolService(...).create, fetch the MCP by that id (via your
existing MCP lookup/service used elsewhere) and assert its project ownership
matches the caller (e.g., compare mcp.projectId to req.auth/projectId/context
value). If it does not match, return/throw a 403 forbidden error; only then
proceed to call virtualToolService(req.log).create with the validated mcpId.
Ensure the check runs in this route (the function handling POST
'/:id/blended-tools') so no MCP-scoped mutation happens without ownership
verification.
- Around line 11-12: The hook entitiesMustBeOwnedByCurrentProject is a
preSerialization-style handler but is incorrectly registered as a preHandler;
remove the app.addHook('preHandler', entitiesMustBeOwnedByCurrentProject)
registration in the MCP controller and keep only app.addHook('preSerialization',
entitiesMustBeOwnedByCurrentProject) so the callback signature matches the
lifecycle hook (look for the app.addHook calls in the mcp-server-controller and
the exported entitiesMustBeOwnedByCurrentProject in authentication/authorization
to confirm behavior).
- Around line 75-83: The current SSRF checks (forbiddenHosts, parsedUrl) are
insufficient; update URL validation to resolve parsedUrl.hostname to all IP
addresses using dns.promises.lookup(..., { all: true }) and reject if any
resolved address is in loopback (127.0.0.0/8, ::1), private (10.0.0.0/8,
172.16.0.0/12, 192.168.0.0/16), link-local (169.254.0.0/16, fe80::/10) or
unspecified ranges using net.isIP and CIDR checks, and also reject if
parsedUrl.hostname is already a literal IP in those ranges; when fetching the
URL, set fetch(..., { redirect: 'manual' }) or otherwise prevent automatic
redirects and after any allowed redirect resolve and re-check the destination
IPs (use response.headers.get('location') only for manual redirect handling) so
final target IPs are verified before returning content; keep existing
forbiddenHosts check as an early fast-fail but rely on DNS resolution + CIDR
checks and use symbols parsedUrl, forbiddenHosts, and the fetch call in
mcp-server-controller to locate code to change.
In `@packages/server/api/src/app/mcp/mcp-server.ts`:
- Around line 54-56: The dedupe logic uses raw vt.name/action.name while
registration calls server.tool with a truncated/prefixed name (vt.name.slice(0,
MAX_TOOL_NAME_LENGTH)), which allows collisions; change the code to compute the
actual registeredName once (the same truncated/prefixed value used in
server.tool) and use that registeredName everywhere (both when calling
server.tool and as the key in the dedupe map/lookup for action registration and
the code around lines where action.name is used) so dedupe and registration use
the identical identifier (refer to MAX_TOOL_NAME_LENGTH, server.tool, vt.name,
and action.name).
- Around line 72-77: The outbound HTTP call in mcp-server.ts currently passes
the whole params object as queryParams while also sending params['body'], which
can leak body fields into the URL and lacks local error handling; update the
httpClient.sendRequest invocation (where vt.metadata.method and vt.metadata.url
are used) to build queryParams by excluding the body key (e.g., construct
queryParams from params minus 'body') and pass params['body'] only as the
request body, and wrap the sendRequest call in a try/catch that converts any
failure into a controlled MCP error response (map the caught error into the MCP
error shape your callers expect and return or throw that structured error
instead of letting the raw exception propagate).
In `@README.md`:
- Line 68: Update the sentence that begins "🛠️ Largest open source MCP toolkit"
to use the hyphenated compound adjective "open-source" (i.e., replace "Largest
open source MCP toolkit" with "Largest open-source MCP toolkit") so the README
grammar is correct.
---
Nitpick comments:
In `@ARCHITECTURE.md`:
- Around line 1-82: There are duplicate architecture docs; decide whether to
consolidate into a single source or differentiate their scopes, then apply the
change: either merge content from docs/route-x/architecture.md into
ARCHITECTURE.md and remove or redirect the doc, or make ARCHITECTURE.md an
executive summary and move detailed sections (pillars, Phase diagrams, System
Components like mcp-server.ts, cactus-utils.ts, virtual-tool-service.ts,
nanda-manifest-service.ts, evaluate-rag.ts) into docs/route-x/architecture.md
and add clear cross-references at the top of both files indicating canonical
source and intended audience; update any README links to point to the chosen
canonical file.
In `@packages/pieces/community/mcp/src/lib/actions/cactus-benchmark.ts`:
- Around line 23-42: The run function currently returns static metrics (metrics,
status, verdict) and only echoes context.propsValue.dataset, so update the logic
in run to either compute dataset-specific benchmark outputs or explicitly mark
the response as a placeholder: inspect context.propsValue.dataset and branch to
produce different metrics (or seeded, synthetic values based on dataset id) and
adjust the metrics object keys (f1_score, on_device_ratio, avg_latency_ms,
rescue_rate) accordingly; alternatively, set a clear placeholder flag and change
verdict/status to indicate "SIMULATED" and include a human-readable message
explaining results are synthetic. Ensure you modify the run function, the
returned metrics object, and the verdict/status fields (and any descriptive
text) so the dataset selection meaningfully affects the response or the
placeholder nature is explicit.
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (25)
AGENT_FRAMEWORK.mdARCHITECTURE.mdREADME.mddocs/handbook/teams/ai.mdxdocs/mint.jsondocs/route-x/CONTRIBUTING.mddocs/route-x/SECURITY.mddocs/route-x/about.mddocs/route-x/architecture.mddocs/route-x/quickstart.mddocs/route-x/user-guide.mdpackages/cli/src/lib/commands/agent-optimize.tspackages/cli/src/lib/commands/agent-samples.tspackages/pieces/community/common/src/lib/ai/providers/mistral/index.tspackages/pieces/community/mcp/src/index.tspackages/pieces/community/mcp/src/lib/actions/cactus-benchmark.tspackages/pieces/community/mcp/src/lib/actions/evaluate-hallucination.tspackages/pieces/community/mcp/src/lib/actions/evaluate-rag.tspackages/pieces/community/route-x/README.mdpackages/pieces/community/route-x/package.jsonpackages/pieces/community/route-x/src/index.tspackages/pieces/community/route-x/src/lib/actions/optimal-workflow.tspackages/server/api/src/app/mcp/mcp-server-controller.tspackages/server/api/src/app/mcp/mcp-server.tspackages/server/api/src/app/mcp/nanda-manifest-service.ts
✅ Files skipped from review due to trivial changes (2)
- docs/route-x/quickstart.md
- docs/handbook/teams/ai.mdx
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/pieces/community/common/src/lib/ai/providers/mistral/index.ts
- docs/mint.json
Comment on lines
70
to
+72
| ## 📚 Resources | ||
| - **[Quickstart Guide](docs/agent-os/quickstart.md)**: 3-step setup for research-backed Agents. | ||
| - **[Sample Templates](examples/agent-os/)**: Example workflows for CRM, Support, and Discovery. | ||
| - **[Quickstart Guide](docs/route-x/quickstart.md)**: 3-step setup for research-backed Agents. | ||
| - **[Sample Templates](examples/route-x/)**: Example workflows for CRM, Support, and Discovery. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Verify referenced documentation files exist
# Check if the referenced files and directories exist
if [ -f "docs/route-x/quickstart.md" ]; then
echo "✓ docs/route-x/quickstart.md exists"
else
echo "✗ docs/route-x/quickstart.md NOT FOUND"
fi
if [ -d "examples/route-x" ]; then
echo "✓ examples/route-x/ directory exists"
fd . examples/route-x --max-depth 1
else
echo "✗ examples/route-x/ directory NOT FOUND"
fiRepository: AGI-Corporation/Route.X
Length of output: 142
🏁 Script executed:
# Find all directories under examples/ to locate the correct path
fd --type d . examples --max-depth 2 | head -20Repository: AGI-Corporation/Route.X
Length of output: 85
🏁 Script executed:
# List the full examples directory structure
find examples -type f -o -type d | head -30
# Also search for any references to "route-x" or "examples/" in markdown files
rg "examples/" --type md -A 2 -B 2 | head -50Repository: AGI-Corporation/Route.X
Length of output: 832
Fix the broken documentation link to examples directory.
The referenced path examples/route-x/ does not exist in the repository. The actual examples are located in examples/agent-os/. Update the reference from examples/route-x/ to examples/agent-os/ to point to the correct directory containing example workflows.
Note: This same broken reference also appears in docs/route-x/quickstart.md and should be corrected there as well.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@AGENT_FRAMEWORK.md` around lines 70 - 72, Update the broken examples link by
replacing the string "examples/route-x/" with "examples/agent-os/" in the
AGENT_FRAMEWORK.md Resources section (the line that currently reads "- **[Sample
Templates](examples/route-x/)**") and make the same replacement inside
docs/route-x/quickstart.md to point both references to the correct examples
directory.
Comment on lines
+13
to
+28
| ## The Four Pillars | ||
|
|
||
| ### 🌵 1. CactusRoute (Adaptive Routing) | ||
| Inspired by the 7-layer hybrid routing framework, every tool call in Activepieces undergoes a multi-stage optimization process. If a model provides an invalid time format or a negative number where a positive one is expected, Route.X repairs it instantly. | ||
|
|
||
| ### 📜 2. NANDA Protocol (Open Discovery) | ||
| Route.X implements the NANDA stack for the Open Agentic Web. Every project broadcasts its capabilities via `/.well-known/agent.json` using the **AgentFacts** JSON-LD format, enabling trillion-scale indexing and verifiable trust. | ||
|
|
||
| ### 🛡️ 3. Guido Rule Engine (Virtual Tooling) | ||
| Users can "blend" multiple low-level pieces into high-level "Virtual Tools." The Guido-inspired rule engine allows you to define complex logic (if-then-else, negation, pattern matching) that guards the execution of these tools. | ||
|
|
||
| ### 🌬️ 4. Multi-Model Optimization (Mistral & Beyond) | ||
| While we support all major providers, Route.X is deeply optimized for Mistral AI, leveraging its native tool-calling capabilities to deliver low-latency, high-accuracy agentic workflows. | ||
|
|
||
| ### ⚖️ 5. LLM as a Judge (Evaluation Layer) | ||
| Using Mistral-powered evaluation, Route.X automatically scores RAG (Retrieval-Augmented Generation) outputs for context relevance, answer relevance, and groundedness. This ensures your agents aren't just "fast," but "accurate." |
There was a problem hiding this comment.
Fix section/list mismatch: “Four Pillars” currently lists five items.
Line 13 says “Four Pillars”, but Line 27 introduces a fifth pillar. Update title or list structure for consistency.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/route-x/about.md` around lines 13 - 28, The heading "## The Four
Pillars" is inconsistent with five listed items; update the heading or list so
they match: either rename the heading to "## The Five Pillars" or remove/merge
one of the numbered sections (for example combine "### 🌬️ 4. Multi-Model
Optimization (Mistral & Beyond)" and "### ⚖️ 5. LLM as a Judge (Evaluation
Layer)") and renumber the remaining sections to keep the sequence consistent
throughout the file.
| }, | ||
| }); | ||
|
|
||
| const content = response.body.choices[0].message.content; |
There was a problem hiding this comment.
Add defensive extraction for completion content.
Line 67 can throw before the JSON parse fallback when choices is missing/empty.
Suggested fix
- const content = response.body.choices[0].message.content;
+ const content = response.body?.choices?.[0]?.message?.content;
+ if (typeof content !== 'string' || content.length === 0) {
+ return {
+ raw_analysis: null,
+ status: "ANALYSIS_UNAVAILABLE",
+ judge: "Mistral AI"
+ };
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| const content = response.body.choices[0].message.content; | |
| const content = response.body?.choices?.[0]?.message?.content; | |
| if (typeof content !== 'string' || content.length === 0) { | |
| return { | |
| raw_analysis: null, | |
| status: "ANALYSIS_UNAVAILABLE", | |
| judge: "Mistral AI" | |
| }; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/pieces/community/mcp/src/lib/actions/evaluate-hallucination.ts` at
line 67, The assignment const content = response.body.choices[0].message.content
is unsafe when choices is missing or empty; update the extraction in
evaluate-hallucination to defensively obtain content by checking
response.body?.choices?.length > 0 and response.body.choices[0].message?.content
first, fallback to the existing JSON-parse path (or a safe default like an empty
string) if that check fails, and ensure any downstream code that uses content
handles the fallback accordingly.
Comment on lines
13
to
17
| model: Property.ShortText({ | ||
| displayName: "Model Name", | ||
| required: true, | ||
| defaultValue: "gpt-4o" | ||
| defaultValue: "mistral-large-latest" | ||
| }), |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, check if the optimal-workflow.ts file exists and read its content
cat -n packages/pieces/community/route-x/src/lib/actions/optimal-workflow.tsRepository: AGI-Corporation/Route.X
Length of output: 2609
🏁 Script executed:
# Check for README.md in the route-x package
find packages/pieces/community/route-x -name "README.md" -type fRepository: AGI-Corporation/Route.X
Length of output: 110
🏁 Script executed:
# Run the verification script to check if model prop is used
rg -n 'context\.propsValue\.model|propsValue\.model' packages/pieces/community/route-x/Repository: AGI-Corporation/Route.X
Length of output: 49
🏁 Script executed:
# Read the README.md to check for documentation about the model default
cat packages/pieces/community/route-x/README.mdRepository: AGI-Corporation/Route.X
Length of output: 1824
Unused model prop and documentation inconsistency.
The model property is marked as required but is never used in the run function. Additionally, the README.md (Configuration section) still documents the default as gpt-4o, which is now inconsistent with the code default of mistral-large-latest.
Either:
- Wire up the
modelprop to actual execution logic, or - If this is a placeholder for future use, make it optional and add a comment explaining its intended purpose.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/pieces/community/route-x/src/lib/actions/optimal-workflow.ts` around
lines 13 - 17, The model Property.ShortText (named "model" with defaultValue
"mistral-large-latest") is unused in the run function and the README still
claims a default of "gpt-4o"; either wire the "model" prop into execution (read
the model value from inputs/config and pass it to the function or client call
that selects the LLM) or make the Property.ShortText optional and add an inline
comment on its intended future use; additionally update README.md Configuration
to match the chosen behavior and defaultValue so code and docs stay consistent.
Comment on lines
+61
to
+62
| ['query', z.string().optional().describe('The original user query text for adaptive routing and extraction.')] | ||
| ]), |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check the exact file path
if [ ! -f "packages/server/api/src/app/mcp/mcp-server.ts" ]; then
echo "File not found at expected path"
exit 1
fi
echo "=== File info ==="
wc -l packages/server/api/src/app/mcp/mcp-server.ts
echo
echo "=== Lines 1-20 (import section) ==="
head -20 packages/server/api/src/app/mcp/mcp-server.ts
echo
echo "=== Lines 55-70 (around line 61) ==="
sed -n '55,70p' packages/server/api/src/app/mcp/mcp-server.ts
echo
echo "=== Lines 128-140 (around line 133) ==="
sed -n '128,140p' packages/server/api/src/app/mcp/mcp-server.ts
echo
echo "=== Check for 'z' imports from zod ==="
rg "from 'zod'|from \"zod\"|import.*z" packages/server/api/src/app/mcp/mcp-server.ts | head -20
echo
echo "=== Check for all z. usages ==="
rg "z\." packages/server/api/src/app/mcp/mcp-server.ts | head -20Repository: AGI-Corporation/Route.X
Length of output: 3268
z is not imported but used at lines 61 and 133 (compile blocker).
The code references z.string() without importing z from 'zod', which will prevent TypeScript compilation.
Add the missing import:
Fix
+import { z } from 'zod'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/server/api/src/app/mcp/mcp-server.ts` around lines 61 - 62, The code
uses the Zod symbol z (e.g., z.string() in the array schema and later around
line 133) but never imports it, causing a compile error; fix by adding the Zod
import (import { z } from 'zod') at the top of the file so references to z,
z.string(), etc. in mcp-server.ts resolve correctly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Explain How the Feature Works
Relevant User Scenarios
Fixes # (issue)
Summary by CodeRabbit
New Features
Documentation