feat: Add source references requirement to code-explorer prompt

[buger]

Summary

• Updates the code-explorer persona prompt to require source code references in responses
• Instructs the AI agent to always include a "References" section at the end of answers
• References use the format file_path:line_number or file_path#symbol_name
• Refactors: Removes duplicate prompts from ProbeAgent.js to use shared/prompts.js as single source of truth

Changes

1. Added References Section to code-explorer prompt

When providing answers:
- Always include a "References" section at the end of your response
- List all relevant source code locations you found during exploration
- Use the format: file_path:line_number or file_path#symbol_name
- Group references by file when multiple locations are from the same file
- Include brief descriptions of what each reference contains

2. Prompt Architecture Refactoring

• Removed local predefinedPrompts definition from ProbeAgent.js that was shadowing the imported one
• Updated shared/prompts.js to be the single source of truth for all prompts
• Added missing code-review-template prompt to shared/prompts.js
• Reduced code duplication by ~120 lines

Benefits

• Makes it easier for users to navigate to relevant code sections
• Provides traceability for AI-generated explanations
• Improves the developer experience when exploring codebases
• Single source of truth for prompts makes maintenance easier

Test plan

• Run probe agent with a code exploration question
• Verify response includes a "References" section
• Verify references use correct format (file:line or file#symbol)
• Test different prompt types (architect, code-review, engineer, support, code-review-template)

🤖 Generated with Claude Code

[probelabs]

PR Overview: Add Source References Requirement to Code-Explorer Prompt

Summary

This PR enhances the code-explorer AI persona by requiring agents to always include a "References" section at the end of responses. This section lists all relevant source code locations discovered during code exploration using standardized formats (file_path:line_number or file_path#symbol_name).

Files Changed

Modified Files:

• npm/src/agent/ProbeAgent.js (+1 line, -57 lines)
• npm/src/agent/shared/prompts.js (+56 lines, -110 lines)

Total Changes: 2 files modified, +57 additions, -167 deletions

The PR primarily refactors prompt definitions by removing duplicate inline predefinedPrompts from ProbeAgent.js and consolidating them in the shared module, while adding the new "References" requirement to the code-explorer persona.

Architecture & Impact Assessment

What This PR Accomplishes

1. Improved Traceability: AI-generated explanations now include explicit source code references, enabling users to verify and navigate to relevant code sections directly.

2. Standardized Reference Format: Establishes consistent formatting (file_path:line_number or file_path#symbol_name) for code references across all code-explorer responses.

3. Code Deduplication: Removes duplicate predefinedPrompts definition from ProbeAgent.js, using only the shared module as the single source of truth.

4. Enhanced Developer Experience: Users can quickly jump to specific code locations mentioned in AI explanations without manual searching.

Key Technical Changes

New Prompt Section Added to code-explorer persona:

When providing answers:
- Always include a "References" section at the end of your response
- List all relevant source code locations you found during exploration
- Use the format: file_path:line_number or file_path#symbol_name
- Group references by file when multiple locations are from the same file
- Include brief descriptions of what each reference contains

Refactoring:

• Removed 57 lines of duplicate predefinedPrompts from ProbeAgent.js (previously defined inline in getSystemMessage())
• Now uses only the imported predefinedPrompts from ./shared/prompts.js (line 58)
• Simplified prompt selection logic in getSystemMessage() method

Affected System Components

graph TD
    A[ProbeAgent] --> B[predefinedPrompts from shared/prompts.js]
    B --> C[code-explorer persona]
    C --> D[getClaudeNativeSystemPrompt]
    C --> E[getCodexNativeSystemPrompt]
    C --> F[getSystemMessage]
    D --> G[Claude Native Engines]
    E --> H[Codex CLI]
    F --> I[XML Tool Format]
    
    style C fill:#90EE90
    style B fill:#87CEEB
    style D fill:#FFE4B5
    style E fill:#FFE4B5
    style F fill:#FFE4B5

Loading

Impact Flow:

1. The code-explorer prompt is now sourced from a single location (shared/prompts.js)
2. Used by three system prompt generation methods:
   • getClaudeNativeSystemPrompt() (line 1548)
   • getCodexNativeSystemPrompt() (line 1599)
   • getSystemMessage() (line 1821)
3. These methods are called when initializing AI conversations across different engines (Claude native, Codex CLI, XML format)
4. The new "References" requirement will be enforced in all AI responses using the code-explorer persona

Component Relationships

graph LR
    A[User Query] --> B[ProbeAgent]
    B --> C{Prompt Type?}
    C -->|code-explorer| D[Enhanced Prompt with References]
    C -->|other| E[Standard Prompt]
    D --> F[AI Response + References Section]
    E --> G[AI Response]
    
    style D fill:#90EE90
    style F fill:#87CEEB
    style B fill:#FFE4B5

Loading

Scope Discovery & Context Expansion

Immediate Impact

Directly Affected:

• npm/src/agent/shared/prompts.js - Now the single source of truth for all predefined prompts
• npm/src/agent/ProbeAgent.js - Refactored to use shared prompts, removed duplicate definition
• All AI interactions using the code-explorer persona

Integration Points:

• MCP Server (npm/src/agent/index.js) - Uses prompt types in tool handlers
• CLI/SDK interfaces - All code-explorer interactions will now include references
• Web interface - If it uses the npm package, will automatically benefit

Architecture Improvements

✅ Resolved Code Duplication Issue:

Previously, the predefinedPrompts object was defined in two separate locations:

1. Local definition in ProbeAgent.js within getSystemMessage() method
2. Shared export in npm/src/agent/shared/prompts.js

This PR successfully eliminates this duplication by:

• Removing the inline definition from ProbeAgent.js (57 lines deleted)
• Using only the imported predefinedPrompts from ./shared/prompts.js
• Establishing a single source of truth for all persona prompts

This resolves the maintenance burden where changes had to be synchronized manually.

Related Files (Not Modified)

• npm/src/agent/ProbeAgent.d.ts - TypeScript definitions (no changes needed)
• npm/tests/unit/probe-agent-delegate.test.js - Unit tests for prompt handling
• npm/src/agent/engines/enhanced-claude-code.js - Claude Code engine integration
• npm/src/agent/engines/codex.js - Codex CLI engine integration

Testing Recommendations

Manual Testing:

1. Run probe agent with code-explorer persona: probe agent "How does authentication work?" --prompt code-explorer
2. Verify response includes "References" section
3. Check format matches file:line or file#symbol
4. Verify references are grouped by file when applicable

Automated Testing (Future):

• Add unit test verifying getSystemMessage() contains the new References section
• Add integration test for response format verification
• Test that shared prompts are correctly imported and used

Potential Expansion Points

Consider for Future Updates:

1. Other Personas - Evaluate if similar reference requirements should be added to:

   • architect persona (architecture analysis)
   • code-review persona (code review findings)
   • engineer persona (implementation references)
   • support persona (troubleshooting references)

2. Configurable References - Make the references requirement configurable via a flag

3. Validation Layer - Add post-processing to verify References section is present and correctly formatted

Review Notes

This is a low-risk, high-value change:

• ✅ Focused enhancement with clear user benefit
• ✅ No breaking changes
• ✅ Backward compatible
• ✅ Improves developer experience
• ✅ Resolves code duplication issue
• ✅ Establishes single source of truth for prompts

Labels:

• tags.review-effort: 1 (trivial - simple prompt text addition and code deduplication)
• tags.label: enhancement (improves existing functionality)

Metadata

• Review Effort: 1 / 5
• Primary Label: enhancement

---

Powered by Visor from Probelabs

Last updated: 2026-01-19T09:21:11.304Z | Triggered by: pr_updated | Commit: e3d6331

💡 TIP: You can chat with Visor using /visor ask <your question>

[probelabs]

✅ Security Check Passed

No security issues found – changes LGTM.

Architecture Issues (4)

Severity	Location	Issue
🟢 Info	npm/src/agent/shared/prompts.js:17	The 'References' requirement is hard-coded into the code-explorer prompt as a special case. This makes it difficult to reuse this functionality for other personas or to make it configurable. If you later want to add references to other personas or make it optional, you'll need to modify the prompt text directly rather than using a more flexible architectural approach. 💡 Suggestion Consider a more composable approach where prompt requirements are built from smaller, reusable modules. For example, you could have a 'referencesRequirement' module that can be optionally included in any persona prompt. This would make the system more extensible and allow for easier configuration.
🟢 Info	npm/src/agent/shared/prompts.js:5	The prompts.js file contains all persona prompts as a single object export. As more personas are added or existing ones are modified, this monolithic structure will become harder to maintain. The current change adds a new requirement to code-explorer without considering how this pattern scales to future modifications across multiple personas. 💡 Suggestion Consider breaking down the prompts into separate modules or using a more structured approach (e.g., a PromptBuilder class) that allows for composition and reuse. This would make it easier to add new features like the 'References' section to multiple personas consistently.
🟡 Warning	npm/src/agent/ProbeAgent.js:1810	The PR removes a duplicate local definition of predefinedPrompts from getSystemMessage() method (lines 1810-1835), which is good. However, the architectural issue remains: the prompts are now centralized in npm/src/agent/shared/prompts.js but there's no validation or synchronization mechanism to ensure this shared module is the single source of truth. The comment at line 1810 says 'Use predefined prompts from shared module (imported at top of file)' but this relies on developer discipline rather than architectural enforcement. 💡 Suggestion Consider adding a runtime validation or build-time check to ensure the shared prompts.js module is always used. For example, you could add a test that verifies no duplicate prompt definitions exist in the codebase, or use a linting rule to prevent local prompt definitions. This would prevent future code duplication.
🟡 Warning	npm/src/agent/shared/prompts.js:5	The code-explorer prompt now has a special 'References' requirement (lines 17-21) that other personas don't have. This creates an inconsistency where code-explorer responses are structurally different from architect, code-review, engineer, and support responses. This special case could confuse users who switch between personas and expect consistent output formats. 💡 Suggestion Consider making the 'References' section a configurable feature that can be enabled/disabled per persona, or apply it consistently across all personas that deal with code exploration (architect, code-review, engineer). Alternatively, document this as an intentional difference in behavior so users understand the distinction.

✅ Performance Check Passed

No performance issues found – changes LGTM.

Quality Issues (5)

Severity	Location	Issue
🟢 Info	npm/src/agent/shared/prompts.js:6	The code-explorer prompt has been simplified compared to other personas (architect, code-review, etc.). While the old version had detailed numbered instructions and explicit 'NOT' constraints, the new version uses a simpler bullet-point format. This inconsistency makes it harder to maintain and extend prompts uniformly across personas. 💡 Suggestion Consider standardizing the prompt structure across all personas. Either use numbered lists with explicit constraints for all, or use simplified bullet points for all. Consistency makes prompts easier to maintain and compare.
🟡 Warning	npm/src/agent/ProbeAgent.js:1810	The PR removes a duplicate local definition of predefinedPrompts from ProbeAgent.js (lines 1810-1867 in the old version), which is good. However, the comment on line 1810 says 'Use predefined prompts from shared module (imported at top of file)' but there's no visible import statement in the diff. The import exists at line 58 but wasn't shown in the diff context. This creates a potential maintenance issue where future developers might not realize the prompts are centralized in shared/prompts.js. 💡 Suggestion Add a more explicit comment that references the actual import location, or consider adding a JSDoc comment above the getSystemMessage method that documents the dependency on shared/prompts.js
🟡 Warning	npm/src/agent/shared/prompts.js:17	The new 'References' requirement added to the code-explorer prompt (lines 17-21) has no automated test coverage. While the PR description mentions manual testing steps, there are no unit or integration tests that verify the prompt contains the expected references section. This makes it easy for future changes to accidentally remove or modify this requirement without detection. 💡 Suggestion Add a unit test in npm/tests/unit/ that verifies the code-explorer prompt contains the 'References' section text. Test should check for key phrases like 'References section', 'file_path:line_number', and 'file_path#symbol_name'. This would be similar to existing tests in probe-agent-delegate.test.js that verify prompt content.
🟡 Warning	npm/src/agent/shared/prompts.js:17	The prompt instructs the AI to 'Always include a References section' but doesn't specify what should happen when no code references are found during exploration. This ambiguity could lead to inconsistent behavior - sometimes including an empty References section, sometimes omitting it entirely. 💡 Suggestion Clarify the expected behavior when no references are found. For example: 'Include a References section even if empty, or omit it entirely if no code was explored.' This provides clear guidance for consistent AI responses.
🟡 Warning	npm/src/agent/shared/prompts.js:17	No tests verify edge cases for the References requirement, such as: responses with no code exploration, responses with only tool results but no file references, or responses where the AI fails to include the References section. These negative test scenarios are critical for ensuring the feature works reliably in production. 💡 Suggestion Add integration tests that simulate various response scenarios and verify the References section is present and correctly formatted. Test cases should include: 1) Response with multiple file references, 2) Response with symbol references, 3) Response with no code exploration, 4) Response where AI forgets to include references.

---

Powered by Visor from Probelabs

Last updated: 2026-01-19T09:21:14.457Z | Triggered by: pr_updated | Commit: e3d6331

💡 TIP: You can chat with Visor using /visor ask <your question>