Skip to content

runs: extract recommended companies from answer text (not grounding sources) #198

@arberx

Description

@arberx

Problem

The current competitor detection system checks project competitor domains against grounding source domains. This is fundamentally wrong:

  • When AI answers "best AI consulting firm", it reads an article on techloy.com and recommends Accenture
  • Grounding source = techloy.com (the article site)
  • Recommended company = Accenture (the actual competitor)
  • computeCompetitorOverlap() looks for accenture.com in grounding sources → not found
  • Result: Accenture is never flagged as a competitor, even though AI explicitly recommended them

The answerText brand-name check partially works, but requires pre-registering every competitor domain — which is impractical.

Current Flow

  1. User manually adds competitor domains to project
  2. computeCompetitorOverlap() checks: citedDomains (grounding sources) → groundingSources URIs → answerText for brand names
  3. Only matches against pre-registered domains

Proposed Fix

Extract recommended company names directly from answerText using LLM analysis (same approach as snapshot feature, #193/#194):

  1. After each provider response, run a lightweight extraction pass on the answer text
  2. Store extracted company names in a new recommended_competitors column on query_snapshots
  3. Surface in the API and UI (Competitors tab, ui: show competitors and grounding sources in tabbed sidebar view #196/feat: tabbed sidebar — Competitors + Sources #197)
  4. No pre-registration required — the extraction finds whoever AI actually recommended

This makes the existing manual competitor list optional (useful for tagging known competitors) rather than the sole detection method.

Impact

Without this fix, the competitor intelligence shown in the dashboard is fundamentally misleading — it shows article source domains, not the companies AI recommends to users.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions