Topic summary content quality — resident-focused, skeptical, knowledgeable

[AndreRobitaille]

Problem

Topic summaries read like bureaucratic briefings, not like a neighbor explaining what's going on. Real-user testing shows the content is confusing and unhelpful.

Specific Quality Issues

1. Repeats procedural noise — "Motion to approve was made by Ald. X, seconded by Ald. Y" is not useful to residents. They care about what was decided and what it means.
2. Accepts institutional framing uncritically — If the city calls something "routine maintenance," the summary parrots it. No skepticism, no context about whether that framing is accurate or complete.
3. Doesn't leverage knowledgebase — We have admin-curated knowledge sources with relevant background (past controversies, context, relationships). Summaries should use this to add depth and skepticism, not just restate what the city published.
4. Doesn't highlight resident impact — Cost to taxpayers, timeline effects, who's affected, what changes in the neighborhood — this is what people care about. Current summaries bury it under procedural detail.
5. Internal structure leaks into output — The Factual Record / Institutional Framing / Civic Sentiment separation is good for generation governance, but the rendered output shouldn't expose these categories. Residents don't think in those buckets.

What Good Looks Like

A good topic summary should read like a well-informed neighbor at a coffee shop:

• "The city wants to spend $50k repaving Oak Street. They say it's routine, but this is the third time in five years they've patched this stretch instead of doing a full rebuild. Council voted 5-2 to approve — Smith and Johnson voted no, saying the money should go to a permanent fix."

Not:

• "Factual Record: A motion to approve expenditure of $50,000 for street repair on Oak Street was approved 5-2. Institutional Framing: City staff characterized the work as routine maintenance."

Scope

• Prompt engineering in Ai::OpenAiService (analyze_topic_summary, render_topic_summary)
• Possibly Topics::SummaryContextBuilder to better incorporate knowledgebase context
• May overlap with the summary architecture issue — content quality and generation architecture are related

Constraints

• Must still conform to TOPIC_GOVERNANCE.md (no fabrication, no motive inference, citations required)
• "Structural skepticism, not editorializing" — surface patterns without assigning motive
• Must be testable — prompt changes should be validated against real topic data

[AndreRobitaille]

Resolved

All five quality issues from this ticket have been addressed through prompt rewrites, architectural changes, and rendering updates.

What Changed

1. No more procedural noise
Editorial content ("What's Going On") is now a 100–200 word prose narrative. Motion/second/roll-call details live only in the Key Decisions section, rendered from Motion model data — not in the summary text.

2. Skeptical editorial voice
System prompts for both analyze_topic_briefing and analyze_topic_summary now use a "neighborhood reporter" role with explicit guidance:

• Skeptical of process and decisions, not of people
• Translate jargon ("general obligation promissory notes" → "borrowing")
• Surface patterns (deferrals, recurrence, cross-body progression) without ascribing motive
• Prompt includes GOOD/BAD examples to anchor tone

3. Knowledgebase context integrated
Topics::SummaryContextBuilder feeds KB chunks into Pass 1 (analysis) of the two-pass pipeline. Context is labeled as background knowledge and distinguished from document content, so the AI can use it to identify glossed-over details without conflating sources.

4. Resident impact highlighted
analyze_topic_summary scores resident impact 1–5 with a rubric calibrated to Two Rivers (property taxes, development changes, community services). Score gates homepage visibility (approved + impact ≥ 3 + future meeting → "Coming Up" card).

5. Internal structure not exposed
Factual Record / Institutional Framing / Civic Sentiment categories exist only in generation_data JSON (Pass 1 output). The rendering pass (Pass 2) synthesizes them into unified editorial prose. render_topic_summary_content helper in TopicsHelper additionally strips any leaked section headers via regex filter.

Architecture

The two-pass generation pipeline (analyze_* → render_*) separates reasoning from communication:

• Pass 1: Structured JSON analysis (editorial analysis, factual record, civic sentiment, continuity signals, resident impact)
• Pass 2: Markdown rendering into editorial_content (prose) and record_content (chronological cited bullets)

Citations use meeting names and dates (Council, Feb 18) instead of internal IDs ([agenda-309]).

Key Commits

• ac11db5 — Rewrite briefing prompts for scannable, plain-language output
• 92a4504 — Tune prompts for readability; separate editorial from record
• 05ce21a — Replace internal citation IDs with meeting name + date
• 68c7413 — Update per-meeting TopicSummary prompts to editorial voice
• 4344a57 — Add governance context (Committee on Aging, cross-body nuances)

Documentation

• docs/DEVELOPMENT_PLAN.md updated with editorial voice rules, two-pass architecture, citation rules, and three-tier briefing pipeline
• docs/plans/2026-02-21-topic-briefing-architecture-design.md has full design details
• docs/AUDIENCE.md has comprehensive audience persona and tone guidance