Report building / customisation

[digitalWestie]

Report building / customisation

Reports can be compiled using the Sensemaking tools report tool.

The goal is to allow the admin/analyst to customise the report by choosing what sections they want to add.

At the moment, the tool produces a report using this structure:

Current report structure

1. Report Header

• Report title (from metadata)

2. About This Report (Introduction Section)

• Summary paragraph with; Total statements count, Total votes count, Number of topics identified, Number of subtopics identified
• Explanation of alignment terminology:
  • High alignment: 70%+ of participants voted the same way (e.g., 70% agree, or 70% disagree)
  • Low alignment: Votes were about split between participants (e.g., 40% agree, 60% disagree, or vice versa)
  • Uncertainty: More than 30% of participants voted "Unsure/pass"
• Breakdown widgets showing: Total statements, Total votes, Topics captured

3. Conversation Overview

• High-level overview of topics discussed
• Percentage of statements categorised under each topic (note: percentages may add up to greater than 100% when statements fall under more than one topic)
• Topics overview visualisation (interactive chart showing topic distribution)

4. Participant Alignment (Cross-Topic View)

• Toggle buttons to switch between:
  • High alignment
  • Low alignment
  • Uncertainty
• Shows top statements across all topics and subtopics for the selected alignment type
• Displays up to 12 statements per category, sorted by alignment score

5. Topic Sections (One per Topic)

Each topic section includes:

Topic Header

• Topic name
• Share button
• Breakdown showing: no. of subtopics, total statements, total votes

Topic Visualisation

• Interactive chart showing alignment within the topic
• Toggle between "Groupings" (solid) and "Statements" (waffle) views

Subtopic Sections (Expandable Accordion)

For each subtopic:

Subtopic Stats

• Statement count
• Vote count
• Relative alignment level (high/moderately high/moderately low/low) compared to other subtopics
• Relative engagement level compared to other subtopics

Prominent Themes

• Up to 5 themes identified from statements within the subtopic
• Short descriptions of each theme
• Based on statement text analysis (not vote information)
• Sensemaker attempts to account for differing viewpoints in how it presents themes

High Alignment Statements

• Statements where 70% or more of participants agreed or disagreed
• Up to 12 top statements shown (sorted by alignment score)
• Includes citations to original statements with vote counts
• Only statements with 20+ votes are included (to avoid misleading impressions from small sample sizes)

Low Alignment Statements

• Statements where opinions were split (40–60% of voters either agreed or disagreed)
• Up to 12 top statements shown (sorted by alignment score)
• Includes citations to original statements

High Uncertainty Statements

• Statements with high levels of uncertainty
• Criteria: Among the 25% most passed-on in the conversation as a whole, OR passed on by at least 20% of participants
• Up to 12 top statements shown (sorted by uncertainty score)
• Includes citations to original statements

View All Statements Button

• Opens a side drawer showing all statements in the subtopic
• Organised by alignment category (high/low/uncertainty/uncategorized)

6. Statement Drawer (Side Panel)

When "View all statements" is clicked, displays:

High Alignment Statements

• All statements (not just top 12) where 70%+ voted the same way
• Sorted by alignment score

Low Alignment Statements

• All statements where opinions were split (40–60%)
• Sorted by alignment score

High Uncertainty Statements

• All statements meeting uncertainty criteria
• Sorted by uncertainty score

Uncategorized Statements

• Statements that don't meet criteria for high alignment, low alignment, or high uncertainty
• Sorted by agree rate

Navigation

• Left sidebar with:
  • "Report Overview" button (scrolls to overview section)
  • Expandable topic list showing statement counts for each topic
  • Clickable subtopic links that expand the accordion and scroll to the subtopic

Data Sources

The report content is generated from three JSON files:

1. Summary JSON (summary.json): Contains LLM-generated narrative text including:

   • Overview section
   • Theme descriptions for each subtopic
   • Common ground and differences of opinion summaries
   • Citations to referenced statements

2. Topic Stats JSON (topic-stats.json): Contains:

   • Topic and subtopic structure
   • Statement and vote counts
   • Alignment and engagement metrics
   • Relative alignment/engagement labels

3. Comments JSON (comments-with-scores.json): Contains:

   • Individual statements with full text
   • Vote counts (agree/disagree/pass)
   • Alignment scores (high/low/uncertainty)
   • Topic and subtopic assignments
   • Filter flags

Notes

• Statements can belong to multiple topics, which is why topic percentages may sum to more than 100%
• Only statements with 20+ votes are included in alignment analysis to ensure statistical significance
• The report combines AI-generated summaries (themes, overview) with data-driven visualizations and statement cards based on vote statistics
• All citations are interactive and can be hovered to see original statement text and vote counts

[digitalWestie]

For a first step, let's allow admin to toggle various sections on / off.