Skip to content

Member Research framework design (#186)#540

Open
jonschull wants to merge 1 commit intomainfrom
issue/186
Open

Member Research framework design (#186)#540
jonschull wants to merge 1 commit intomainfrom
issue/186

Conversation

@jonschull
Copy link
Owner

Summary

Phase 1 design document for the Member Research framework (Issue #186).

  • BigMap Legacy is already done (PR feat: Convert 4,073 legacy BigMap entries to BigMap2 format #188). This PR scopes to Member Research only.
  • Answers Jon's 5 open questions from the Feb 12 comment: delta detection, trigger mechanism, output integration, API costs, quality control.
  • Revised architecture from the original Phase 1 comment: flat structure, clear API vs Interactive mode distinction, staged review flow.

Key Design Decisions

  • Staged output: AI-researched projects go to era_projects_staging.csv (not direct to live CSV). Jon reviews before promoting to era_projects_export.csv.
  • Delta detection: era_projects_state.json tracks {participant_id: last_researched_at + profile_hash}. Reruns only process new/changed members by default.
  • Mode B (Interactive): Claude Code IS the pipeline — not Python calling Claude Code tools. Python manages state; Claude does the research. Free on Max subscription.
  • Mode A (API): SerpAPI + Anthropic, ~$50 for all 906 members, deployable to Render.

Deliverables

Files Created For This Task

  • era2/scripts/member_research/DESIGN.md - Full architecture document with decisions, prompts, error handling, and cost estimates

Stale Branch Warning

This branch was created from main. Cherry-pick only files listed above.

Validation

Design document is self-contained and doesn't touch any running systems.

Next Steps

Issue #186 is labeled needs-discussion. Jon should review the design, answer Q1–Q6 in DESIGN.md, and comment "Approved" to proceed to Phase 2 implementation.

Phase 1 design: architecture for researching 906 ERA members to find
their publications and projects for the BigMap. Addresses Jon's Feb 12
open questions: delta detection, trigger mechanism, output integration,
API costs (~$50/full run), and quality control (staged review flow).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant