EdStem Smart Search

A browser extension that augments EdStem's native search with semantic (vector) search powered by Pinecone. Help students find relevant discussion threads based on meaning rather than just keyword matching.

Features

• Semantic Search: Find threads based on meaning, not just keywords
• Smart Matches UI: Displays AI-powered results above native search results
• Course Sync: Index course threads to Pinecone for fast semantic search
• Automatic Auth: Captures EdStem auth tokens automatically from network traffic
• Real-time Status: Shows sync progress and backend connection status

Prerequisites

• Node.js >= 22.15.1
• pnpm >= 10.11.0
• Python >= 3.10
• Pinecone account (sign up)

Setup

1. Pinecone Index Setup

First, create a Pinecone index with integrated embeddings:

# Install Pinecone CLI (macOS)
brew tap pinecone-io/tap
brew install pinecone-io/tap/pinecone

# Authenticate
pc login

# Create the index with integrated embeddings
pc index create -n edstem-smart-search -m cosine -c aws -r us-east-1 --model llama-text-embed-v2 --field_map text=content

2. Backend Setup

cd src

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r ../requirements.txt

# Configure environment
cp .env.example .env
# Edit .env and add your PINECONE_API_KEY

# Run the server
uvicorn server:app --reload --port 8000

3. Extension Setup

cd chrome-extension

# Install dependencies
pnpm install

# Development mode (with hot reload)
pnpm dev

# Or build for production
pnpm build

4. Load Extension in Chrome

1. Open Chrome and navigate to chrome://extensions/
2. Enable "Developer mode" (top right)
3. Click "Load unpacked"
4. Select the chrome-extension/dist directory

Usage

1. Navigate to EdStem: Go to any EdStem course discussion page
2. Capture Auth Token: Use EdStem's search bar once (this captures your auth token)
3. Sync Course: Click the extension icon and click "Sync Current Course"
4. Search: Type in EdStem's search bar - smart results appear above native results

API Endpoints

Backend (FastAPI)

Endpoint	Method	Description
/health	GET	Health check
/search	GET	Semantic search (q, course_id, k, min_score)
/sync_course	POST	Index course threads to Pinecone
/sync_status/{course_id}	GET	Get sync status for a course
/index_stats	GET	Get Pinecone index statistics

Example Search Request

curl "http://localhost:8000/search?q=how+does+paxos+work&course_id=74827&k=5"

Configuration

Environment Variables

Variable	Required	Default	Description
PINECONE_API_KEY	Yes	-	Pinecone API key
PINECONE_INDEX_NAME	No	edstem-smart-search	Index name
ED_API_KEY	No	-	EdStem API key (optional)

Extension Settings

The extension stores settings in Chrome's local storage:

• edstem_auth_token: Captured EdStem auth token
• current_course_id: Currently detected course ID
• backend_status: Backend connection status

Development

Backend Development

cd src
uvicorn server:app --reload --port 8000

Extension Development

cd chrome-extension
pnpm dev

Changes to the extension will hot-reload automatically.

Building for Production

# Backend: No build step needed (Python)

# Extension
cd chrome-extension
pnpm build
pnpm zip  # Creates distributable zip

Troubleshooting

"Course not indexed" error

• Click "Sync Current Course" in the extension popup
• Make sure you've used EdStem's search bar at least once to capture auth token

"Backend Disconnected" status

• Ensure the Python backend is running on port 8000
• Check that PINECONE_API_KEY is set correctly

No results appearing

• Verify the course has been synced (check popup for indexed thread count)
• Try a different search query
• Check browser console for errors

Tech Stack

• Backend: FastAPI, Pinecone, Python
• Extension: React 19, TypeScript, Vite, Tailwind CSS
• Vector Search: Pinecone with llama-text-embed-v2 embeddings
• Reranking: bge-reranker-v2-m3 for improved relevance

License

MIT