Releases: uditgoenka/indexer
Releases · uditgoenka/indexer
v1.0.4
v1.0.3
What's Changed
Frontend Redesign with shadcn/ui
- Replaced raw HTML with shadcn/ui components (Card, Button, Textarea, Table, Badge, Progress, Select)
- Dark theme with oklch color system and Geist font family
- Frosted glass navbar with backdrop-blur effect
- Submit page: Clean Card layout with URL count badge
- Dashboard: Summary stat cards + color-coded quota progress bars
- History: Filterable table with status badges and pagination
Full Changelog: v1.0.2...v1.0.3
v1.0.2
Refactor: Shared helpers and code quality improvements
- Extract shared constants for topic names and state groups
- Add
updateSubmission()helper (8x repeated pattern → 1 function) - Add
jsonResponse(),getBody(),getQueryParams()HTTP helpers - Cache
GoogleAuthinstances (avoid re-parsing JSON keys per request) - Use
Promise.allfor independent parallel operations - Fix fragile error status extraction in google-indexer
- Fix inconsistent retry logic in indexnow-submitter
- Fix tsconfig include path (src → steps)
v1.0.1 — Motia Runtime Compatibility Fix
What's Changed
Bug Fixes
- Resolved Motia/iii runtime compatibility issues preventing server startup
- Migrated step files from
src/tosteps/(Motia auto-discovery convention) - Switched to iii-native trigger types (
http,queue,cron) - Fixed HTTP handler input path for iii-sdk wrapping (
request.request.body) - Fixed cron expression to 6-field format (iii requires seconds position)
- Switched runtime from
buntonode(monitorEventLoopDelaynot supported in bun)
Configuration
- Added complete
iii-config.yamlwith all required modules (Stream, State, RestAPI, OTel, Queue, PubSub, Cron, Exec) - Added
motia.config.tswith@motiadevplugins - Updated API port from
3000to3111across all files
Requirements
- iii engine v0.7.0+ (v0.6.1 has SDK response format mismatch)
Verified Endpoints
POST /api/submit-urls— accepts URLs, fans out to Google + IndexNowGET /api/status— returns per-key quota statsGET /api/history— returns paginated submission history- Full event flow: submit → dispatch → google + indexnow → retry with backoff
v1.0.0 — URL Indexer with Motia.dev
URL Indexer v1.0.0
Bulk URL submission tool for Google Indexing API and IndexNow with intelligent multi-key rotation, quota tracking, and a real-time Next.js dashboard.
Highlights
- Multi-key rotation across 10+ GCP service accounts (2,000+ URLs/day)
- Dual submission to Google Indexing API + IndexNow (Bing, Yandex, DuckDuckGo)
- Smart retry with exponential backoff and jitter (max 3 attempts)
- Quota overflow queuing with automatic midnight drain
- Next.js dashboard with URL submit form, quota bars, and submission history
- Event-driven architecture built on Motia.dev framework
Backend Steps (Motia)
| Step | Type | Purpose |
|---|---|---|
submit-urls |
HTTP POST | Accept bulk URLs, validate, emit events |
get-status |
HTTP GET | Return per-key quota usage + pending queue |
get-history |
HTTP GET | Paginated submission history with filters |
url-dispatcher |
Queue | Fan-out URLs to Google + IndexNow queues |
google-indexer |
Queue | Call Google Indexing API with key rotation |
indexnow-submitter |
Queue | POST to IndexNow API |
retry-handler |
Queue | Exponential backoff retry logic |
quota-reset-cron |
Cron | Daily midnight reset + queue drain |
Frontend Pages (Next.js)
- / — URL submit form (paste URLs, one per line)
- /dashboard — Per-key quota table with visual progress bars (auto-refresh 30s)
- /history — Paginated submission history with status filters
Quick Start
git clone https://github.com/uditgoenka/indexer.git
cd indexer
npm install
cp .env.example .env # Configure your keys
iii -c iii-config.yaml # Start backend
# In separate terminal:
cd frontend && npm install && npm run devTech Stack
- Motia.dev + iii runtime (backend)
- Google Indexing API v3
- IndexNow API
- Next.js 15 + Tailwind CSS (frontend)
- TypeScript (strict mode, 0 errors)