Add PostgreSQL conversation storage and NLWeb Protocol v0.54 updates

[rvguha]

Summary

• Add PostgreSQL backend for conversation storage with compressed JSON serialization
• Implement pluggable conversation storage backends (PostgreSQL, Azure Table Storage)
• Add conversation management utilities and authentication framework
• Update protocol models to properly serialize @type fields using Pydantic aliases
• Rename query.decontextualized_text to query.decontextualized_query for consistency
• Add TEST_USER environment variable support for testing
• Consolidate LLM and embedding providers into provider-specific packages
• Remove duplicate provider implementations from bundles package
• Add request context tracking and rate limiting infrastructure
• Clean up debug logging throughout codebase

Key Features

PostgreSQL Conversation Storage

• Full conversation turn storage (request + response)
• JSONB columns with compressed JSON (separators=(',', ':'))
• Proper @type serialization in stored data
• Async PostgreSQL client using asyncpg
• Schema initialization on server startup
• dump_conversations.py utility for viewing stored conversations

Protocol Updates

• Fixed Pydantic serialization to use @type instead of schema_type
• Added ser_json_by_alias=True to model configs
• Consistent field naming across request/response
• Support for meta.user and meta.remember fields

Provider Consolidation

• Moved all LLM providers to packages/providers/*/models
• Removed duplicate implementations from packages/bundles/models
• Cleaner separation of concerns

Test plan

• Server starts successfully with PostgreSQL conversation storage
• Conversations are saved to PostgreSQL with proper @type serialization
• dump_conversations.py displays saved conversations correctly
• JSON is compressed (no whitespace) in database
• TEST_USER environment variable is loaded and sent with queries
• decontextualized_query field works correctly

🤖 Generated with Claude Code