feat: complete fastn-p2p streaming API and remote shell functionality

[amitu]

Summary

Main Tracking PR - Complete implementation plan for fastn-p2p streaming API and remote shell functionality (rshell/rexec). This PR tracks the complete plan originally outlined in PR #2202, building on the fastn-context foundation from PR #2203.

This feature will be built incrementally through focused, reviewable PRs for better development and review process.

🎯 Complete Implementation Plan

Phase 1: Basic fastn-context (Minimal) ✅ Completed

• Create fastn-context crate with basic Context struct
• Implement simple context tree (parent/child relationships)
• Add basic cancellation tokens per context
• Move #[fastn_p2p::main] to #[fastn_context::main]
• Basic spawn() with context inheritance
• Status: PR feat: create fastn-context crate with hierarchical context system for debugging and operations #2203 ✅ (merged to main)

Phase 2: fastn-p2p Streaming API 🚧 API Stubs Only

• Add client module with connect() function signature
• Implement client::Session struct definition
• Add basic open_uni()/accept_uni() function signatures
• Add server::Session struct definition with context integration
• Add Session -> Request conversion signature (into_request())
• Preserve existing Request methods (get_input, handle)
• Basic context integration structure
• Status: PR feat: implement fastn-p2p streaming API foundation #2208 🚧 (API stubs with TODOs, needs actual implementation)

Phase 2a: Implement Core Streaming ⏳ Next - Must Do

• Implement client::connect() - actual iroh connection establishment
• Implement Session stream creation and management
• Add real context integration to Session structs
• Implement Session::into_request() conversion
• Scope: Make the API actually work, not just compile

Phase 3: Implement Multi-Stream Support ⏳ After 2a

• Implement accept_uni()/open_uni() for stderr streams
• Implement accept_bi()/open_bi() for bidirectional streams
• Test multiple concurrent streams
• Validate stream lifecycle management

Phase 4: Test and Validate ⏳ After 3

• Update fastn-p2p-test sender/receiver to use new API
• Test both RPC (Session->Request) and streaming patterns
• Validate iroh connection reuse works correctly
• Scope: End-to-end P2P streaming validation

Phase 5: Remote Shell Integration ⏳ After 4

• Update fastn-remote to use fastn-p2p streaming API
• Implement rshell using client::connect() -> Session
• Implement server-side rshell using Session streams
• Add stderr handling via open_uni()/accept_uni()
• Test fastn rshell and fastn rexec commands work
• Scope: Working remote shell over P2P

Phase 6: Codebase Migration ⏳ Critical

• Migrate existing fastn-p2p usage across main workspace
• Update fastn-daemon to use new fastn-context main macro
• Migrate existing fastn-p2p call() usage to ensure compatibility
• Update all existing tests to pass with new APIs
• Ensure v0.5 workspace tests continue to pass
• Verify main workspace tests pass with fastn-context integration
• Fix any regressions in existing functionality
• Scope: Zero breaking changes for existing code

🔗 Related PRs

• PR feat: create fastn-context crate with hierarchical context system for debugging and operations #2203 ✅ - fastn-context foundation (merged to main)
• PR feat: implement fastn-p2p streaming API foundation #2208 🚧 - fastn-p2p streaming API stubs (needs actual implementation)
• Future PRs - Will be created against this feature branch for Phases 2a-6

🎯 Success Criteria

1. ✅ fastn-context integrated - Hierarchical contexts for debugging (PR feat: create fastn-context crate with hierarchical context system for debugging and operations #2203)
2. 🚧 Streaming API stubs - Type definitions exist (PR feat: implement fastn-p2p streaming API foundation #2208)
3. ⏳ Streaming API works - Functions actually implemented (Phase 2a)
4. ⏳ Multi-stream support - Uni/bi streams work (Phase 3)
5. ⏳ fastn-p2p-test passes - New streaming API validated (Phase 4)
6. ⏳ fastn rshell works - Interactive shell over P2P (Phase 5)
7. ⏳ fastn rexec works - Command execution with separate stderr (Phase 5)
8. ⏳ All existing tests pass - Main and v0.5 workspaces (Phase 6)

🚨 Current Reality

PR #2208 Status: API stubs with 7 TODOs

• client::connect() - todo!()
• client::Session context - commented out
• Session::accept_uni/bi() - todo!()
• Session::into_request() - todo!()
• Session::open_uni/bi() - todo!()

Next: Implement Phase 2a to make these functions actually work.

🔧 Development Strategy

This feature branch approach provides:

• 📋 Incremental development through focused, reviewable PRs
• 🔍 Better review process with smaller, targeted changes
• 🚀 Continuous integration on the feature branch
• 📊 Clear progress tracking through this main PR
• 🧪 Isolated testing of the complete feature before main merge
• 🔒 Migration safety through comprehensive testing in Phase 6

Goal: Working remote shell over P2P with minimal API surface, built through reviewable increments, with zero breaking changes to existing code.

🤖 Generated with Claude Code