HTTP Client: Simpler Streaming with Callbacks and Type-Safe Headers

[dcrockwell]

HTTP Client: Simpler Streaming with Callbacks and Type-Safe Headers

What This Does

Makes HTTP streaming dramatically simpler by replacing manual selector loops with callbacks, while adding type safety for headers throughout the module.

Why These Changes

The Problem:

When streaming HTTP responses, users had to write 40+ lines of boilerplate:

• Build selectors manually
• Write recursive receive loops
• Handle request ID filtering
• Mix HTTP messages with app messages in their mailbox

This violated our ergonomics principles and didn't follow BEAM best practices (one process per concern).

Additionally, recordings were lost if stop() wasn't called, and headers were untyped tuples throughout.

The Solution:

Process-based streaming with callbacks + type-safe headers.

What Changed

New Streaming API (Breaking)

Before (2.x):

let selector = process.new_selector()
  |> client.select_stream_messages(HttpStream)

let assert Ok(req_id) = client.stream_messages(request)
// 40+ lines of recursive receive loop...

After (3.0):

let assert Ok(stream) = client.new
  |> client.on_stream_chunk(fn(data) { io.print(data) })
  |> client.start_stream()

client.await_stream(stream)  // Optional

New Functions:

• start_stream() - Spawns dedicated process with callbacks
• on_stream_start(), on_stream_chunk(), on_stream_end(), on_stream_error() - Builder callbacks
• await_stream() - Wait for completion
• cancel_stream_handle(), is_stream_active() - Stream control

Removed:

• stream_messages() - now internal
• select_stream_messages() - now internal

Header Type (Breaking)

Added Header(name, value) type for type safety:

• get_headers() returns List(Header) not List(#(String, String))
• StreamStart/StreamEnd use List(Header)
• add_header(name, value) still takes strings (builds Header internally)

Immediate Recording Saves

• Recordings saved to disk immediately when captured
• recorder.stop() now optional (just cleanup)
• Never lose recordings even if process crashes

Documentation

• All 9 README code examples are tested snippets (following dream_ets pattern)
• Created test/snippets/ with runnable examples
• World-class hexdocs for all public functions and types

How It Works

Each stream runs in its own BEAM process:

1. start_stream() spawns unlinked process
2. Process calls internal stream_messages() to start HTTP stream
3. Selects HTTP messages from its own mailbox
4. Calls user callbacks as messages arrive
5. Exits when stream completes/errors

Your mailbox stays clean. Your callbacks can send messages back to your actor if needed.

Testing

• 97 tests passing (was 112)
• Deleted 25 obsolete selector tests
• Added 10 new start_stream() tests
• Added 9 snippet tests
• All examples verified working

Migration

See CHANGELOG.md for detailed migration guide. Summary:

// Old selector pattern
let selector = process.new_selector() |> client.select_stream_messages(HttpStream)
let assert Ok(req_id) = client.stream_messages(request)
// Recursive loop...

// New callback pattern  
let assert Ok(stream) = client.new |> client.on_stream_chunk(process_chunk) |> client.start_stream()
client.await_stream(stream)

Headers: Use pattern matching with Header(name, value) instead of #(name, value).

---

Version: 3.0.0 (breaking)
Tests: 97 passing
Branch: feature/http-client-auto-record