dream_http_client Release 3.0.0: Process-Based Streaming and Type-Safe Headers #35
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… add Header type ## Why This Change Was Made - The old stream_messages() API exposed selector boilerplate to users, violating ergonomics principles - Users had to write 40+ lines of recursive receive loops for every stream - Mailbox pollution: HTTP messages mixed with application messages - Not following BEAM best practices (one process per concern) - Header tuples lacked type safety throughout the module - Recordings were lost if stop() wasn't called (original issue on this branch) ## What Was Changed Breaking Changes: - Removed stream_messages() and select_stream_messages() from public API (now internal) - Added start_stream() with builder-based callbacks (on_stream_start, on_stream_chunk, on_stream_end, on_stream_error) - Each stream runs in dedicated BEAM process for isolation - Added Header type replacing List(#(String, String)) throughout module - get_headers() now returns List(Header) - StreamStart/StreamEnd use List(Header) for headers - add_header(name, value) still takes strings (ergonomic builder pattern) New API: - start_stream() - spawn stream with callbacks - await_stream(handle) - wait for completion - cancel_stream_handle(handle) - cancel stream - is_stream_active(handle) - check status - Header(name, value) type for type safety Recording (original feature): - Recordings saved immediately when captured - recorder.stop() now optional - Added storage.save_recording_immediately() Documentation: - All README examples are tested snippets (dream_ets pattern) - Added test/snippets/ directory with 9 tested examples - World-class hexdocs for Header and StreamHandle types - Comprehensive migration guide in CHANGELOG Tests: - 97 tests passing (was 112, deleted 25 obsolete selector tests, added 10 new start_stream tests) - Deleted stream_messages_integration_test.gleam and stream_messages_unit_test.gleam (obsolete) - Added start_stream_test.gleam with callback API tests ## Note to Future Engineer - The Header type uses .name and .value fields, not tuples. Convert at FFI boundaries. - Callbacks execute in the SPAWNED process, not the caller's process. Send messages to parent if needed. - await_stream() polls with sleep(50) - no timeout, it waits forever. Use cancel_stream_handle() for timeout behavior. - ETS tables must be initialized before spawning (ensure_ets_tables() in start_stream). Yes, this bit me. - stream_messages() still exists internally - it's used by start_stream(). Don't delete it. If you're wondering why we didn't just improve the selector API instead of replacing it entirely: we tried. The fundamental issue is that putting HTTP messages in the user's mailbox is the wrong abstraction. One process per stream is the BEAM way. Should have been designed this way from the start, but better late than never. Enjoy not writing recursive receive loops anymore. You're welcome.
HTTP Client: Simpler Streaming with Callbacks and Type-Safe Headers
dcrockwell
added
documentation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
dream_http_client 3.0.0 Release
Overview
Major redesign of the HTTP client streaming API with process isolation, callback-based handling, and type-safe headers throughout the module.
Key Changes
1. Redesigned Streaming API (Breaking)
Old approach: Manual selector loops, 40+ lines of boilerplate
New approach: Callback-based with dedicated processes
Benefits:
2. Header Type (Breaking)
Added
Header(name, value)type for type safety throughout module.3. Immediate Recording Saves
Recordings now saved immediately when captured - never lose data even if process crashes.
4. Tested Snippets
All README examples are now tested code in
test/snippets/following dream_ets pattern.Migration
See CHANGELOG.md and Release Notes for complete migration guide.
Testing
Breaking Changes
stream_messages()removed from public APIselect_stream_messages()removed from public APIget_headers()returnsList(Header)not tuplesStreamStart/StreamEnduseList(Header)Ready for release to hex.pm.