Skip to content

Conversation

@Aditya1404Sal
Copy link
Contributor

@Aditya1404Sal Aditya1404Sal commented Nov 21, 2025

Feature or Problem

This PR adds comprehensive testing and example components for streaming HTTP responses in wasmCloud, providing:

  • Integration tests to verify streaming behavior
  • Example components demonstrating real-world streaming use cases
  • Documentation through working examples

The streaming capability is essential for:

  • AI/LLM applications that stream generated text token-by-token
  • Large file downloads with progress feedback
  • Server-Sent Events (SSE) style responses
  • Any scenario requiring low time-to-first-byte

Related Issues

Release Information

Target: next release

Consumer Impact

Breaking Changes: None

Benefits:

  • Validated streaming behavior through comprehensive integration tests
  • Reference implementations for developers building streaming components
  • Documented patterns for AI streaming and SSE-style responses

Testing

Unit Test(s)

  • No new unit tests (streaming support tested via integration tests)

Acceptance or Integration

Added comprehensive integration test: tests/integration_http_streaming.rs

Test Design:

  • Uses a simplified streaming component that generates 20 random data chunks at fixed 0.5-second intervals
  • Validates precise timing and chunk count requirements for reliable CI/CD testing
  • Independent of external API dependencies (no API key expiration failures)

Example Components Included:

  1. http_streaming.wasm - Deterministic test component for CI/CD
  2. http_ai_proxy.wasm - Real-world Gemini API proxy demonstrating AI streaming use cases

Test Assertions:

  • Verifies exactly 20 chunks are received
  • Validates total streaming duration is approximately 10 seconds (9-11s range)
  • Confirms each chunk arrives at ~0.5 second intervals (0.4-0.6s range)
  • Ensures response is non-empty and properly formatted
  • Tests that streaming works correctly

Why this approach?
The integration test prioritizes reliability and determinism over realism. Using a controlled streaming pattern ensures tests won't fail due to:

  • Expired API keys
  • Network connectivity issues
  • External service rate limits
  • Variable upstream response times

The real-world Gemini proxy example is included to demonstrate practical streaming applications, but the CI test uses a predictable pattern.

Manual Verification

  1. Built and ran both components:

    • http_streaming.wasm (test component with deterministic timing)
    • http_ai_proxy.wasm (Gemini proxy example for real-world use)
  2. Deterministic Test Component:

✓ All streaming assertions passed!
  - Received exactly 20 chunks
  - Total time: 10.09s (~10s expected)
  - Average interval: 0.504s (~0.5s expected)
  1. Gemini Proxy Example (Real-world streaming):
  • Sent POST request with prompt: "Explain how AI works"
  • Observed streamed response chunks arriving progressively
  • Verified components can continue streaming after the handler returns
  • Results:
✓ 29 chunks received
✓ 15,355 bytes total
✓ 9.83 seconds total duration
✓ Response includes valid SSE-formatted Gemini API data
✓ Content-Type: text/event-stream
✓ Transfer-Encoding: chunked

Observed streaming behavior (Gemini example):

  • First chunk arrives at 10μs (immediate header response)
  • Subsequent chunks arrive every 300-500ms (realistic streaming from Gemini API)
  • Average chunk size: ~530 bytes per chunk
  • Total response: 1,512 tokens streamed progressively

Both approaches confirm that the streaming implementation works correctly and components can continue streaming after the handler returns.

@Aditya1404Sal Aditya1404Sal requested a review from a team as a code owner November 21, 2025 07:54
cpu_limit: 2,
config: HashMap::new(),
environment: HashMap::new(),
volume_mounts: vec![],
Copy link

@thomas9911 thomas9911 Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont know if LocaResources implemented Default, might be cleaner to use that instead:

                local_resources: LocalResources {
                    allowed_hosts: vec!["generativelanguage.googleapis.com".to_string()],
                    ..Default::default()
                }

Copy link
Contributor Author

@Aditya1404Sal Aditya1404Sal Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, LocalResources::default() is present

@Aditya1404Sal Aditya1404Sal force-pushed the feat_http_stream branch 4 times, most recently from aab131e to 81e45c9 Compare December 9, 2025 01:27
@Aditya1404Sal
Copy link
Contributor Author

CC @lxfontes pretty sure the CI failure is a flake, So it's ready for a review

@Aditya1404Sal
Copy link
Contributor Author

@if0ne it seems that #195 had the same changes present in this PR, do you reckon if I should keep the streaming examples/moreover this entire PR open or just close it ?

@if0ne
Copy link
Contributor

if0ne commented Dec 10, 2025

I like the examples with streaming. Rebase on upstream/main.

use url::Url;

mod bindings {
wit_bindgen::generate!({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can make the generated async trait and get rid of the code related to the executor

https://docs.rs/wit-bindgen/latest/wit_bindgen/macro.generate.html#options-to-generate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's neat! didn't know that was possible 😮

@Aditya1404Sal Aditya1404Sal changed the title feat(host): add streaming HTTP support with integration test and example component feat(tests): add streaming HTTP integration test and example components Dec 10, 2025
@lxfontes lxfontes moved this to Discuss in wash v2 release Jan 7, 2026
…ni proxy component

Signed-off-by: Aditya <aditya.salunkh919@gmail.com>
Signed-off-by: Aditya <aditya.salunkh919@gmail.com>
…onent

Signed-off-by: Aditya <aditya.salunkh919@gmail.com>
@Aditya1404Sal
Copy link
Contributor Author

@lxfontes I've refactored the fixture part here and rebased, should be good for a review!

@lxfontes lxfontes moved this from Discuss to Todo in wash v2 release Jan 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

3 participants