Skip to content

Wrap channel in an asyncio.Transport to eliminate loop back connection#303

Open
bdraco wants to merge 106 commits intoNabuCasa:mainfrom
bdraco:direct_connect_v2
Open

Wrap channel in an asyncio.Transport to eliminate loop back connection#303
bdraco wants to merge 106 commits intoNabuCasa:mainfrom
bdraco:direct_connect_v2

Conversation

@bdraco
Copy link
Copy Markdown
Collaborator

@bdraco bdraco commented Oct 24, 2024

Breaking change

SniTunClientAioHttp.start() no longer uses endpoint_connection_error_callback. Passing it will now generate a DeprecationWarning that it will be removed in a future release. This failure condition is no longer possible since there is no longer a loop back connection, and the protocol is connected directly to the aiohttp RequestHandler (server).

hass_nabucasa currently uses this callback in https://github.com/NabuCasa/hass-nabucasa/blob/cb14a2cad747b882f407e72e69f539f5d8fa20bb/hass_nabucasa/remote.py#L321

Its unclear if hass_nabucasa can simply remove passing the callback or another change will be needed.

Technically Breaking change

The signature of snitun.client.connector.Connector has changed since it no longer connects to loop-back. Instead of end_host and end_port it takes the ssl_context and protocol_factory (an asyncio.Protocol factory, in Home Assistant's case this is an aiohttp RequestHandler)

This doesn't appear to be used externally as SniTunClientAioHttp is the entry point that creates these.

Deployment considerations

The new ChannelTransport is hard coded to pass the remote IP as 127.0.0.1 for compatibility with the currently deployed Cloud servers.

https://github.com/NabuCasa/snitun/pull/303/files#diff-962be76744351102b2907e3aab99d5a45bc091cac1cc65f53f0fe2db9033c47fR30 should be adjusted to True when channel.ip_address is the actual remote IP address of the client that connected to the cloud server and not the IP Address of the internal forwarder.

Proposed change

The concept is that the SelectorTransport is replaced with the new ChannelTransport that is a wrapper around the channel so we don't need to connect back to localhost since we can create the request handler, connect up the transport and protocol and start_tls

Since aiohttp gets the ip address from the transport it can be set to whatever is desired in a future up. See Deployment considerations

Testing TODO:

@bdraco bdraco changed the title Direct connect PoC Wrap channel in an asyncio.Transport to eliminate loop back connection PoC Oct 24, 2024
@bdraco

This comment was marked as outdated.

@bdraco
Copy link
Copy Markdown
Collaborator Author

bdraco commented Feb 8, 2025

Last step is to replace all the connector tests with ones that test connecting to the handler

@bdraco bdraco changed the title Wrap channel in an asyncio.Transport to eliminate loop back connection PoC Wrap channel in an asyncio.Transport to eliminate loop back connection Feb 8, 2025
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 8, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@8315a3f). Learn more about missing BASE report.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #303   +/-   ##
=======================================
  Coverage        ?   84.59%           
=======================================
  Files           ?       20           
  Lines           ?     1298           
  Branches        ?      127           
=======================================
  Hits            ?     1098           
  Misses          ?      168           
  Partials        ?       32           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bdraco bdraco force-pushed the direct_connect_v2 branch 3 times, most recently from 51befd7 to 8757bea Compare February 8, 2025 20:40
@bdraco bdraco force-pushed the direct_connect_v2 branch from 27a149b to 04993e8 Compare February 8, 2025 21:03
@bdraco bdraco force-pushed the direct_connect_v2 branch 4 times, most recently from 31128d2 to ad98e36 Compare February 8, 2025 22:42
@bdraco bdraco marked this pull request as ready for review February 8, 2025 22:42
@bdraco bdraco force-pushed the direct_connect_v2 branch 2 times, most recently from 8bcc0f4 to ba155d2 Compare February 8, 2025 23:30
@bdraco
Copy link
Copy Markdown
Collaborator Author

bdraco commented Jul 19, 2025

Whatever changed in pytest-asyncio broke the test, and it's not immediately apparent what's wrong

@bdraco
Copy link
Copy Markdown
Collaborator Author

bdraco commented Feb 13, 2026

Almost working again except the test_connector tests

@bdraco
Copy link
Copy Markdown
Collaborator Author

bdraco commented Feb 13, 2026

we use abort in newer aiohttp .. and I didn't implement that

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the loopback TCP connection previously used to bridge SniTun channels into aiohttp, replacing it with an asyncio.Transport wrapper (ChannelTransport) so TLS (loop.start_tls) and the aiohttp request handler can be layered directly on top of a MultiplexerChannel.

Changes:

  • Introduces ChannelTransport and updates Connector/ConnectorHandler to wire channels directly into TLS + protocol handling (no localhost socket).
  • Deprecates endpoint_connection_error_callback in SniTunClientAioHttp.start() and updates the aiohttp client implementation accordingly.
  • Adds/updates end-to-end and transport-level tests (including TLS sanity checks via trustme) to validate backpressure/pause-resume and TLS behaviors.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
snitun/multiplexer/transport.py New asyncio.Transport wrapper around MultiplexerChannel used for direct TLS + protocol integration.
snitun/client/connector.py Refactors connector to use ChannelTransport + start_tls and connect protocols directly (no loopback).
snitun/utils/aiohttp_client.py Removes loopback SockSite usage; deprecates error callback; constructs Connector from aiohttp runner server factory.
snitun/multiplexer/channel.py Adds write_no_wait() and refactors message creation to support transport writes.
snitun/multiplexer/core.py Uses create_eager_task() for channel tasks.
tests/client/test_connector.py Reworks connector tests into end-to-end TLS/http flows via a custom aiohttp connector over multiplexer channels.
tests/client/helpers.py Adds a test-only aiohttp connector that routes requests over ChannelTransport + TLS for e2e testing.
tests/multiplexer/test_transport.py Adds focused tests for ChannelTransport reader lifecycle, pause/resume, and exception behavior.
tests/multiplexer/test_channel.py Adds coverage for queue-full behavior when writing without waiting.
tests/test_trustme_configuration.py Adds TLS loopback sanity test to validate trustme cert setup.
tests/conftest.py Adds TLS fixtures, aiohttp server fixtures, and end-to-end loopback wiring helpers.
tests/utils/test_aiohttp_client.py Updates tests for new aiohttp client behavior and deprecation warning.
tests/utils/test_server.py Removes unused test_endpoint dependency from fernet token tests.
tests/client/test_client_peer.py Removes test_endpoint dependency and switches to connector fixture.
pyproject.toml Adds trustme to test dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

data = data[chunk_size:]
if data and self._pause_future:
await self._pause_future
except (SystemExit, KeyboardInterrupt):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should asyncio.CancelledError be caught here as well, so it is properly caught and handled later, or is it also considered a fatal error?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants