Support IPv6 nodes and transport options by erhlee-bird · Pull Request #374 · whatyouhide/xandra

erhlee-bird · 2024-12-04T21:56:56Z

Hi there, thanks for the awesome library!

I'm running infrastructure on a platform called fly.io which runs IPv6 private networks.
I was unable to connect to my ScyllaDB cluster using Xandra and found that there were a few limitations to IPv6 support.

This pull request adds proposed fixes for the below 2 issues. I've added some tests and tried to ensure that this change would not break Xandra.start_link() and Xandra.Cluster.start_link() but there may be other places using these options that I may have missed.

In my fork, I'm now able to establish an IPv6 connection to my cluster and get kicked back due to lack of authentication whereas before connections would hang or hit sync_connection_timeout errors.

iex(1)> Xandra.start_link(nodes: ["my-scylla-cluster.internal"], transport_options: [:inet6])     
{:ok, #PID<0.2711.0>}
** (EXIT from #PID<0.2710.0>) shell process exited with reason: an exception was raised:
    ** (RuntimeError) Cassandra server requires authentication but the :authentication option was not provided
        (xandra 0.19.1) lib/xandra/protocol/v4.ex:48: Xandra.Protocol.V4.encode_request/3
        (xandra 0.19.1) lib/xandra/connection/utils.ex:143: Xandra.Connection.Utils.authenticate_connection/5
        (xandra 0.19.1) lib/xandra/connection.ex:458: Xandra.Connection.disconnected/3
        (stdlib 4.3.1.5) gen_statem.erl:1426: :gen_statem.loop_state_callback/11
        (stdlib 4.3.1.5) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Cannot specify IPv6 node addresses.

iex(1)> Xandra.start_link(nodes: ["::1"])   
** (NimbleOptions.ValidationError) invalid list in :nodes option: invalid value for list element at position 0: invalid node: "::1"
    (nimble_options 1.1.1) lib/nimble_options.ex:359: NimbleOptions.validate!/2
    (xandra 0.19.1) lib/xandra.ex:508: Xandra.start_link/1
    iex:1: (file)

Cannot pass `:inet6` as a transport option for `:gen_tcp`

iex(1)> Xandra.start_link(nodes: ["localhost"], transport_options: [:inet6])
** (NimbleOptions.ValidationError) invalid value for :transport_options option: expected keyword list, got: [:inet6]
    (nimble_options 1.1.1) lib/nimble_options.ex:359: NimbleOptions.validate!/2
    (xandra 0.19.1) lib/xandra.ex:508: Xandra.start_link/1
    iex:1: (file)

:gen_tcp accepts options that are not strictly keywords.

iex(2)> :gen_tcp.connect('my-scylla-cluster.internal', 9042, [:inet6])
{:ok, #Port<0.27>}
iex(3)> :gen_tcp.connect('my-scylla-cluster.internal', 9042, [{:inet6, true}])
{:error, :nxdomain}
iex(4)> :gen_tcp.connect('my-scylla-cluster.internal', 9042, [inet6: true])
{:error, :nxdomain}

If you want to support IPv6 connections, you need to support non-keyword options that :gen_tcp supports. https://www.erlang.org/docs/19/man/gen_tcp#type-option connect_option() supports inet:address_family() which is inet | inet6 | local.

… update validate_node to support IPv6 nodes

whatyouhide

Great catch, especially when it comes to non-keyword-value transport options. Left a couple of comments but this is already looking great!

whatyouhide · 2024-12-06T08:35:54Z

lib/xandra.ex

    ],
    transport_options: [
-      type: :keyword_list,
+      type: {:or, [:keyword_list, {:list, :any}]},


{:list, :any} is a superset of :keyword_list. Maybe we can go with this instead:

Suggested change

type: {:or, [:keyword_list, {:list, :any}]},

type: {:list, {:or, [{:tuple, [:atom, :any]}, :atom]}},

or just go with {:list, :any}.

I see that the :gen_tcp options support additionally the inet_family atoms as well as lists and binaries. I don't have any examples of what those might be, but, if you're okay with it, we could start with {:list, :any}.

option() = {active, true | false | once | -32768..32767} | ... {low_watermark, integer() >= 0} | {mode, list | binary} | list | binary | ...

Yep sure let's go with tat.

whatyouhide · 2024-12-06T08:36:54Z

lib/xandra/cluster/control_connection.ex

+    # Construct the transport options.
+    {keyword_options, other_options} =
+      Enum.split_with(transport_opts, fn x -> Keyword.keyword?([x]) end)


You don't need to do this. Do:

options: @forced_transport_options ++ transport_opts

as putting the forced options at the beginning overrides later options. Please double check this but I’m pretty sure 🙃

I added a conflicting transport option here: {:active, true} which causes the connection to fail.

I tried both flipped configurations to see how ordering affected option precedence, but in both cases the connection failed.

options: @forced_transport_options ++ transport_opts and options: transport_opts ++ @forced_transport_options

{:ok, conn} = Xandra.Cluster.start_link(nodes: ["my-scylla-cluster.internal"], transport_options: [:inet6, {:active, false}], authentication: {Xandra.Authenticator.Password, [username: System.get_env("SCYLLA_USERNAME"), password: System.get_env("SCYLLA_PASSWORD")]}, sync_connect: 1000)

IO.inspect(@forced_transport_options ++ transport_opts) IO.inspect(transport_opts ++ @forced_transport_options) # [{:packet, :raw}, {:mode, :binary}, {:active, false}, :inet6, {:active, true}] # [:inet6, {:active, true}, {:packet, :raw}, {:mode, :binary}, {:active, false}]

whatyouhide · 2024-12-06T08:41:06Z

lib/xandra/connection.ex

        nil -> data.original_options
      end

+    # Construct the transport options.


Same here, but you can use proplists:

transport_options = Keyword.get(options, :transport_options, []) buffer = :proplists.get_value(transport_options, :buffer, @default_transport_buffer_size) transport_options = [buffer: buffer] ++ @forced_transport_options ++ :proplists.delete(:buffer, transport_options)

Should work but please double check 😬

This ran into the same issue as in the other comment where I could override the @forced_transport_options unexpectedly with {:active, true}.

Relevant to both of these comments, I've added tests to check if the forced transport opts are being respected.

@erhlee-bird mh I’m confused:

iex> {:ok, socket} = :gen_tcp.connect(~c"google.com", 80, [{:active, true}, {:active, false}]) {:ok, #Port<0.4>} iex> :inet.getopts(socket, [:active]) {:ok, [active: false]}

Seems like overriding works here?

whatyouhide · 2024-12-06T08:50:28Z

lib/xandra/options_validators.ex

-
-      [address] ->
-        {:ok, {address, 9042}}
+    # Remove surrounding square brackets from IPv6 values.


Instead of this logic, we could "pop" out the port like this, regardless of the type of IP using Regex.split/2 with lookahead:

case Regex.split(~r/:(?=\d+$)/, value) do [address, port] -> # Same as before [address] -> # Also same as before end

This way we don't have to parse the IPv6 address, we can just leave what the user passed in

I thought about doing that, but there's ambiguity if you have IPv6 addresses without a port. The first 2 examples below using the Regex method above parse the address incorrectly.

iex(3)> Xandra.OptionsValidators.validate_node("::1") {:ok, {":", 1}} iex(4)> Xandra.OptionsValidators.validate_node("2001:db8::1") {:ok, {"2001:db8:", 1}} iex(5)> Xandra.OptionsValidators.validate_node("[2001:db8::1]:9042") {:ok, {"2001:db8::1", 9042}}

Works with a lookbehind assertion:

iex(2)> Regex.split(~r/(?<!:):(?=\d+$)/, "::1") ["::1"] iex(3)> Regex.split(~r/(?<!:):(?=\d+$)/, "2001:db8::1") ["2001:db8::1"] iex(5)> Regex.split(~r/(?<!:):(?=\d+$)/, "[2001:db8::1]:9042") ["[2001:db8::1]", "9042"]

(need to then address |> String.trim_leading("[") |> String.trim_trailing("]"))

erhlee-bird added 3 commits December 4, 2024 14:57

fix: add tests for IPv6 transports

f5e8052

fix: update Xandra.Cluster to pass in transport_options correctly and…

3305615

… update validate_node to support IPv6 nodes

whatyouhide reviewed Dec 6, 2024

View reviewed changes

chore: add tests to ensure that forced transport options are respected

6bfa807

	type: {:or, [:keyword_list, {:list, :any}]},
	type: {:list, {:or, [{:tuple, [:atom, :any]}, :atom]}},

Uh oh!

Conversation

erhlee-bird commented Dec 4, 2024

Cannot specify IPv6 node addresses.

Cannot pass :inet6 as a transport option for :gen_tcp

Uh oh!

whatyouhide left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

whatyouhide Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Cannot pass `:inet6` as a transport option for `:gen_tcp`

whatyouhide Dec 7, 2024 •

edited

Loading