Implement a fast lookup for wildcard certificates #2815

arontsang · 2025-04-04T07:59:49Z

No description provided.

gfoidl

Nit: there are quite a few blank lines that could be removed too.

gfoidl · 2025-05-09T08:24:38Z

src/Kubernetes.Controller/Certificates/ImmutableCertificateCache.cs

+                var charA = x[^i] & 0x5F;
+                var charB = y[^i] & 0x5F;


You compare it backwards?
Is it better then, to write the loop backwards and have normal index access?

gfoidl · 2025-05-09T08:29:06Z

src/Kubernetes.Controller/Certificates/ImmutableCertificateCache.cs

+            var length = Math.Min(x.Length, y.Length);
+
+            for (var i = 1; i <= length; i++)


So the JIT can't elide the bound-check for indexing into the spans.
I think it's better to slice the spans to the common (= min) length and use one span in the loop, in order to elide at least one bound check (the JIT should handle the downward loop (see other comment) in a recent version).

Roughly:

Suggested change

var length = Math.Min(x.Length, y.Length);

for (var i = 1; i <= length; i++)

if (x.Length < y.Length)

{

y = y.Slice(0, x.Length);

}

else if (y.Length < x.Length)

{

x = x.Slice(0, y.Length);

}

for (var i = x.Length; i >= 1; --i)

KLuuKer · 2025-10-28T07:37:38Z

src/Kubernetes.Controller/Certificates/ServerCertificateSelector.cs

+        _hasBeenUpdated = true;
    }

    public X509Certificate2 GetCertificate(ConnectionContext connectionContext, string domainName)


domainName CAN BE NULL!
when SNI is not available.

https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.server.kestrel.https.httpsconnectionadapteroptions.servercertificateselector?view=aspnetcore-9.0

Well poop, forgot about SNI. I would expect that everything supports SNI by now. But yes, get your point...sigh...

i would suggest skipping certificate lookups and immediately fallback to the default cert when string.IsNullOrEmpty
you WILL get this traffic anytime someone runs ssllabs server test, or some other kind of auditing tool
or for the handful of people running this on their own internal network and connecting with some kind of weird legacy system

Updated to handle non SNI TLS hand shake.

KLuuKer · 2025-11-11T05:04:18Z

If this pull also needs to work for the Kubernetes.Controller the following needs to be modified also

https://github.com/dotnet/yarp/blob/785e5fc80540baf3f34db17686d90cad595000a3/src/Kubernetes.Controller/Caching/IngressCache.cs#L100C9-L103C10

note: i don't work for MS but i have this code running right now with those lines patched out.
and from my limited testing (cloudflare strict certificate check) it seems to work.

arontsang · 2025-11-12T10:24:37Z

If this pull also needs to work for the Kubernetes.Controller the following needs to be modified also

https://github.com/dotnet/yarp/blob/785e5fc80540baf3f34db17686d90cad595000a3/src/Kubernetes.Controller/Caching/IngressCache.cs#L100C9-L103C10

note: i don't work for MS but i have this code running right now with those lines patched out. and from my limited testing (cloudflare strict certificate check) it seems to work.

Thanks for that

MihaZupan

Looks like there are some related build & test failures in CI

MihaZupan · 2025-12-03T12:46:28Z

src/Kubernetes.Controller/Caching/IngressCache.cs

        if (!string.Equals(namespacedName.ToString(), _options.DefaultSslCertificate, StringComparison.OrdinalIgnoreCase))
        {
-            return;
+            _logger.LogInformation("Found secret `{NamespacedName}` to use as default certificate for HTTPS traffic", namespacedName);


Are these if/else logs inverted? (note the !string.Equals)

MihaZupan · 2025-12-03T12:59:42Z

src/Kubernetes.Controller/Caching/IngressCache.cs

+        }
+        else
+        {
+            _logger.LogInformation("Found secret `{NamespacedName}` to use for HTTPS traffic", namespacedName);


This might result in a bunch of misleading logs (including warnings) since we're assuming all secrets are certs

yarp/src/Kubernetes.Controller/Certificates/CertificateHelper.cs

Lines 33 to 37 in e04fc5a

if (cert == null || cert.Length == 0 || privateKey == null || privateKey.Length == 0)

{

_logger.LogWarning("TLS secret '{NamespacedName}' contains invalid data.", namespacedName);

return null;

}

We should probably check that it's a relevant secret before logging.

MihaZupan · 2025-12-03T13:15:58Z

src/Kubernetes.Controller/Certificates/ImmutableCertificateCache.cs

+#nullable enable
+namespace Yarp.Kubernetes.Controller.Certificates;
+
+public abstract class ImmutableCertificateCache<TCert> where TCert : class


Please make this internal.

Why does it need to be abstract and generic? We should be able to merge this and ImmutableX509CertificateCache into one type.

MihaZupan · 2025-12-03T13:17:12Z

src/Kubernetes.Controller/Certificates/ServerCertificateSelector.cs

+        // Poll every 10 seconds for updates to
+        while (!stoppingToken.IsCancellationRequested)
+        {
+            await Task.Delay(TimeSpan.FromSeconds(10), stoppingToken);


Nit: this can use PeriodicTimer instead

MihaZupan · 2025-12-03T13:17:51Z

src/Kubernetes.Controller/Certificates/ServerCertificateSelector.cs

+            if (_hasBeenUpdated)
+            {
+                _hasBeenUpdated = false;


We might want to handle _hasBeenUpdated changes under a lock so that we don't accidentally miss the last update in a race condition

MihaZupan · 2025-12-03T13:20:10Z

src/Kubernetes.Controller/Certificates/ImmutableCertificateCache.cs

+    }
+
+
+
+    protected abstract TCert? GetDefaultCertificate();


Suggested change

}

protected abstract TCert? GetDefaultCertificate();

}

protected abstract TCert? GetDefaultCertificate();

Please remove all the extra new lines (across all touched files)

MihaZupan · 2025-12-03T13:20:32Z

src/Kubernetes.Controller/Certificates/ImmutableCertificateCache.cs

+    /// This allows us to use a Binary Search to achieve a suffix
+    /// search.
+    /// </summary>
+    private class DomainNameComparer : IComparer<WildCardDomain>


Suggested change

private class DomainNameComparer : IComparer<WildCardDomain>

private sealed class DomainNameComparer : IComparer<WildCardDomain>

MihaZupan · 2025-12-03T13:23:22Z

src/Kubernetes.Controller/Certificates/ImmutableCertificateCache.cs

+                var charA = x[index] & 0x5F;
+                var charB = y[index] & 0x5F;


This looks like it might corrupt various characters. If the intention is to normalize the case for only ASCII characters, these should first check that these are ASCII letters.

MihaZupan · 2025-12-03T13:25:24Z

src/Kubernetes.Controller/Certificates/ServerCertificateSelector.CertificateCache.cs

+        }
+
+    }
+
+
+
+}


Suggested change

}

}

}

}

}

}

MihaZupan · 2025-12-03T13:28:15Z

src/Kubernetes.Controller/Certificates/ImmutableCertificateCache.cs

+        {
+            foreach (var domain in getDomains(certificate))
+            {
+                if (domain.StartsWith("*."))


Suggested change

if (domain.StartsWith("*."))

if (domain.StartsWith("*.", StringComparison.Ordinal))

arontsang requested review from MihaZupan and benjaminpetit as code owners April 4, 2025 07:59

arontsang force-pushed the feature/fast-wildcard-certificate-lookup branch 3 times, most recently from 5540ac3 to d979058 Compare April 5, 2025 10:16

gfoidl reviewed May 9, 2025

View reviewed changes

KLuuKer reviewed Oct 28, 2025

View reviewed changes

arontsang added 4 commits November 11, 2025 14:35

Implement a fast lookup for wildcard certificates

ac3d38d

Refactor for Unit Tests

9fc3a43

Fix code and add Unit Tests for CertificateCache

6f7c9a7

Switch to using record struct for better cache locality

0d20eac

arontsang force-pushed the feature/fast-wildcard-certificate-lookup branch from e56abc3 to 0d20eac Compare November 11, 2025 06:37

arontsang added 3 commits November 12, 2025 18:00

Reverse the loop logic for normal index access

d0cdea6

Handle non SNI TLS case

d13a5fb

Handle default certificate

d5a7a07

KLuuKer approved these changes Nov 16, 2025

View reviewed changes

KLuuKer mentioned this pull request Nov 28, 2025

I am unable to access the YARP Ingress via SSL. #2976

Open

MihaZupan reviewed Dec 3, 2025

View reviewed changes

		var length = Math.Min(x.Length, y.Length);

		for (var i = 1; i <= length; i++)

-            var length = Math.Min(x.Length, y.Length);
-            for (var i = 1; i <= length; i++)
+            if (x.Length < y.Length)
+            {
+                y = y.Slice(0, x.Length);
+            }
+            else if (y.Length < x.Length)
+            {
+                x = x.Slice(0, y.Length);
+            }
+            for (var i = x.Length; i >= 1; --i)

	if (cert == null \|\| cert.Length == 0 \|\| privateKey == null \|\| privateKey.Length == 0)
	{
	_logger.LogWarning("TLS secret '{NamespacedName}' contains invalid data.", namespacedName);
	return null;
	}

	private class DomainNameComparer : IComparer<WildCardDomain>
	private sealed class DomainNameComparer : IComparer<WildCardDomain>

+                      }
+                  }
+              }

-        }
-    }
-}
+        }
+    }
+}

	if (domain.StartsWith("*."))
	if (domain.StartsWith("*.", StringComparison.Ordinal))

Implement a fast lookup for wildcard certificates #2815

Are you sure you want to change the base?

Implement a fast lookup for wildcard certificates #2815

Uh oh!

Conversation

arontsang commented Apr 4, 2025

Uh oh!

gfoidl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KLuuKer Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KLuuKer Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KLuuKer commented Nov 11, 2025

Uh oh!

arontsang commented Nov 12, 2025

Uh oh!

MihaZupan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

KLuuKer Oct 28, 2025 •

edited

Loading

KLuuKer Nov 11, 2025 •

edited

Loading