Complicated Determination Of SOMAXCONN

[ldo]

You have this routine _ThreadedTCPServer.get_somaxconn() in agi/fastagi.py which goes to a lot of trouble to figure out the value of the system constant SOMAXCONN.

This is completely unnecessary, since that constant has been in the Python socket module since just about forever—certainly since Python 2.7—in other words, in every version of Python you could conceivably care about. Just use that value.

[karthicraghupathi]

I agree with you that the value for SOMAXCONN in Python has been constant for a long time. Let me clarify that we are not attempting to determine what the constant value in Python is.

I'm copying and pasting a well written article from https://utcc.utoronto.ca/~cks/space/blog/python/AvoidSOMAXCONN that explains the rationale behind why we have implemented the get_somaxconn() routine.

---

You should avoid using socket.SOMAXCONN

February 24, 2013

There you are, writing a socket-based server in Python, and you've decided that you should heed my advice and make its listen() backlog as large as possible. So you write the following code:

s = socket.socket(....)
s.bind(....)
s.list(socket.SOMAXCONN)

Given that SOMAXCONN is the traditional BSD sockets API name for 'the maximum value you can give listen()', this looks good and should do what you want. Sadly it does not.

The problem is that socket.SOMAXCONN is a constant. On many modern systems, the system-wide maximum listen() value is a kernel tuneable; it can be adjusted up (or down). If it's adjusted down, there is no problem with this code since the kernel will reduce the listen() backlog value down to the system setting. But if the system limit has been raised, the kernel does not adjust your listen() backlog up for you; it quietly stays at what you told the kernel. Which means that if someone has adjusted the limit up on your system, your code is not actually specifying a maximum backlog.

I can't blame Python for this one, because Unix doesn't actually expose an API for finding out the maximum socket listen() backlog that's allowed. The best that the socket module can do is set SOMAXCONN to some compile-time value, and in this case it's using the C-level API SOMAXCONN constant definition (which is usually 128).

So what should your code do? Since the kernel will quietly limit the listen() backlog to the system maximum for you, the obvious answer is to specify a large number, or a very large number, or even an absurdly large number. My tastes run to:

s.listen(max(1024, socket.SOMAXCONN))

This is a hack, of course, and you can argue that 1024 is not large enough and so on (and also that being paranoid about SOMAXCONN being larger than 1024 is pointless, so I should take the max() out). This is just what I do.

You may even feel that a large listen() backlog is pointless because in reality clients will start timing out long before you can service their requests. This is also a valid view. The whole area is complicated. But I still think you should avoid using socket.SOMAXCONN because it simply doesn't mean what people might think it means, not in real life on real systems.

---

Also newer kernels come with higher SOMAXCONN values. The default value for net.core.somaxconn comes from the SOMAXCONN constant, which is set to 128 on linux kernels up through v5.3, while SOMAXCONN was raised to 4096 in v5.4.

In our production systems, we have certainly hit the limit associated with the default value of SOMAXCONN Python comes with. In fact our values are higher than the new kernel defaults. The routine allows us the freedom to tune our machines and the code automatically uses the updated value without further intervention.

[ldo]

Interesting. I just checked my Linux system (Debian Unstable, kernel version 6.1.0-9). The definition of SOMAXCONN comes from /usr/include/x86_64-linux-gnu/bits/socket.h, and equals 4096. And when I check the actual kernel value with

/usr/sbin/sysctl net.core.somaxconn

it returns ... 4096.

Note also this part from the listen(2) man page:

The behavior of the backlog argument on TCP sockets changed with Linux 2.2. Now it specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests. The maximum length of the queue for incomplete sockets can be set using /proc/sys/net/ipv4/tcp_max_syn_backlog. When syncookies are enabled there is no logical maximum length and this setting is ignored. See tcp(7) for more information.

[ldo]

If you really want to get the system-configured dynamic value, don’t spawn a separate process, just read it directly, e.g.

print(int(open("/proc/sys/net/core/somaxconn", "rt").read().strip()))

[karthicraghupathi]

That would work if we were only dealing with linux machines. Unfortunately the macOS does not have a procfs. You'll need to spawn a subprocess to use sysctl to read these values. The current implementation works on macOS and Linux. The function runs once when the FastAGI server starts.

[clearinterface]

The code is getting used and taking massive volume of calls.

Developers using various different OS make changes tk fastagi scripts and test it locally.

What Karthic says is true as most of us use mac os. Unless this causes severe issue then the changes is not necessary.

[ldo]

I’m sure somebody else can come up with a corresponding improvement for particular proprietary systems. All we need is somebody with expertise in them to contribute.

My contribution is to improve things under Linux.

[ldo]

Did you know you can specify 0 as the backlog value?

[ldo]

I’ve been looking a little into the Linux kernel source code, to try to understand how this backlog value is actually interpreted.

It seems to me that raising this limit isn’t going to improve overall performance, it’s just going to help somewhat with transient peaks in the rate of incoming requests. If the sustained connection rate is larger than you can handle, then no amount of increasing the backlog will help.

Something that might help with dealing with higher connection rates is the TCP_FASTOPEN socket option documented in tcp(7).