Update dependency aiohttp to v3.10.11 [SECURITY] - autoclosed #36
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
==3.8.5->==3.10.11GitHub Vulnerability Alerts
CVE-2023-47627
Summary
The HTTP parser in AIOHTTP has numerous problems with header parsing, which could lead to request smuggling.
This parser is only used when
AIOHTTP_NO_EXTENSIONSis enabled (or not using a prebuilt wheel).Details
Bug 1: Bad parsing of
Content-LengthvaluesDescription
RFC 9110 says this:
AIOHTTP does not enforce this rule, presumably because of an incorrect usage of the builtin
intconstructor. Because theintconstructor accepts+and-prefixes, and digit-separating underscores, usingintto parse CL values leads AIOHTTP to significant misinterpretation.Examples
Suggested action
Verify that a
Content-Lengthvalue consists only of ASCII digits before parsing, as the standard requires.Bug 2: Improper handling of NUL, CR, and LF in header values
Description
RFC 9110 says this:
AIOHTTP's HTTP parser does not enforce this rule, and will happily process header values containing these three forbidden characters without replacing them with SP.
Examples
Suggested action
Reject all messages with NUL, CR, or LF in a header value. The translation to space thing, while technically allowed, does not seem like a good idea to me.
Bug 3: Improper stripping of whitespace before colon in HTTP headers
Description
RFC 9112 says this:
AIOHTTP does not enforce this rule, and will simply strip any whitespace before the colon in an HTTP header.
Example
Suggested action
Reject all messages with whitespace before a colon in a header field, as the standard requires.
PoC
Example requests are embedded in the previous section. To reproduce these bugs, start an AIOHTTP server without llhttp (i.e.
AIOHTTP_NO_EXTENSIONS=1) and send the requests given in the previous section. (e.g. byprintfing intonc)Impact
Each of these bugs can be used for request smuggling.
GHSA-pjjw-qhg8-p2p9
Summary
llhttp 8.1.1 is vulnerable to two request smuggling vulnerabilities.
Details have not been disclosed yet, so refer to llhttp for future information.
The issue is resolved by using llhttp 9+ (which is included in aiohttp 3.8.6+).
CVE-2023-49082
Summary
Improper validation makes it possible for an attacker to modify the HTTP request (e.g. insert a new header) or even create a new HTTP request if the attacker controls the HTTP method.
Details
The vulnerability occurs only if the attacker can control the HTTP method (GET, POST etc.) of the request.
Previous releases performed no validation on the provided value. If an attacker controls the HTTP method it will be used as is and can lead to HTTP request smuggling.
PoC
A minimal example can be found here:
https://gist.github.com/jnovikov/7f411ae9fe6a9a7804cf162a3bdbb44b
Impact
If the attacker can control the HTTP version of the request it will be able to modify the request (request smuggling).
Workaround
If unable to upgrade and using user-provided values for the request method, perform manual validation of the user value (e.g. by restricting it to a few known values like GET, POST etc.).
Patch: https://github.com/aio-libs/aiohttp/pull/7806/files
CVE-2023-49081
Summary
Improper validation make it possible for an attacker to modify the HTTP request (e.g. to insert a new header) or even create a new HTTP request if the attacker controls the HTTP version.
Details
The vulnerability only occurs if the attacker can control the HTTP version of the request (including its type).
For example if an unvalidated JSON value is used as a version and the attacker is then able to pass an array as the
versionparameter.Furthermore, the vulnerability only occurs when the
Connectionheader is passed to theheadersparameter.At this point, the library will use the parsed value to create the request. If a list is passed, then it bypasses validation and it is possible to perform CRLF injection.
PoC
The POC below shows an example of providing an unvalidated array as a version:
https://gist.github.com/jnovikov/184afb593d9c2114d77f508e0ccd508e
Impact
CRLF injection leading to Request Smuggling.
Workaround
If these specific conditions are met and you are unable to upgrade, then validate the user input to the
versionparameter to ensure it is astr.Patch: https://github.com/aio-libs/aiohttp/pull/7835/files
CVE-2024-23829
Summary
Security-sensitive parts of the Python HTTP parser retained minor differences in allowable character sets, that must trigger error handling to robustly match frame boundaries of proxies in order to protect against injection of additional requests. Additionally, validation could trigger exceptions that were not handled consistently with processing of other malformed input.
Details
These problems are rooted in pattern matching protocol elements, previously improved by PR #3235 and GHSA-gfw2-4jvh-wgfg:
The expression
HTTP/(\d).(\d)lacked another backslash to clarify that the separator should be a literal dot, not just any Unicode code point (result:HTTP/(\d)\.(\d)).The HTTP version was permitting Unicode digits, where only ASCII digits are standards-compliant.
Distinct regular expressions for validating HTTP Method and Header field names were used - though both should (at least) apply the common restrictions of rfc9110
token.PoC
GET / HTTP/1ö1GET / HTTP/1.𝟙GET/: HTTP/1.1Content-Encoding?: chunkedImpact
Primarily concerns running an aiohttp server without llhttp:
Patch: https://github.com/aio-libs/aiohttp/pull/8074/files
CVE-2024-23334
Summary
Improperly configuring static resource resolution in aiohttp when used as a web server can result in the unauthorized reading of arbitrary files on the system.
Details
When using aiohttp as a web server and configuring static routes, it is necessary to specify the root path for static files. Additionally, the option 'follow_symlinks' can be used to determine whether to follow symbolic links outside the static root directory. When 'follow_symlinks' is set to True, there is no validation to check if a given file path is within the root directory.This can lead to directory traversal vulnerabilities, resulting in unauthorized access to arbitrary files on the system, even when symlinks are not present.
i.e. An application is only vulnerable with setup code like:
Impact
This is a directory traversal vulnerability with CWE ID 22. When using aiohttp as a web server and enabling static resource resolution with
follow_symlinksset to True, it can lead to this vulnerability. This vulnerability has been present since the introduction of thefollow_symlinksparameter.Workaround
Even if upgrading to a patched version of aiohttp, we recommend following these steps regardless.
If using
follow_symlinks=Trueoutside of a restricted local development environment, disable the option immediately. This option is NOT needed to follow symlinks which point to a location within the static root directory, it is only intended to allow a symlink to break out of the static directory. Even with this CVE fixed, there is still a substantial risk of misconfiguration when using this option on a server that accepts requests from remote users.Additionally, aiohttp has always recommended using a reverse proxy server (such as nginx) to handle static resources and not to use these static resources in aiohttp for production environments. Doing so also protects against this vulnerability, and is why we expect the number of affected users to be very low.
Patch: https://github.com/aio-libs/aiohttp/pull/8079/files
CVE-2024-27306
Summary
A XSS vulnerability exists on index pages for static file handling.
Details
When using
web.static(..., show_index=True), the resulting index pages do not escape file names.If users can upload files with arbitrary filenames to the static directory, the server is vulnerable to XSS attacks.
Workaround
We have always recommended using a reverse proxy server (e.g. nginx) for serving static files. Users following the recommendation are unaffected.
Other users can disable
show_indexif unable to upgrade.Patch: https://github.com/aio-libs/aiohttp/pull/8319/files
CVE-2024-30251
Summary
An attacker can send a specially crafted POST (multipart/form-data) request. When the aiohttp server processes it, the server will enter an infinite loop and be unable to process any further requests.
Impact
An attacker can stop the application from serving requests after sending a single request.
For anyone needing to patch older versions of aiohttp, the minimum diff needed to resolve the issue is (located in
_read_chunk_from_length()):This does however introduce some very minor issues with handling form data. So, if possible, it would be recommended to also backport the changes in:
aio-libs/aiohttp@cebe526
aio-libs/aiohttp@7eecdff
aio-libs/aiohttp@f21c6f2
CVE-2024-52304
Summary
The Python parser parses newlines in chunk extensions incorrectly which can lead to request smuggling vulnerabilities under certain conditions.
Impact
If a pure Python version of aiohttp is installed (i.e. without the usual C extensions) or
AIOHTTP_NO_EXTENSIONSis enabled, then an attacker may be able to execute a request smuggling attack to bypass certain firewalls or proxy protections.Patch: aio-libs/aiohttp@259edc3
Release Notes
aio-libs/aiohttp (aiohttp)
v3.10.11Compare Source
====================
Bug fixes
Authentication provided by a redirect now takes precedence over provided
authwhen making requests with the client -- by :user:PLPeeters.Related issues and pull requests on GitHub:
:issue:
9436.Fixed :py:meth:
WebSocketResponse.close() <aiohttp.web.WebSocketResponse.close>to discard non-close messages within its timeout window after sending close -- by :user:lenard-mosys.Related issues and pull requests on GitHub:
:issue:
9506.Fixed a deadlock that could occur while attempting to get a new connection slot after a timeout -- by :user:
bdraco.The connector was not cancellation-safe.
Related issues and pull requests on GitHub:
:issue:
9670, :issue:9671.Fixed the WebSocket flow control calculation undercounting with multi-byte data -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9686.Fixed incorrect parsing of chunk extensions with the pure Python parser -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9851.Fixed system routes polluting the middleware cache -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9852.Removals and backward incompatible breaking changes
Improved performance of the connector when a connection can be reused -- by :user:
bdraco.If
BaseConnector.connecthas been subclassed and replaced with custom logic, theceil_timeoutmust be added.Related issues and pull requests on GitHub:
:issue:
9600.Miscellaneous internal changes
Improved performance of the client request lifecycle when there are no cookies -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9470.Improved performance of sending client requests when the writer can finish synchronously -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9485.Improved performance of serializing HTTP headers -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9603.Passing
enable_cleanup_closedto :py:class:aiohttp.TCPConnectoris now ignored on Python 3.12.7+ and 3.13.1+ since the underlying bug that caused asyncio to leak SSL connections has been fixed upstream -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
9726, :issue:9736.v3.10.10Compare Source
====================
Bug fixes
Fixed error messages from :py:class:
~aiohttp.resolver.AsyncResolverbeing swallowed -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
9451, :issue:9455.Features
Added :exc:
aiohttp.ClientConnectorDNSErrorfor differentiating DNS resolution errors from other connector errors -- by :user:mstojcevich.Related issues and pull requests on GitHub:
:issue:
8455.Miscellaneous internal changes
Simplified DNS resolution throttling code to reduce chance of race conditions -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9454.v3.10.9Compare Source
===================
Bug fixes
Fixed proxy headers being used in the
ConnectionKeyhash when a proxy was not being used -- by :user:bdraco.If default headers are used, they are also used for proxy headers. This could have led to creating connections that were not needed when one was already available.
Related issues and pull requests on GitHub:
:issue:
9368.Widened the type of the
trace_request_ctxparameter of:meth:
ClientSession.request() <aiohttp.ClientSession.request>and friends-- by :user:
layday.Related issues and pull requests on GitHub:
:issue:
9397.Removals and backward incompatible breaking changes
Fixed failure to try next host after single-host connection timeout -- by :user:
brettdh.The default client :class:
aiohttp.ClientTimeoutparams has changed to include asock_connecttimeout of 30 seconds so that this correct behavior happens by default.Related issues and pull requests on GitHub:
:issue:
7342.Miscellaneous internal changes
Improved performance of resolving hosts with Python 3.12+ -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9342.Reduced memory required for timer objects created during the client request lifecycle -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9406.v3.10.8Compare Source
===================
Bug fixes
Fixed cancellation leaking upwards on timeout -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9326.v3.10.7Compare Source
===================
Bug fixes
Fixed assembling the :class:
~yarl.URLfor web requests when the host contains a non-default port or IPv6 address -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
9309.Miscellaneous internal changes
Improved performance of determining if a URL is absolute -- by :user:
bdraco.The property :attr:
~yarl.URL.absoluteis more performant than the methodURL.is_absolute()and preferred when newer versions of yarl are used.Related issues and pull requests on GitHub:
:issue:
9171.Replaced code that can now be handled by
yarl-- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
9301.v3.10.6Compare Source
===================
Bug fixes
Added :exc:
aiohttp.ClientConnectionResetError. Client code that previously threw :exc:ConnectionResetErrorwill now throw this -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9137.Fixed an unclosed transport
ResourceWarningon web handlers -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8875.Fixed resolve_host() 'Task was destroyed but is pending' errors -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8967.Fixed handling of some file-like objects (e.g.
tarfile.extractfile()) which raiseAttributeErrorinstead ofOSErrorwhenfilenofails for streaming payload data -- by :user:ReallyReivax.Related issues and pull requests on GitHub:
:issue:
6732.Fixed web router not matching pre-encoded URLs (requires yarl 1.9.6+) -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8898, :issue:9267.Fixed an error when trying to add a route for multiple methods with a path containing a regex pattern -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8998.Fixed
Response.textwhen body is aPayload-- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
6485.Fixed compressed requests failing when no body was provided -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9108.Fixed client incorrectly reusing a connection when the previous message had not been fully sent -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8992.Fixed race condition that could cause server to close connection incorrectly at keepalive timeout -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9140.Fixed Python parser chunked handling with multiple Transfer-Encoding values -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8823.Fixed error handling after 100-continue so server sends 500 response instead of disconnecting -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8876.Stopped adding a default Content-Type header when response has no content -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8858.Added support for URL credentials with empty (zero-length) username, e.g.
https://:password@host-- by :user:shuckcRelated issues and pull requests on GitHub:
:issue:
6494.Stopped logging exceptions from
web.run_app()that would be raised regardless -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
6807.Implemented binding to IPv6 addresses in the pytest server fixture.
Related issues and pull requests on GitHub:
:issue:
4650.Fixed the incorrect use of flags for
getnameinfo()in the Resolver --by :user:GitNMLeeLink-Local IPv6 addresses can now be handled by the Resolver correctly.
Related issues and pull requests on GitHub:
:issue:
9032.Fixed StreamResponse.prepared to return True after EOF is sent -- by :user:
arthurdarcet.Related issues and pull requests on GitHub:
:issue:
5343.Changed
make_mocked_request()to use empty payload by default -- by :user:rahulnht.Related issues and pull requests on GitHub:
:issue:
7167.Used more precise type for
ClientResponseError.headers, fixing some type errors when using them -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8768.Changed behavior when returning an invalid response to send a 500 response -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8845.Fixed response reading from closed session to throw an error immediately instead of timing out -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8878.Fixed
CancelledErrorfrom one cleanup context stopping other contexts from completing -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8908.Fixed changing scheme/host in
Response.clone()for absolute URLs -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8990.Fixed
Site.namewhen host is an empty string -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8929.Updated Python parser to reject messages after a close message, matching C parser behaviour -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9018.Fixed creation of
SSLContextinside of :py:class:aiohttp.TCPConnectorwith multiple event loops in different threads -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
9029.Fixed (on Python 3.11+) some edge cases where a task cancellation may get incorrectly suppressed -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9030.Fixed exception information getting lost on
HttpProcessingError-- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9052.Fixed
If-None-Matchnot using weak comparison -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9063.Fixed badly encoded charset crashing when getting response text instead of falling back to charset detector.
Related issues and pull requests on GitHub:
:issue:
9160.Rejected
\ninreasonvalues to avoid sending broken HTTP messages -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9167.Changed :py:meth:
ClientResponse.raise_for_status() <aiohttp.ClientResponse.raise_for_status>to only release the connection when invoked outside anasync withcontext -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9239.Features
Improved type on
paramsto match the underlying type allowed byyarl-- by :user:lpetre.Related issues and pull requests on GitHub:
:issue:
8564.Declared Python 3.13 supported -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8748.Removals and backward incompatible breaking changes
Improved middleware performance -- by :user:
bdraco.The
set_current_appmethod was removed fromUrlMappingMatchInfobecause it is no longer used, and it was unlikely external caller would ever use it.Related issues and pull requests on GitHub:
:issue:
9200.Increased minimum yarl version to 1.12.0 -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9267.Improved documentation
Clarified that
GracefulExitneeds to be handled inAppRunnerandServerRunnerwhen usinghandle_signals=True. -- by :user:Daste745Related issues and pull requests on GitHub:
:issue:
4414.Clarified that auth parameter in ClientSession will persist and be included with any request to any origin, even during redirects to different origins. -- by :user:
MaximZemskov.Related issues and pull requests on GitHub:
:issue:
6764.Clarified which timeout exceptions happen on which timeouts -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8968.Updated
ClientSessionparameters to match current code -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8991.Packaging updates and notes for downstreams
Fixed
test_client_session_timeout_zeroto not require internet access -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
9004.Miscellaneous internal changes
Improved performance of making requests when there are no auto headers to skip -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8847.Exported
aiohttp.TraceRequestHeadersSentParams-- by :user:Hadock-is-ok.Related issues and pull requests on GitHub:
:issue:
8947.Avoided tracing overhead in the http writer when there are no active traces -- by user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9031.Improved performance of reify Cython implementation -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9054.Use :meth:
URL.extend_query() <yarl.URL.extend_query>to extend query params (requires yarl 1.11.0+) -- by :user:bdraco.If yarl is older than 1.11.0, the previous slower hand rolled version will be used.
Related issues and pull requests on GitHub:
:issue:
9068.Improved performance of checking if a host is an IP Address -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9095.Significantly improved performance of middlewares -- by :user:
bdraco.The construction of the middleware wrappers is now cached and is built once per handler instead of on every request.
Related issues and pull requests on GitHub:
:issue:
9158, :issue:9170.Improved performance of web requests -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9168, :issue:9169, :issue:9172, :issue:9174, :issue:9175, :issue:9241.Improved performance of starting web requests when there is no response prepare hook -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9173.Significantly improved performance of expiring cookies -- by :user:
bdraco.Expiring cookies has been redesigned to use :mod:
heapqinstead of a linear search, to better scale.Related issues and pull requests on GitHub:
:issue:
9203.Significantly sped up filtering cookies -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9204.v3.10.5Compare Source
=========================
Bug fixes
Fixed :meth:
aiohttp.ClientResponse.json()not settingstatuswhen :exc:aiohttp.ContentTypeErroris raised -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
8742.Miscellaneous internal changes
Improved performance of the WebSocket reader -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8736, :issue:8747.v3.10.4Compare Source
===================
Bug fixes
Fixed decoding base64 chunk in BodyPartReader -- by :user:
hyzyla.Related issues and pull requests on GitHub:
:issue:
3867.Fixed a race closing the server-side WebSocket where the close code would not reach the client -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8680.Fixed unconsumed exceptions raised by the WebSocket heartbeat -- by :user:
bdraco.If the heartbeat ping raised an exception, it would not be consumed and would be logged as an warning.
Related issues and pull requests on GitHub:
:issue:
8685.Fixed an edge case in the Python parser when chunk separators happen to align with network chunks -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8720.Improved documentation
Added
aiohttp-apischemato supported libraries -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8700.Miscellaneous internal changes
Improved performance of starting request handlers with Python 3.12+ -- by :user:
bdraco.This change is a followup to :issue:
8661to make the same optimization for Python 3.12+ where the request is connected.Related issues and pull requests on GitHub:
:issue:
8681.v3.10.3Compare Source
========================
Bug fixes
Fixed multipart reading when stream buffer splits the boundary over several read() calls -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8653.Fixed :py:class:
aiohttp.TCPConnectordoing blocking I/O in the event loop to create theSSLContext-- by :user:bdraco.The blocking I/O would only happen once per verify mode. However, it could cause the event loop to block for a long time if the
SSLContextcreation is slow, which is more likely during startup when the disk cache is not yet present.Related issues and pull requests on GitHub:
:issue:
8672.Miscellaneous internal changes
Improved performance of :py:meth:
~aiohttp.ClientWebSocketResponse.receiveand :py:meth:~aiohttp.web.WebSocketResponse.receivewhen there is no timeout. -- by :user:bdraco.The timeout context manager is now avoided when there is no timeout as it accounted for up to 50% of the time spent in the :py:meth:
~aiohttp.ClientWebSocketResponse.receiveand :py:meth:~aiohttp.web.WebSocketResponse.receivemethods.Related issues and pull requests on GitHub:
:issue:
8660.Improved performance of starting request handlers with Python 3.12+ -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8661.Improved performance of HTTP keep-alive checks -- by :user:
bdraco.Previously, when processing a request for a keep-alive connection, the keep-alive check would happen every second; the check is now rescheduled if it fires too early instead.
Related issues and pull requests on GitHub:
:issue:
8662.Improved performance of generating random WebSocket mask -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8667.v3.10.2Compare Source
===================
Bug fixes
Fixed server checks for circular symbolic links to be compatible with Python 3.13 -- by :user:
steverep.Related issues and pull requests on GitHub:
:issue:
8565.Fixed request body not being read when ignoring an Upgrade request -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8597.Fixed an edge case where shutdown would wait for timeout when the handler was already completed -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8611.Fixed connecting to
npipe://,tcp://, andunix://urls -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
8632.Fixed WebSocket ping tasks being prematurely garbage collected -- by :user:
bdraco.There was a small risk that WebSocket ping tasks would be prematurely garbage collected because the event loop only holds a weak reference to the task. The garbage collection risk has been fixed by holding a strong reference to the task. Additionally, the task is now scheduled eagerly with Python 3.12+ to increase the chance it can be completed immediately and avoid having to hold any references to the task.
Related issues and pull requests on GitHub:
:issue:
8641.Fixed incorrectly following symlinks for compressed file variants -- by :user:
steverep.Related issues and pull requests on GitHub:
:issue:
8652.Removals and backward incompatible breaking changes
Removed
Request.wait_for_disconnection(), which was mistakenly added briefly in 3.10.0 -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8636.Contributor-facing changes
Fixed monkey patches for
Path.stat()andPath.is_dir()for Python 3.13 compatibility -- by :user:steverep.Related issues and pull requests on GitHub:
:issue:
8551.Miscellaneous internal changes
Improved WebSocket performance when messages are sent or received frequently -- by :user:
bdraco.The WebSocket heartbeat scheduling algorithm was improved to reduce the
asyncioscheduling overhead by decreasing the number ofasyncio.TimerHandlecreations and cancellations.Related issues and pull requests on GitHub:
:issue:
8608.Minor improvements to various type annotations -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8634.v3.10.1Compare Source
====================
Bug fixes
Authentication provided by a redirect now takes precedence over provided
authwhen making requests with the client -- by :user:PLPeeters.Related issues and pull requests on GitHub:
:issue:
9436.Fixed :py:meth:
WebSocketResponse.close() <aiohttp.web.WebSocketResponse.close>to discard non-close messages within its timeout window after sending close -- by :user:lenard-mosys.Related issues and pull requests on GitHub:
:issue:
9506.Fixed a deadlock that could occur while attempting to get a new connection slot after a timeout -- by :user:
bdraco.The connector was not cancellation-safe.
Related issues and pull requests on GitHub:
:issue:
9670, :issue:9671.Fixed the WebSocket flow control calculation undercounting with multi-byte data -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9686.Fixed incorrect parsing of chunk extensions with the pure Python parser -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9851.Fixed system routes polluting the middleware cache -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9852.Removals and backward incompatible breaking changes
Improved performance of the connector when a connection can be reused -- by :user:
bdraco.If
BaseConnector.connecthas been subclassed and replaced with custom logic, theceil_timeoutmust be added.Related issues and pull requests on GitHub:
:issue:
9600.Miscellaneous internal changes
Improved performance of the client request lifecycle when there are no cookies -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9470.Improved performance of sending client requests when the writer can finish synchronously -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9485.Improved performance of serializing HTTP headers -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
9603.Passing
enable_cleanup_closedto :py:class:aiohttp.TCPConnectoris now ignored on Python 3.12.7+ and 3.13.1+ since the underlying bug that caused asyncio to leak SSL connections has been fixed upstream -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
9726, :issue:9736.v3.10.0Compare Source
========================
Bug fixes
Fixed server response headers for
Content-TypeandContent-Encodingforstatic compressed files -- by :user:
steverep.Server will now respond with a
Content-Typeappropriate for the compressedfile (e.g.
"application/gzip"), and omit theContent-Encodingheader.Users should expect that most clients will no longer decompress such responses
by default.
Related issues and pull requests on GitHub:
:issue:
4462.Fixed duplicate cookie expiration calls in the CookieJar implementation
Related issues and pull requests on GitHub:
:issue:
7784.Adjusted
FileResponseto check file existence and access when preparing the response -- by :user:steverep.The :py:class:
~aiohttp.web.FileResponseclass was modified to respond with403 Forbidden or 404 Not Found as appropriate. Previously, it would cause a
server error if the path did not exist or could not be accessed. Checks for
existence, non-regular files, and permissions were expected to be done in the
route handler. For static routes, this now permits a compressed file to exist
without its uncompressed variant and still be served. In addition, this
changes the response status for files without read permission to 403, and for
non-regular files from 404 to 403 for consistency.
Related issues and pull requests on GitHub:
:issue:
8182.Fixed
AsyncResolverto matchThreadedResolverbehavior-- by :user:
bdraco.On system with IPv6 support, the :py:class:
~aiohttp.resolver.AsyncResolverwould not fallbackto providing A records when AAAA records were not available.
Additionally, unlike the :py:class:
~aiohttp.resolver.ThreadedResolver, the :py:class:~aiohttp.resolver.AsyncResolverdid not handle link-local addresses correctly.
This change makes the behavior consistent with the :py:class:
~aiohttp.resolver.ThreadedResolver.Related issues and pull requests on GitHub:
:issue:
8270.Fixed
ws_connectnot respectingreceive_timeout`` on WS(S) connection. -- by :user:arcivanov`.Related issues and pull requests on GitHub:
:issue:
8444.Removed blocking I/O in the event loop for static resources and refactored
exception handling -- by :user:
steverep.File system calls when handling requests for static routes were moved to a
separate thread to potentially improve performance. Exception handling
was tightened in order to only return 403 Forbidden or 404 Not Found responses
for expected scenarios; 500 Internal Server Error would be returned for any
unknown errors.
Related issues and pull requests on GitHub:
:issue:
8507.Features
Added a Request.wait_for_disconnection() method, as means of allowing request handlers to be notified of premature client disconnections.
Related issues and pull requests on GitHub:
:issue:
2492.Added 5 new exceptions: :py:exc:
~aiohttp.InvalidUrlClientError, :py:exc:~aiohttp.RedirectClientError,:py:exc:
~aiohttp.NonHttpUrlClientError, :py:exc:~aiohttp.InvalidUrlRedirectClientError,:py:exc:
~aiohttp.NonHttpUrlRedirectClientError:py:exc:
~aiohttp.InvalidUrlRedirectClientError, :py:exc:~aiohttp.NonHttpUrlRedirectClientErrorare raised instead of :py:exc:
ValueErroror :py:exc:~aiohttp.InvalidURLwhen the redirect URL is invalid. Classes:py:exc:
~aiohttp.InvalidUrlClientError, :py:exc:~aiohttp.RedirectClientError,:py:exc:
~aiohttp.NonHttpUrlClientErrorare base for them.The :py:exc:
~aiohttp.InvalidURLnow exposes adescriptionproperty with the text explanation of the error details.-- by :user:
setla, :user:AraHaan, and :user:bdracoRelated issues and pull requests on GitHub:
:issue:
2507, :issue:3315, :issue:6722, :issue:8481, :issue:8482.Added a feature to retry closed connections automatically for idempotent methods. -- by :user:
DreamsorcererRelated issues and pull requests on GitHub:
:issue:
7297.Implemented filter_cookies() with domain-matching and path-matching on the keys, instead of testing every single cookie.
This may break existing cookies that have been saved with
CookieJar.save(). Cookies can be migrated with this script::Related issues and pull requests on GitHub:
:issue:
7583, :issue:8535.Separated connection and socket timeout errors, from ServerTimeoutError.
Related issues and pull requests on GitHub:
:issue:
7801.Implemented happy eyeballs
Related issues and pull requests on GitHub:
:issue:
7954.Added server capability to check for static files with Brotli compression via a
.brextension -- by :user:steverep.Related issues and pull requests on GitHub:
:issue:
8062.Removals and backward incompatible breaking changes
The shutdown logic in 3.9 waited on all tasks, which caused issues with some libraries.
In 3.10 we've changed this logic to only wait on request handlers. This means that it's
important for developers to correctly handle the lifecycle of background tasks using a
library such as
aiojobs. If an application is usinghandler_cancellation=Truethenit is also a good idea to ensure that any :func:
asyncio.shieldcalls are replaced with:func:
aiojobs.aiohttp.shield.Please read the updated documentation on these points:
https://docs.aiohttp.org/en/stable/web\_advanced.html#graceful-shutdown
https://docs.aiohttp.org/en/stable/web\_advanced.html#web-handler-cancellation
-- by :user:
DreamsorcererRelated issues and pull requests on GitHub:
:issue:
8495.Improved documentation
Added documentation for
aiohttp.web.FileResponse.Related issues and pull requests on GitHub:
:issue:
3958.Improved the docs for the
sslparams.Related issues and pull requests on GitHub:
:issue:
8403.Contributor-facing changes
Enabled HTTP parser tests originally intended for 3.9.2 release -- by :user:
pajod.Related issues and pull requests on GitHub:
:issue:
8088.Miscellaneous internal changes
Improved URL handler resolution time by indexing resources in the UrlDispatcher.
For applications with a large number of handlers, this should increase performance significantly.
-- by :user:
bdracoRelated issues and pull requests on GitHub:
:issue:
7829.Added
nacl_middleware <https://github.com/CosmicDNA/nacl_middleware>_ to the list of middlewares in the third party section of the documentation.Related issues and pull requests on GitHub:
:issue:
8346.Minor improvements to static typing -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8364.Added a 3.11-specific overloads to
ClientSession-- by :user:max-muoto.Related issues and pull requests on GitHub:
:issue:
8463.Simplified path checks for
UrlDispatcher.add_static()method -- by :user:steverep.Related issues and pull requests on GitHub:
:issue:
8491.Avoided creating a future on every websocket receive -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8498.Updated identity checks for all
WSMsgTypetype compares -- by :user:bdraco.Related issues and pull requests on GitHub:
:issue:
8501.When using Python 3.12 or later, the writer is no longer scheduled on the event loop if it can finish synchronously. Avoiding event loop scheduling reduces latency and improves performance. -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8510.Restored :py:class:
~aiohttp.resolver.AsyncResolverto be the default resolver. -- by :user:bdraco.:py:class:
~aiohttp.resolver.AsyncResolverwas disabled by default becauseof IPv6 compatibility issues. These issues have been resolved and
:py:class:
~aiohttp.resolver.AsyncResolveris again now the default resolver.Related issues and pull requests on GitHub:
:issue:
8522.v3.9.5Compare Source
==================
Bug fixes
Fixed "Unclosed client session" when initialization of
:py:class:
~aiohttp.ClientSessionfails -- by :user:NewGlad.Related issues and pull requests on GitHub:
:issue:
8253.Fixed regression (from :pr:
8280) with addingContent-Dispositionto theform-datapart after appending to writer -- by :user:
Dreamsorcerer/:user:Olegt0rr.Related issues and pull requests on GitHub:
:issue:
8332.Added default
Content-Dispositioninmultipart/form-dataresponses to avoid brokenform-data responses -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8335.v3.9.4Compare Source
==================
Bug fixes
The asynchronous internals now set the underlying causes
when assigning exceptions to the future objects
-- by :user:
webknjaz.Related issues and pull requests on GitHub:
:issue:
8089.Treated values of
Accept-Encodingheader as case-insensitive when checkingfor gzip files -- by :user:
steverep.Related issues and pull requests on GitHub:
:issue:
8104.Improved the DNS resolution performance on cache hit -- by :user:
bdraco.This is achieved by avoiding an :mod:
asynciotask creation in this case.Related issues and pull requests on GitHub:
:issue:
8163.Changed the type annotations to allow
dicton :meth:aiohttp.MultipartWriter.append,:meth:
aiohttp.MultipartWriter.append_jsonand:meth:
aiohttp.MultipartWriter.append_form-- by :user:cakemannyRelated issues and pull requests on GitHub:
:issue:
7741.Ensure websocket transport is closed when client does not close it
-- by :user:
bdraco.The transport could remain open if the client did not close it. This
change ensures the transport is closed when the client does not close
it.
Related issues and pull requests on GitHub:
:issue:
8200.Leave websocket transport open if receive times out or is cancelled
-- by :user:
bdraco.This restores the behavior prior to the change in #7978.
Related issues and pull requests on GitHub:
:issue:
8251.Fixed content not being read when an upgrade request was not supported with the pure Python implementation.
-- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8252.Fixed a race condition with incoming connections during server shutdown -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8271.Fixed
multipart/form-datacompliance with :rfc:7578-- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8280.Fixed blocking I/O in the event loop while processing files in a POST request
-- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8283.Escaped filenames in static view -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8317.Fixed the pure python parser to mark a connection as closing when a
response has no length -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8320.Features
Upgraded llhttp to 9.2.1, and started rejecting obsolete line folding
in Python parser to match -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8146, :issue:8292.Deprecations (removal in next major release)
Deprecated
content_transfer_encodingparameter in :py:meth:FormData.add_field() <aiohttp.FormData.add_field>-- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8280.Improved documentation
Added a note about canceling tasks to avoid delaying server shutdown -- by :user:
Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8267.Contributor-facing changes
The pull request template is now asking the contributors to
answer a question about the long-term maintenance challenges
they envision as a result of merging their patches
-- by :user:
webknjaz.Related issues and pull requests on GitHub:
:issue:
8099.Updated CI and documentation to use NPM clean install and upgrade
node to version 18 -- by :user:
steverep.Related issues and pull requests on GitHub:
:issue:
8116.A pytest fixture
hello_txtwas introduced to aidstatic file serving tests in
:file:
test_web_sendfile_functional.py. It dynamicallyprovisions
hello.txtfile variants shared across thetests in the module.
-- by :user:
steverepRelated issues and pull requests on GitHub:
:issue:
8136.Packaging updates and notes for downstreams
Added an
internalpytest marker for tests which should be skippedby packagers (use
-m 'not internal'to disable them) -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8299.v3.9.3Compare Source
==================
Bug fixes
Fixed backwards compatibility breakage (in 3.9.2) of
sslparameter when set outsideof
ClientSession(e.g. directly inTCPConnector) -- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
8097, :issue:8098.Miscellaneous internal changes
Improved test suite handling of paths and temp files to consistently use pathlib and pytest fixtures.
Related issues and pull requests on GitHub:
:issue:
3957.v3.9.2Compare Source
==================
Bug fixes
Fixed server-side websocket connection leak.
Related issues and pull requests on GitHub:
:issue:
7978.Fixed
web.FileResponsedoing blocking I/O in the event loop.Related issues and pull requests on GitHub:
:issue:
8012.Fixed double compress when compression enabled and compressed file exists in server file responses.
Related issues and pull requests on GitHub:
:issue:
8014.Added runtime type check for
ClientSessiontimeoutparameter.Related issues and pull requests on GitHub:
:issue:
8021.Fixed an unhandled exception in the Python HTTP parser on header lines starting with a colon -- by :user:
pajod.Invalid request lines with anything but a dot between the HTTP major and minor version are now rejected.
Invalid header field names containing question mark or slash are now rejected.
Such requests are incompatible with :rfc:
9110#section-5.6.2and are not known to be of any legitimate use.Related issues and pull requests on GitHub:
:issue:
8074.Improved validation of paths for static resources requests to the server -- by :user:
bdraco.Related issues and pull requests on GitHub:
:issue:
8079.Features
Added support for passing :py:data:
Truetosslparameter inClientSessionwhiledeprecating :py:data:
None-- by :user:xiangyan99.Related issues and pull requests on GitHub:
:issue:
7698.Breaking changes
Fixed an unhandled exception in the Python HTTP parser on header lines starting with a colon -- by :user:
pajod.Invalid request lines with anything but a dot between the HTTP major and minor version are now rejected.
Invalid header field names containing question mark or slash are now rejected.
Such requests are incompatible with :rfc:
9110#section-5.6.2and are not known to be of any legitimate use.Related issues and pull requests on GitHub:
:issue:
8074.Improved documentation
Fixed examples of
fallback_charset_resolverfunction in the :doc:client_advanceddocument. -- by :user:henry0312.Related issues and pull requests on GitHub:
:issue:
7995.The Sphinx setup was updated to avoid showing the empty
changelog draft section in the tagged release documentation
builds on Read The Docs -- by :user:
webknjaz.Related issues and pull requests on GitHub:
:issue:
8067.Packaging updates and notes for downstreams
The changelog categorization was made clearer. The
contributors can now mark their fragment files more
accurately -- by :user:
webknjaz.The new category tags are:
Related issues and pull requests on GitHub:
:issue:
8066.Contributor-facing changes
Updated :ref:
contributing/Tests coverage <aiohttp-contributing>section to show how we usecodecov-- by :user:Dreamsorcerer.Related issues and pull requests on GitHub:
:issue:
7916.The changelog categorization was made clearer. The
contributors can now mark their fragment files more
accurately -- by :user:
webknjaz.The new category tags are:
Related issues and pull requests on GitHub:
:issue:
8066.Miscellaneous internal changes
Replaced all
tmpdirfixtures withtmp_pathin test suite.Related issues and pull requests on GitHub:
:issue:
3551.v3.9.1Compare Source
==================
Bugfixes
Fixed importing aiohttp under PyPy on Windows.
#​7848 <https://github.com/aio-libs/aiohttp/issues/7848>_Fixed async concurrency safety in websocket compressor.
#​7865 <https://github.com/aio-libs/aiohttp/issues/7865>_Fixed
ClientResponse.close()releasing the connection instead of closing.#​7869 <https://github.com/aio-libs/aiohttp/issues/7869>_Fixed a regression where connection may get closed during upgrade. -- by :user:
Dreamsorcerer#​7879 <https://github.com/aio-libs/aiohttp/issues/7879>_Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:
Dreamsorcerer`#7895 <https://github.com/aio-libs/a
Configuration
📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.