Slow authentication with arrow cursor results in NETWORK_CONNECTION during HeadObject operation

[pnorman]

When running a query with arrow there is a default timeout of 3 seconds for s3 to respond. This is normally enough but when a role is being assumed with STS sometimes it takes longer and the error in #520 presents itself.

pyathena.error.OperationalError: When reading information for key 'tilelogs/d75a5f52-a89c-4095-a329-1d3be1d1b2d2.csv' in bucket 'openstreetmap-athena-results': AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 28, Timeout was reached

The query itself doesn't matter, but in this case it is returning 0-1 rows. The issue is from Arrow (and hence ArrowCursor) having a default 3s timeout on the HeadObject operation and the operation taking longer than that to authenticate.

This can be seen with the AWS CLI, with times of up to 7s to head the object

time AWS_REGION=eu-north-1 AWS_PROFILE=osm-service-logs aws s3api head-object --bucket openstreetmap-athena-results --key tilelogs/d75a5f52-a89c-4095-a329-1d3be1d1b2d2.csv
{
    "AcceptRanges": "bytes",
    "Expiration": "expiry-date=\"Thu, 30 Oct 2025 00:00:00 GMT\", rule-id=\"Lifecycle\"",
    "LastModified": "Wed, 15 Oct 2025 22:43:38 GMT",
    "ContentLength": 778,
    "ETag": "\"eb5d9149215475d4fbef7892bbbcdc32\"",
    "VersionId": "wAnbK.mUfCK94G3Q0Kt37kQ.IJcX4bKB",
    "ContentType": "application/octet-stream",
    "ServerSideEncryption": "AES256",
    "Metadata": {}
}

real    0m7.419s
user    0m0.551s
sys     0m0.140s

In this case it is because the accounts are set up to auth into one account and assume role into another, and that is adding significant delays.

~/.aws/credentials contains

[osm-main]
aws_access_key_id = A...
aws_secret_access_key = ...
[osm-service-logs]
role_arn = arn:aws:iam::...:role/OrganizationAccountAccessRole
source_profile = osm-main

This is repeatable for me, but if it repeats for someone else will depend on their location, their region, latency, and luck. My dev server is on the US west coast connecting to eu-north-1, so that may be related.

Arrow added the ability to adjust the timeouts to s3fs in apache/arrow#13385 but this isn't exposed in pyathena. The relevant variables are request_timeout and connect_timeout

Repeatable version of #520

My workaround was to switch to Pandas

[laughingman7743]

Hi @pnorman,

Thank you for reporting this issue! I've created PR #610 to address the timeout problems you're experiencing with role assumption.

Solution

The PR adds connect_timeout and request_timeout parameters to ArrowCursor and AsyncArrowCursor, allowing you to configure the S3 timeout values to accommodate slower authentication scenarios.

Usage

You can now configure the timeouts like this:

from pyathena import connect
from pyathena.arrow.cursor import ArrowCursor

cursor = connect(
    s3_staging_dir="s3://YOUR_S3_BUCKET/path/to/",
    region_name="eu-north-1"
).cursor(
    ArrowCursor,
    connect_timeout=10.0,  # Socket connection timeout in seconds
    request_timeout=30.0   # Request timeout in seconds (instead of default ~3s)
)

cursor.execute("SELECT * FROM my_table")
table = cursor.as_arrow()

This should resolve the timeout errors you're seeing when authentication with STS takes longer than 3 seconds.

Note on Version Requirements

This feature requires PyArrow >= 10.0.0, as the timeout configuration support was added in Apache Arrow 10.0.0 (ARROW-16521). The PR updates the minimum PyArrow version from 7.0.0 to 10.0.0.

Please let me know if this resolves your issue or if you need any adjustments!