-
Notifications
You must be signed in to change notification settings - Fork 5
refactor: Migrate WithAthena mixin to use AWS Wrangler for Athena I/O #89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: rnd-13624
Are you sure you want to change the base?
Conversation
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## rnd-13624 #89 +/- ##
=============================================
- Coverage 89.40% 89.31% -0.09%
=============================================
Files 16 16
Lines 1321 1310 -11
Branches 153 154 +1
=============================================
- Hits 1181 1170 -11
Misses 98 98
Partials 42 42
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
9cb7282 to
ab0030a
Compare



Summary
This PR refactors the
WithAthenamixin to use[awswrangler.athena.read_sql_query](https://aws-sdk-pandas.readthedocs.io/en/stable/stubs/awswrangler.athena.read_sql_query.html)for all Athena reads, replacing the previous PyAthena-based implementation.✅ What's changed
WithAthena_run_query→_run_wr_athena_query_wrapped@allow_optionsdirectly on the Wrangler readerAthenaDataSubSectionschema:s3_staging_dir→s3_outputregion_namesince it's optional for Wranglerpyathena.connecttowr.athena.read_sql_queryallow_options(...)usage across mixins💡 Why this matters
regionandboto3_sessionare now not mandatory fields;s3_additional_kwargsandpyarrow_additional_kwargsare handled properly via@allow_options.