-
Notifications
You must be signed in to change notification settings - Fork 107
Closed
Description
Summary
Align PandasCursor's chunk processing implementation and documentation with the recently merged PolarsCursor implementation (#637).
Background
PolarsCursor was updated with improved chunk processing that:
- Enables lazy loading for ALL data access methods (fetchone, fetchmany, fetchall) when
chunksizeis set - Provides
iter_chunks()method on both Cursor and ResultSet - Has clear documentation explaining the lazy loading behavior
PandasCursor has the same underlying behavior but lacks:
- Consistent API (
iter_chunks()on ResultSet) - Clear documentation about lazy loading for fetch methods
as_pandas()method on DataFrameIterator
Tasks
Code Changes
- Add
iter_chunks()method toAthenaPandasResultSet - Add
as_pandas()method toDataFrameIterator(align with Polars) - Update
PandasCursor.iter_chunks()to delegate to ResultSet (like PolarsCursor)
Documentation Changes
- Update
docs/pandas.rstto clarify thatchunksizeenables lazy loading for all fetch methods - Add
chunksizeanditer_chunks()documentation to AsyncPandasCursor section
Import Fix
- Change
import abctofrom collections import abc(same fix as PolarsCursor)
Related
- PR Add chunk processing support for PolarsCursor #637: Add chunk processing support for PolarsCursor
🤖 Generated with Claude Code
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels