Skip to content

Conversation

@Christos-Hadjinikolis
Copy link
Collaborator

@Christos-Hadjinikolis Christos-Hadjinikolis commented Aug 1, 2025

Overview

  • Extended Postgres Type Support: The _type_lookup now explicitly supports all relevant pandas and PostgreSQL types, including Int64, boolean, string, and JSONB, improving robustness and compatibility.

  • Fixes COPY failures: Prevents silent data truncation or mismatched writes by ensuring pandas float64 is handled when writing to Int64 columns using Postgres COPY.

  • Improved Error Transparency: _generate_model_from_schema now skips columns only if they lack a compatible mapping — consider extending to raise if critical fields are dropped.

  • Improved pandas 2.x Compatibility: Replaced session.get_bind() with compiled SQL using sqlalchemy.text(...) for pandas ≥ 2.0.

  • Cleaner Logging: Log messages for table reads now include database host and name for easier traceability.

  • Tests Updated:

    • Migrated test assertions to use ReadOnlyColumnCollection (SQLAlchemy 2.x).
    • Added fallback mocks and a new GenericIO test schema to verify full mixin functionality.
  • Pre-commit Fixes:

    • Corrected invalid regex in the exclude fields (e.g., demo/*demo/.*).
    • Normalized hook stages from commit to pre-commit.
    • Ran pre-commit migrate-config and updated Makefile accordingly.

Checklist

The following won't apply in all cases - mark N/A next to point if not needed.

  • Appropriate unit tests added/updated
  • Module, function, class docstrings updated
  • Comments added to PR where necessary
  • Interface documentation updated
  • Semantic commits used for this PR, so that a CHANGELOG can be generated from them (don't delete your local branch! when tagging a new release from master)
  • Test RC candidate

Release-Steps

  • Merge PR
  • Release new tag

@codecov
Copy link

codecov bot commented Aug 1, 2025

Codecov Report

❌ Patch coverage is 66.66667% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.19%. Comparing base (e1f5c5f) to head (4bc566f).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
dynamicio/mixins/with_postgres.py 62.96% 9 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #90      +/-   ##
==========================================
- Coverage   88.21%   88.19%   -0.03%     
==========================================
  Files          16       16              
  Lines        1256     1262       +6     
  Branches      142      143       +1     
==========================================
+ Hits         1108     1113       +5     
- Misses        110      111       +1     
  Partials       38       38              
Flag Coverage Δ
unittests 88.19% <66.66%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Christos-Hadjinikolis Christos-Hadjinikolis self-assigned this Aug 1, 2025
@Christos-Hadjinikolis Christos-Hadjinikolis added enhancement New feature or request echoes/effort: M Medium effort changes by the team's standards (default) labels Aug 1, 2025
@Christos-Hadjinikolis Christos-Hadjinikolis changed the title fix(postgres): ensure compatibility with pandas>=2.0 by using SQLAlchemy Connection in read_sql fix: ensure compatibility with pandas>=2.0 by using SQLAlchemy Connection in read_sql Aug 1, 2025
@Christos-Hadjinikolis Christos-Hadjinikolis merged commit a705108 into master Aug 5, 2025
13 of 15 checks passed
@Christos-Hadjinikolis Christos-Hadjinikolis deleted the sql-engine branch August 5, 2025 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

echoes/effort: M Medium effort changes by the team's standards (default) enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants