upgrade python and added unittests#50
Conversation
There was a problem hiding this comment.
Pull request overview
This PR upgrades the project’s Python runtime/dependency baseline and introduces a new unittest-based test suite to validate discovery, sync behavior, bookmarks/start-date handling, pagination, and field-selection rules.
Changes:
- Upgrade CI virtualenv Python version to 3.12 and add CI steps to run unit + broader test discovery.
- Bump core dependency pins (backoff/requests/pendulum/singer-python).
- Add a set of unit tests under
tests/unittests/plus additional tests undertests/covering sync/discovery/state behaviors.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
tests/unittests/test_sync_selection_unit.py |
Unit tests for stream selection/custom stream detection and export-row transformation. |
tests/unittests/test_sync_helpers_unit.py |
Unit tests for retry interval logic and bookmark helpers. |
tests/unittests/test_schema_unit.py |
Unit tests for schema helpers (naming, pk selection, field typing). |
tests/unittests/test_discover_unit.py |
Unit tests for discovery metadata/keys behavior via patched schema sources. |
tests/test_sync.py |
Higher-level sync tests with mocked client + Singer output functions. |
tests/test_start_date.py |
Tests for start-date vs bookmark precedence for static + bulk sync paths. |
tests/test_pagination.py |
Tests static endpoint pagination behavior and bookmark writing. |
tests/test_discovery_metadata.py |
Tests forced replication method metadata for known streams. |
tests/test_discovery_enhanced.py |
Broader discovery tests validating catalog structure/metadata/schemas. |
tests/test_bookmark.py |
Tests bookmark writers + static sync bookmark usage. |
tests/test_automatic_fields.py |
Tests that automatic fields are exported even if unselected. |
tests/test_all_fields.py |
Tests selected-field export behavior and 250-field limit enforcement. |
tests/base.py |
Shared base test utilities/expected metadata. |
setup.py |
Updates pinned dependency versions. |
.circleci/config.yml |
Switch CI to Python 3.12, update schema validation path, add unittest execution steps. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…er-io/tap-eloqua into python_update_add_unittests
| if status == 'success': | ||
| break |
There was a problem hiding this comment.
This not required anymore since we moved this condition above. You may also need to update the elif conditions
There was a problem hiding this comment.
Ideally this tap side change doesn't fall into scope of this PR, also we don't have test account to test it thoroughly, therefore I would not mind if we remove these changes to avoid any disruption on the prod connections.
There was a problem hiding this comment.
Removed out of scope changes
| """Convert a date string or pendulum DateTime to the format expected by Eloqua | ||
| API filter expressions (``YYYY-MM-DD HH:MM:SS``). | ||
|
|
||
| Using an explicit ``.format()`` call instead of the deprecated | ||
| ``.to_datetime_string()`` guards against Pendulum 2→3 behavioural differences. | ||
| """ |
There was a problem hiding this comment.
| """Convert a date string or pendulum DateTime to the format expected by Eloqua | |
| API filter expressions (``YYYY-MM-DD HH:MM:SS``). | |
| Using an explicit ``.format()`` call instead of the deprecated | |
| ``.to_datetime_string()`` guards against Pendulum 2→3 behavioural differences. | |
| """ | |
| """Convert a date string or pendulum DateTime to the format expected by Eloqua | |
| API filter expressions (``YYYY-MM-DD HH:MM:SS``). | |
| """ |
There was a problem hiding this comment.
Also I observed that .to_datetime_string() still works
Description of change
This PR upgrades the project’s Python runtime/dependency baseline and introduces a new unittest-based test suite to validate discovery, sync behavior, bookmarks/start-date handling, pagination, and field-selection rules.
SAC-28706
Manual QA steps
Risks
Rollback steps
AI generated code
https://internal.qlik.dev/general/ways-of-working/code-reviews/#guidelines-for-ai-generated-code