Skip to content

Added support for Dremio. #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
erezsh merged 7 commits into from
Oct 23, 2025
Merged

Added support for Dremio. #41

erezsh merged 7 commits into from
Oct 23, 2025

Conversation

@scottkwalker
Copy link
Contributor

@scottkwalker scottkwalker commented Aug 28, 2025

  • Added Dremio docker image to the CI integration testing.
  • The vertica docker image is missing from dockerhub and is returning 404, so the vertica test is commented out.
  • Added python version markers to pyarrow, pandas and numpy as they each drop support for Python 3.8 so for working with Python 3.8 we need to install the last compatible version.
  • The cursor is the implementation in sqlalchemy-dremio. The current release of sqlalchemy-dremio (3.0.4) does not support datetime[ms]. The fix is on the master branch of sqlalchemy-dremio but the master branch dependency versions are not compatible with Python 3.8. To have the CI tests pass with Python 3.8, we use a fork of sqlalchemy-dremio that contains the fix but has compatible dependencies. sqlalchemy_dremio/issues/58

@scottkwalker
Added support for Dremio.
03c1d59 
* Added Dremio docker image to the CI integration testing.
* The `vertica` docker image is missing from dockerhub and is returning 404, so the vertica test is commented out.
* Added python version markers to pyarrow, pandas and numpy as they each drop support for Python 3.8 so for working with Python 3.8 we need to install the last compatible version.
* The cursor is the implementation in `sqlalchemy-dremio`. The current release of `sqlalchemy-dremio` (3.0.4) does not support datetime[ms]. The fix is on the master branch of `sqlalchemy-dremio` but the master branch dependency versions are not compatible with Python 3.8. To have the CI tests pass with Python 3.8, we use a fork of `sqlalchemy-dremio` that contains the fix but has compatible dependencies.
This was referenced Aug 28, 2025
Closed
Open
Open
erezsh
erezsh requested changes Aug 30, 2025
View reviewed changes
Copy link
Owner

@erezsh erezsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Overall looks acceptable.

@erezsh
Copy link
Owner

erezsh commented Sep 4, 2025

(Just a few changes / responses necessary)

@erezsh
Copy link
Owner

erezsh commented Sep 26, 2025

Great, thanks!

As for the UTC, maybe I missed something, but why not implement set_timezone_to_utc() as a no-op (i.e. pass)? Seems to me like it would be a bit cleaner, and semantically it would remain consistent.

@scottkwalker
Copy link
Contributor Author

scottkwalker commented Sep 29, 2025

Great, thanks!

As for the UTC, maybe I missed something, but why not implement set_timezone_to_utc() as a no-op (i.e. pass)? Seems to me like it would be a bit cleaner, and semantically it would remain consistent.

Done, changed to a pass keyword. Ready for re-review.

@erezsh erezsh mentioned this pull request Oct 23, 2025
Merged
@erezsh erezsh merged commit 2e8f5f5 into erezsh:master Oct 23, 2025
6 checks passed
@erezsh
Copy link
Owner

erezsh commented Oct 23, 2025

Thank you for your patience, and sorry it took so long!

Dremio is now officially supported.

@erezsh
Copy link
Owner

erezsh commented Oct 27, 2025

@scottkwalker I cannot make a release, since PyPI doesn't allow direct dependencies.

Until you can point to a regular dependency, or come up with some different solution, I won't be able to make a release.

As a last resort, I can remove the extra dependency, and let users install it manually if they wish. For now, I'm not in a hurry.

Publishing sqeleton (0.1.8) to PyPI
 - Uploading sqeleton-0.1.8-py3-none-any.whl FAILED

HTTP Error 400: Can't have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == "dremio". See https://packaging.python.org/specifications/core-metadata for more information. | b'<html>\n <head>\n  <title>400 Can\'t have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == "dremio". See https://packaging.python.org/specifications/core-metadata for more information.\n \n <body>\n  <h1>400 Can\'t have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == "dremio". See https://packaging.python.org/specifications/core-metadata for more information.\n  The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>\nCan&#x27;t have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == &quot;dremio&quot;. See https://packaging.python.org/specifications/core-metadata for more information.\n\n\n \n'

@scottkwalker
Copy link
Contributor Author

scottkwalker commented Oct 31, 2025

@scottkwalker I cannot make a release, since PyPI doesn't allow direct dependencies.

Until you can point to a regular dependency, or come up with some different solution, I won't be able to make a release.

As a last resort, I can remove the extra dependency, and let users install it manually if they wish. For now, I'm not in a hurry.

Publishing sqeleton (0.1.8) to PyPI
 - Uploading sqeleton-0.1.8-py3-none-any.whl FAILED

HTTP Error 400: Can't have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == "dremio". See https://packaging.python.org/specifications/core-metadata for more information. | b'<html>\n <head>\n  <title>400 Can\'t have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == "dremio". See https://packaging.python.org/specifications/core-metadata for more information.\n \n <body>\n  <h1>400 Can\'t have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == "dremio". See https://packaging.python.org/specifications/core-metadata for more information.\n  The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>\nCan&#x27;t have direct dependency: sqlalchemy-dremio@ git+https://github.com/scottkwalker/sqlalchemy_dremio@308e686b166787941180850ce5813c313d3a9985 ; extra == &quot;dremio&quot;. See https://packaging.python.org/specifications/core-metadata for more information.\n\n\n \n'

Thanks @erezsh , I'm still waiting on a merge and release of sqlalchemy_dremio !57 but if this does not happen then yes we will need to remove the extra dependency.

@erezsh
Copy link
Owner

erezsh commented Oct 31, 2025

@scottkwalker Great, keep me posted.

@erezsh
Copy link
Owner

erezsh commented Oct 31, 2025

P.S. would removing support for 3.8 in sqeleton also solve this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erezsh erezsh erezsh requested changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@scottkwalker @erezsh