Migrate to uv for python package management

[JayThomason]

Summary

Migrate from setup.py to uv for python package management.

Rationale

uv has recently become the de-facto standard for python package management due to its speed and ease of use.
This change should improve the developer experience and speed up builds, especially in fresh environments.
It should also become easier to open up some of the dependencies to a wider set of versions since uv is so much faster at resolving dependencies.

Changes

• replace setup.py with pyproject.toml w/ hatchling build backend
• consolidate requirements.txt and dev-requirements.txt into pyproject.toml
• update Makefile to use uv instead of venv/pip
• update Github Actions to use uv instead of venv/pip
• lock all current direct and indirect requirements in uv.lock for more reproducible builds
• upgraded the typing-extensions dependency version to get the dependencies to resolve
  • this should be the only dependency version that changed

Impact

This change should not affect the underlying code or dependencies (with the one exception noted above). The impact should primarily be an improved developer experience.

Testing

I have set up a local environment and verified I can run unit tests, pre-commit, etc.
I have tested CI in my own fork and it is passing.
I have not tested every command in the Makefile that changed, though I have manually run most of the same commands.

Regression Risk

Does not apply.

Checklist

• Unit tests covering the changes have been added
  • If this is a bugfix, regression tests have been added
• E2E testing has been performed

Additional Notes

This change was loosely suggested here #588 but I am curious if there is any reason not to switch to uv.

Note that this change only seems large due to the uv.lock file which has been committed; other than that it's roughly +160/-240

[pdames]

Impressive - thanks for picking this work up and getting the CI/CD workflows passing so quickly!

Before merging this, I also wanted to get some other eyes on the proposed shift of DeltaCAT 2.X to UV (DeltaCAT 1.X will remain as-is for now).

@pfaraone, @Zyiqin-Miranda, @yankevn, @IvanPartsunev - any concerns related to continuing to support your existing dev, build, and deploy workflows?

IMO, unless there's a hard blocker, I'd prefer to just merge this PR and transition to UV now vs. continuing to support the legacy build & deploy process side-by-side for future DeltaCAT 2.X work.

[JayThomason]

Thanks for the review @pdames, the documentation should be updated.

One nit I noticed is a reference in the dev README to a make test-integration target which doesn't exist. Should this be make test-integration-rebuild?

No rush at all on merging this.

[pdames]

LGTM!

[Zyiqin-Miranda]

Regarding the make test-integration part, it was originally for iceberg integration tests then we removed a few months ago. I think we can remove all reference to integration tests at all @pdames?

[pdames]

Thanks for the review @pdames, the documentation should be updated.

One nit I noticed is a reference in the dev README to a make test-integration target which doesn't exist. Should this be make test-integration-rebuild?

No rush at all on merging this.

Hmmm, good catch on the missing test-integration target. It looks like the old publish Makefile target had a missing dependency on this as well. Agree with @Zyiqin-Miranda here after looking into this a bit more - we should be able to remove both existing integration test Makefile targets for now (both test-integration-rebuild and test-converter-integration).

[JayThomason]

we should be able to remove both existing integration test Makefile targets for now (both test-integration-rebuild and test-converter-integration).

Sounds reasonable as a separate PR.

[Zyiqin-Miranda]

LGTM! Thanks for getting DeltaCAT set up with uv!

[yankevn]

LGTM

[pdames]

Merged - thanks @JayThomason!