Update project to support python 3.14

[asnare]

Changes

What does this PR do?

This PR updates this project so that the CLI supports Python 3.14. Aside the from the project metadata, this involved:

• Updating to current versions of mypy, pylint and pytest that work properly with python 3.14. Some code-changes were required to satisfy the new versions of these tools.
• Changing the project setup so that Hatch manages the version of python in the environment rather than using the system python: this allows for easier local development and testing.
• Expands both unit and integration testing to cover all supported python versions.

Relevant implementation details

Downstream dependencies have already been released with support for Python 3.14:

• Blueprint
• Bladebridge

The pytester and lsql dependencies already work with python 3.14, although those projects also have tooling updates to ensure that CI/CD tests it explicitly. The changes in this PR broadly match those from those repositories.

A further improvement here is integration testing across the python version matrix.

Caveats/things to watch out for when reviewing:

We have our first system imports that depend on the version of python:

• importlib.abc.Traversable was moved to importlib.abs.resources.Traversable in Python 3.11, with the old location deprecated and removed in 3.14.
• An import-guard based on the python version is the only way to both ensure the import works and that mypy is aware of which import is currently active.

The changes to .codegen.json are aimed at ensuring that the labs runner (used to run integration tests) honours supports the use of HATCH_ENV to choose which Hatch environment should be used to run the tests. This is needed for the integration-testing matrix introduced by this PR.

@gueniai: Due to the expanded unit and integration tests, the set of mandatory checks on PRs for this repository will need to be adjusted.

Linked issues

Resolves #2137.

Functionality

• updated relevant user documentation

• modifies existing commands:

  • databricks labs install lakebridge
  • databricks labs lakebridge install-transpile
  • etc.

Tests

• manually tested
• existing unit tests, extended to cover python 3.14.
• existing integration tests, extended to cover all supported python versions.

[github-actions]

✅ 145/145 passed, 6 flaky, 4 skipped, 44m45s total

Flaky tests:

• 🤪 test_installs_and_runs_local_bladebridge (17.416s)
• 🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (13.25s)
• 🤪 test_transpile_teradata_sql (16.596s)
• 🤪 test_transpile_teradata_sql_non_interactive[False] (6.033s)
• 🤪 test_transpile_teradata_sql_non_interactive[True] (6.045s)
• 🤪 test_transpiles_informatica_to_sparksql (11.622s)

_{Running from acceptance #4036}

[codecov]

Codecov Report

❌ Patch coverage is 68.42105% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.42%. Comparing base (af61713) to head (73976f7).

Files with missing lines	Patch %	Lines
.../labs/lakebridge/assessments/dashboards/execute.py	0.00%	3 Missing ⚠️
...atabricks/labs/lakebridge/transpiler/installers.py	0.00%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2333   +/-   ##
=======================================
  Coverage   66.41%   66.42%           
=======================================
  Files          99       99           
  Lines        9094     9102    +8     
  Branches      974      977    +3     
=======================================
+ Hits         6040     6046    +6     
- Misses       2878     2880    +2     
  Partials      176      176           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.