Replace docker-compose with official Docker SDK#214
Replace docker-compose with official Docker SDK#214Abhishek Krishna (abh1sh5k) merged 13 commits intomasterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Migrates away from the deprecated docker-compose Python library by introducing a lightweight “compose-like” layer backed by the official Docker SDK, while updating CI/runtime dependencies to support the new implementation.
Changes:
- Replace
docker-composeusage with a newComposeProject/ComposeService/ComposeContainerimplementation usingdockerSDK. - Update dependencies (remove
docker-compose, addPyYAML/boto3) and loosen version pins. - Update local/CI Python tooling configuration to target Python 3.14.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tox.ini | Switch tox env from generic py3 to explicit py314. |
| requirements.txt | Drop docker-compose; add PyYAML/boto3; loosen dependency pins. |
| confluent/docker_utils/compose.py | New compose replacement module built on Docker SDK. |
| confluent/docker_utils/init.py | Refactor utilities to use Docker SDK + new compose module; update ECR login/build/run helpers. |
| .semaphore/semaphore.yml | Update CI Python version selection to 3.14. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
airlock-confluentinc
bot
force-pushed
the
replace-docker-compose-with-sdk
branch
2 times, most recently
from
f4c907e to
dafd607
Compare
airlock-confluentinc
bot
force-pushed
the
replace-docker-compose-with-sdk
branch
from
dafd607 to
27ed162
Compare
requirements.txt
Outdated
| docker-compose==1.29.2 | ||
| Jinja2==3.1.6 | ||
| requests==2.32.5 | ||
| boto3>=1.36.0 |
There was a problem hiding this comment.
I believe adding this back here will bring in the urllib3 CVE ref: https://confluent.slack.com/archives/C055SK92MD1/p1768830959037109?thread_ts=1768826828.012589&cid=C055SK92MD1 . Can you please confirm the same once?
There was a problem hiding this comment.
urllib is latest version in the common-docker image. Since we updated the python version, this will fetch latest version of boto3 hence picking the latest version of dependencies.
Tested with dev image of common-docker
Krish Vora (KrishVora01)
left a comment
Krish Vora (KrishVora01)
left a comment
There was a problem hiding this comment.
Our tox report has a few warnings: https://semaphore.ci.confluent.io/jobs/5629abf4-2ec9-446e-9ebc-e016d89698d3#L379-L406 . Could you please take a look at the same, and make any changes if needed?
The warnings fall into two categories: |
confluent/docker_utils/__init__.py
Outdated
|
|
||
| def api_client(): | ||
| return docker.from_env().api | ||
| def api_client() -> docker.DockerClient: |
There was a problem hiding this comment.
return type is changed here. Will this need changes in downstream? seems like a breaking change.
Also will it be better to rename the method name be docker_client()?
There was a problem hiding this comment.
I've searched for the api-client org wide its not used directly in any of the confluentinc repos. I'll rename this method. It wont break any downstream component.
…ease versions, add specific type hints
Abhishek Krishna (abh1sh5k)
deleted the
replace-docker-compose-with-sdk
branch
3 participants
Summary
Migrates from the deprecated
docker-composePython library to the official Docker SDK for Python.Changes
docker-compose==1.29.2dependency (deprecated, last release 2021)PyYAMLfor compose file parsing,boto3for ECR supportcompose.pymodule providing Docker SDK-based compose functionalitytox.inifor Python 3.14Testing
All existing tests pass with the new implementation.
Integration tests are passing on ksql-images here