Proposal: Use Docker Python SKD instead of CLI #921
Description
Hello, I'd like to suggest replacing the docker CLI that's used by PyAirbyte with the Python Docker SDK and wanted to hear your opinion on that.
From what I see, DockerExecutor is the only place using it. The current implementation is rather minimal and reuses most of the process handling from the base Executor, but it requires a docker binary to be installed and available, and it pumps stdin from a subprocess call.
Instead, DockerExecutor could manage containers using the Python Docker SDK, which is a lightweight Python library and requires only the Docker socket to work.
I don't have anything concrete yet. I'm just looking around for options to reduce the size of dependencies, and the requirement for the docker binary may not be necessary. But I can handle the implementation of it.
Do you have any thoughts on this?