Reduce size of docker images #1
Description
Deployment/update times coud be reduced by reducing the size of the docker images.
The Fed-BioMed services are run as docker containers using images are built by CDK at deploy time. This has various advantages including straightfoward ECS deployment, reproducibility, and security (no internet access is required by the running containers).
However some of the images are large, due to the large dependencies required by Fed-Biomed (such as torch, itk etc):
- local node
- local node gui
- researcher image (a shared image used by researcher containers e.g. jupyter and tensorboard)
The large image size leads to slow build and deployment. It can also cause memory/disk issues on the machine performing the deployment due to the limited resources allocate to the local docker engine
Possible approaches:
- Removing unnecessary dependencies (for example the gui may not require so many dependencies);
- Removing conda, since multiple environments on the same machine are not required for this setup;
- sharing layers between containers. One issue with this is that the base images for the node and other images are different because of the node's GPU support;
- Building a local private repository of shared libraries and locally installing packages at runtime