Free disk space threshold

[tenekev]

Is your feature request related to a problem? Please describe.

I use Watchtower in watch + download mode. I let watchtower look for new updates and download them. I perform the updates manually to ensure everything works after them.

If left unattended for prolonged periods of time, Watchtower downloads new images and eventually fills up all of the available space.

Describe the solution you'd like

Implement an option a free space threshold. Once reached, e.g. 90% full, Watchtower preforms a cleanup. Or just stops downloading new updates. Or prunes the oldest ones.

Describe alternatives you've considered

Notifications still require manual fixing. I could write a script that prunes on low disk space but it wouldn't be a clean approach like Watchtower.

Additional context

No response

[beatkind]

Hi @tenekev, I think this would be a super improvement to watchtower. Scoping it to only delete images once a low / high watermark on disk is reached seems counterintuitive to me. I would rather expect a different default behavior:

One a new tag is released, watchtower picks it up, downloads it, checks if there is a tag between the currently running one and the downloaded one, if yes, deletes the tag in the middle.

In addition, it could make sense to keep the tags in-between.

What do you think?

[tenekev]

I suggested it without being very familiar with how Watchtower works. I'm still using the original Watchtower image and there might be differences introduced by you since its last version but I feel the issue is still there. I just copied the issue over because there hasn't been an answer in 2 years.

The way you explain it, makes me think that Watchtower already manages what's stored. To be clear, I haven't tested your version to see how the storage usage grows but on the original image, a lot of extra data is stored from updates.

My suggestion is more of a failsafe feature than a feature for unattended work. Let's say your storage is 85% full. 65% of which is docker images of lots of working containers. LLM-related images are notoriously big. Watchtower will begin downloading updates, disregarding available storage. Once the storage is full, any container with a volume will cease working due to read-only filesystem error. It's a rick that I would like to avoid.

From this perspective, it makes sense to me, to stop downloading above a certain threshold. And maybe send out a notification to the connected notification services with "disk full" message.

[beatkind]

Understood, I will think of it and come up with some kind of proposal :)