(Enhancement) Add built-in retention for backups

[StanMar-bit]

Hi @Minituff,

First off, thank you for your fantastic work on Nautical—it’s an awesome tool and I use it for backing up my entire stack!

I have a feature suggestion that I believe could benefit many users: an option to automatically delete older backups created by Nautical. This cleanup mechanism would help manage disk space and prevent the accumulation of outdated backups. I propose that this feature be configurable via an environment variable, allowing users to set their preferred retention policy.

Although I'm still gaining experience with Python, I'd be happy to contribute to this feature's development and learn along the way. Please let me know if this aligns with your vision for Nautical or if you have any thoughts on the approach.

Thanks again for all your hard work on this awesome project!

[Minituff]

Hey, yeah I've thought about this a little before.

There are a few things to consider:

• Nautical allows you to customize your backup folder format, so this may make it difficult to find old folders due to this .

• Also what happens if you change the folder format when you have backups already? Nautical might night be able to see them?

• Maybe we can use some type of database to store the folder names and the dates?

[StanMar-bit]

Thanks for your prompt response! Oh that's cool to hear, that you were thinking about it as well :)

• Oh yeah that could be an issue.. Maybe we could somehow extract the current backup folder format from the environment variable? Or where you thinking of a situation when the backup folder format is changed in between backups?

• This is also tricky, but maybe also then an issue for the admin, isn't it kind of expected when changing directory configuration, that the auto built-in retention might not work anymore? To maybe prevent the implementation of the feature getting too complicated? But it would be cool if it's fully foolproofed.

• That's a good idea. Maybe some kind of history file with the dates and paths of the backups, kind of like a db but seems less complicated to me haha. But maybe a db is more practical and standardized.

[Minituff]

Hey @StanMar-bit, I saw you created a fork and are working on the retention.

How is that going? I'd love to get it incorporated it's working well. Do you need any help?

[StanMar-bit]

Hey @StanMar-bit, I saw you created a fork and are working on the retention.

How is that going? I'd love to get it incorporated it's working well. Do you need any help?

Hey @Minituff , yeah, I gave it a shot! I tried adding a new script, but I ran into dependency errors when running the forked dev Docker image, which prevented me from properly testing the solution. I also tinkered with the image’s dependencies a bit, but those commits can probably be ignored, haha.

I don’t have much experience working with custom Docker images, so that might have contributed to the issue. Plus, most of the code was written with ChatGPT’s help, so I’m not sure about the quality.

I ended up putting my efforts on developing the feature on hold until I have time to figure out why I was getting these errors.

But I’d definitely appreciate any help with developing the retention feature!

By the way, I really love the app—it kind of saved my ass tonight, haha. I was tinkering with the configuration of my homelab’s SSO and messed things up, but luckily, the backups helped me restore my config

[StanMar-bit]

Hey @StanMar-bit, I saw you created a fork and are working on the retention.

How is that going? I'd love to get it incorporated it's working well. Do you need any help?

Hey @Minituff , yeah, I gave it a shot! I tried adding a new script, but I ran into dependency errors when running the forked dev Docker image, which prevented me from properly testing the solution. I also tinkered with the image’s dependencies a bit, but those commits can probably be ignored, haha.

I don’t have much experience working with custom Docker images, so that might have contributed to the issue. Plus, most of the code was written with ChatGPT’s help, so I’m not sure about the quality.

I ended up putting my efforts on developing the feature on hold until I have time to figure out why I was getting these errors.

But I’d definitely appreciate any help with developing the retention feature!

By the way, I really love the app—it kind of saved my ass tonight, haha. I was tinkering with the configuration of my homelab’s SSO and messed things up, but luckily, the backups helped me restore my config

Oh, I just realized I never actually pushed the commits where I tinkered with the dependencies to the fork. So those changes shouldn’t be an issue after all.

[globalnamespace]

Just started using nautical-backup and it's awesome so far! A retention policy is the only thing that really feels missing, adding that would make nautical-backup pretty much perfect for my needs.

Might this still be in the works? Seems like this thread has stalled out. I can certainly write a post exec script to handle this for my specific setup but a first-class feature would be fantastic.

[seluce]

I found now a simpe way to keep X Days on Backups. Here is my YAML Project.

services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    restart: unless-stopped

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /volume2/docker:/app/source:ro
      - /mnt/@usb/sdc1/docker_backups:/app/destination

    environment:
      TZ: Europe/Berlin
      BACKUP_ON_START: false
      CRON_SCHEDULE: 0 2 * * *
      SKIP_CONTAINERS: nautical-backup-cleanup
      ADDITIONAL_FOLDERS: nautical-backup
      ADDITIONAL_FOLDERS_USE_DEST_DATE_FOLDER: true
      USE_DEST_DATE_FOLDER: true
      REPORT_FILE_LOG_LEVEL: INFO
                  
  docker-backup-cleanup:
      image: alpine:latest
      container_name: nautical-backup-cleanup
      restart: unless-stopped
      volumes:
        - /mnt/@usb/sdc1/docker_backups:/backups
      command: sh -c 'while true; do find /backups -maxdepth 1 -not -path "/backups" -mtime +6 -exec rm -rf {} \;; sleep 86400; done'

The Key is "docker-backup-cleanup", which check the same destination for older backups than X days and delete them. pretty simple, very clean and lightweight. works good :)

Delete older Backups than 6 Days: -mtime +6
Run Clean Up more than once per day: modify 86400 to another value.

[seluce]

I’ve been working on a cleanup method for Nautical Backup using some AI logic to refine the process. I now have a solid YAML configuration (see attached) that handles the rotation of old backups.

The core idea is:

When USE_DEST_DATE_FOLDER: true is active, the cleanup script identifies and deletes expired backup files/folders based on a defined retention policy.

I wanted to share this with you to see if there is a "native" way to integrate this logic directly into Nautical Backup, or if you've considered adding a similar lifecycle management feature.

No clue, if you can use it but it works good for me so far :)

services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    restart: unless-stopped

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /volume2/docker/nautical-backup/config:/config
      - /volume2/docker:/app/source:ro
      - /mnt/@usb/sdc1/docker_backups:/app/destination

    environment:
      TZ: Europe/Berlin
      BACKUP_ON_START: false
      CRON_SCHEDULE: 0 2 * * *
      SKIP_CONTAINERS: nautical-backup-cleanup
      ADDITIONAL_FOLDERS: nautical-backup
      ADDITIONAL_FOLDERS_USE_DEST_DATE_FOLDER: true
      USE_DEST_DATE_FOLDER: true
      REPORT_FILE_LOG_LEVEL: INFO

  docker-backup-cleanup:
    image: alpine:latest
    container_name: nautical-backup-cleanup
    restart: unless-stopped
    environment:
      - TZ=Europe/Berlin        # Set the timezone for accurate daily scheduling and log timestamps
      - CLEANUP_ENABLED=true    # true: enable actual deletion | false: simulation mode (logs only)
      - KEEP_DAYS=6             # Retention period: Backups older than this number of days will be removed
      - MIN_BACKUPS=6           # Safety Floor: Never delete if it would leave fewer than X backups total
      - EXECUTION_TIME=21:43    # The specific time (24h format) when the daily cleanup cycle starts
    volumes:
      - /mnt/@usb/sdc1/docker_backups:/backups
    command: |
      sh -c '
      # 1. Ensure timezone data is available
      [ ! -f /usr/share/zoneinfo/$$TZ ] && apk add --no-cache tzdata > /dev/null

      echo "---------------------------------------------------------------"
      echo "[INIT] BACKUP CLEANUP & INTEGRITY SERVICE STARTED"
      echo "[INIT] Schedule: Daily at $$EXECUTION_TIME | Keep: $$KEEP_DAYS days"
      echo "---------------------------------------------------------------"

      while true; do
        # 2. Time Validation
        TARGET_TIME=$${EXECUTION_TIME:-03:00}
        if ! echo "$$TARGET_TIME" | grep -Eq "^([01][0-9]|2[0-3]):[0-5][0-9]$$"; then
            TARGET_TIME="03:00"
        fi

        # 3. Smart Sleep
        CURRENT_SEC=$$(date +%s)
        NEXT_SEC=$$(date -d "$$(date +%Y-%m-%d) $$TARGET_TIME" +%s)
        [ "$$NEXT_SEC" -le "$$CURRENT_SEC" ] && NEXT_SEC=$$(($$NEXT_SEC + 86400))
        
        SLEEP_SEC=$$(($$NEXT_SEC - $$CURRENT_SEC))
        echo "[IDLE] Sleeping $$(($$SLEEP_SEC / 3600))h $$(($$SLEEP_SEC % 3600 / 60))m until $$TARGET_TIME"
        sleep $$SLEEP_SEC

        # 4. Cleanup & Integrity Scan
        if [ -d "/backups" ]; then
          FOLDERS=$$(ls -1d /backups/20??-??-?? 2>/dev/null | sort)
          TOTAL=$$(echo "$$FOLDERS" | grep -c "20")
          CUTOFF=$$(date -d "@$$(($$(date +%s) - $$KEEP_DAYS * 86400))" +%Y-%m-%d)
          DELETED=0
          REMAINING=$$TOTAL

          echo "[SCAN] Starting daily routine. Folders found: $$TOTAL"

          for d in $$FOLDERS; do
            NAME=$${d##*/}
            
            # Integrity Check: Alert if a folder we intend to keep is empty
            if [ -z "$$(ls -A "$$d")" ]; then
                echo "[WARN] Backup folder $$NAME is EMPTY! Check Nautical Backup logs."
            fi

            # Cleanup Logic
            if [ "$$NAME" \< "$$CUTOFF" ]; then
              if [ $$REMAINING -gt $$MIN_BACKUPS ]; then
                if [ "$$CLEANUP_ENABLED" = "true" ]; then
                  echo "[DEL] Removing old backup: $$NAME"
                  rm -rf "$$d"
                  DELETED=$$(($$DELETED + 1))
                else
                  echo "[SIM] Would delete: $$NAME"
                fi
                REMAINING=$$(($$REMAINING - 1))
              else
                echo "[SAFE] Floor reached ($$MIN_BACKUPS). Keeping: $$NAME"
              fi
            fi
          done

          # 5. Storage Report
          FREE_INFO=$$(df -h /backups | awk "NR==2 {print \$$4 \" free (\"\$$5 \" used)\"}")
          echo "[DONE] Deleted: $$DELETED | Active: $$REMAINING | Storage: $$FREE_INFO"
        else
          echo "[ERROR] /backups not found! Check mount point."
        fi
      done'

[globalnamespace]

I ended up doing something similar to your simple solution and just added it to my host system's crontab:

#!/bin/bash
backup_dir='/mnt/data/backups'
find $backup_dir/* -type d -ctime +25 -prune -exec rm -rf {} +

I'm using 25 days to prune backups older than 4 weeks
Working well so far
Not everything has to be in a docker container I suppose :P

I thought about doing this with nautical's command execution feature (PRE_BACKUP_EXEC env variable) but it appears the version of find within nautical's container doesn't see the folder's ctime properly for some reason and treats it the same as mtime, so this script wouldn't work.

[seluce]

Oh god, you are smart. Actually I can work with post backup script variable in nautical backup and don't need the alpine image with the command.

post_backup_script=/config/backup_cleanup.sh

The script has now as target /app/destination to find older backups after a successful cron run.

[rikardronnkvist]

Great stuff @seluce 👍

I added a few lines so I can run the script once for all my servers/containers

I have this kind of structure

/volume1/backup/server1/yyyy-mm-dd/ContainerX
/volume1/backup/server2/yyyy-mm-dd/ContainerY

And here is the updated script

  nautical-backup-cleanup:
    image: alpine:latest
    container_name: nautical-backup-cleanup
    restart: unless-stopped
    environment:
      - TZ=Europe/Stockholm                # Set the timezone for accurate daily scheduling and log timestamps
      - CLEANUP_ENABLED=true               # true: enable actual deletion | false: simulation mode (logs only)
      - KEEP_DAYS=4                        # Retention period: Backups older than this number of days will be removed
      - MIN_BACKUPS=4                      # Safety Floor: Never delete if it would leave fewer than X backups total
      - EXECUTION_TIME=05:00               # The specific time (24h format) when the daily cleanup cycle starts
      - CLEANUP_ON_START=true              # true: run cleanup immediately on container start | false: wait for scheduled time
      - SKIP_SERVERS=homeassistant,pve01   # Comma-separated list of server folder names to skip during cleanup (no spaces)
    volumes:
      - /volume1/backup:/backup
    command: |
      sh -c '
      # 1. Ensure timezone data is available
      [ ! -f /usr/share/zoneinfo/$$TZ ] && apk add --no-cache tzdata > /dev/null

      echo "[INIT] BACKUP CLEANUP & INTEGRITY SERVICE STARTED"
      echo "[INIT] Schedule: Daily at $$EXECUTION_TIME | Keep: $$KEEP_DAYS days"
      echo ""

      # Function to run cleanup
      run_cleanup() {
          if [ -d "/backup" ]; then
              # Calculate cutoff: KEEP_DAYS includes today, so subtract (KEEP_DAYS - 1) days
              CUTOFF=$$(date -d "@$$(($$(date +%s) - ($$KEEP_DAYS - 1) * 86400))" +%Y-%m-%d)
              TOTAL_DELETED=0
              TOTAL_REMAINING=0
              TOTAL_FOLDERS=0

              echo "[SCAN] Starting daily routine across all servers..."

              # Iterate through each server directory
              for SERVER_DIR in /backup/*/; do
              if [ ! -d "$$SERVER_DIR" ]; then
                  continue
              fi
              
              SERVER_NAME=$$(basename "$$SERVER_DIR")
              
              # Skip special directories
              if [ "$$SERVER_NAME" = "@eaDir" ] || [ "$$SERVER_NAME" = ".DS_Store" ]; then
                  # echo "[DEBUG] Skipping special folder: $$SERVER_DIR"
                  continue
              fi
              
              # Skip servers in SKIP_SERVERS list
              if [ -n "$${SKIP_SERVERS}" ]; then
                  if echo ",$${SKIP_SERVERS}," | grep -q ",$$SERVER_NAME,"; then
                      # echo "[DEBUG] Skipping server (in SKIP_SERVERS): $$SERVER_NAME"
                      continue
                  fi
              fi
                  
              FOLDERS=$$(ls -1d "$$SERVER_DIR"20??-??-?? 2>/dev/null | sort)
              
              # Count folders properly
              if [ -z "$$FOLDERS" ]; then
                  TOTAL=0
              else
                  TOTAL=$$(echo "$$FOLDERS" | wc -l)
              fi
              
              if [ $$TOTAL -eq 0 ]; then
                  echo "[INFO] $$SERVER_DIR: No backup folders found"
                  continue
              fi
              
              DELETED=0
              REMAINING=$$TOTAL
              TOTAL_FOLDERS=$$((TOTAL_FOLDERS + TOTAL))

              echo "[SCAN] $$SERVER_DIR: $$TOTAL backup folders found"

              for d in $$FOLDERS; do
                  NAME=$${d##*/}
                  
                  # Integrity Check: Alert if a folder we intend to keep is empty
                  if [ -z "$$(ls -A "$$d")" ]; then
                  echo "[WARN] $$d is EMPTY! Check Nautical Backup logs."
                  fi

                  # Cleanup Logic
                  if [ "$$NAME" \< "$$CUTOFF" ]; then
                      if [ $$REMAINING -gt $$MIN_BACKUPS ]; then
                          if [ "$$CLEANUP_ENABLED" = "true" ]; then
                              echo "[DEL] Removing old backup: $$d"
                              rm -rf "$$d"
                              DELETED=$$((DELETED + 1))
                              TOTAL_DELETED=$$((TOTAL_DELETED + 1))
                          else
                              echo "[SIM] Would delete: $$d"
                          fi
                          REMAINING=$$((REMAINING - 1))
                      else
                          echo "[SAFE] Floor reached ($$MIN_BACKUPS). Keeping: $$d"
                      fi
                  else
                      echo "[KEEP] Within retention period. Keeping: $$d"
                  fi
              done
              
              TOTAL_REMAINING=$$((TOTAL_REMAINING + REMAINING))
              echo "[INFO] $$SERVER_DIR: Deleted: $$DELETED | Active: $$REMAINING"
              done

              # Storage Report
              echo "[DONE] Total - Deleted: $$TOTAL_DELETED | Active: $$TOTAL_REMAINING/$$TOTAL_FOLDERS"
          else
              echo "[ERROR] /backup not found! Check mount point."
          fi
          echo ""
      }

      # Run cleanup immediately if requested
      if [ "$${CLEANUP_ON_START:-false}" = "true" ]; then
          echo "[START] Running immediate cleanup as requested..."
          run_cleanup
      fi

      while true; do
          # 2. Time Validation
          TARGET_TIME=$${EXECUTION_TIME:-03:00}
          if ! echo "$$TARGET_TIME" | grep -Eq "^([01][0-9]|2[0-3]):[0-5][0-9]$$"; then
              TARGET_TIME="03:00"
          fi

          # 3. Smart Sleep
          CURRENT_SEC=$$(date +%s)
          NEXT_SEC=$$(date -d "$$(date +%Y-%m-%d) $$TARGET_TIME" +%s)
          [ "$$NEXT_SEC" -le "$$CURRENT_SEC" ] && NEXT_SEC=$$(($$NEXT_SEC + 86400))

          SLEEP_SEC=$$(($$NEXT_SEC - $$CURRENT_SEC))
          echo "[IDLE] Sleeping $$(($$SLEEP_SEC / 3600))h $$(($$SLEEP_SEC % 3600 / 60))m until $$TARGET_TIME"
          sleep $$SLEEP_SEC

          # Run scheduled cleanup
          run_cleanup
      done'

[seluce]

Have now a lot smaller, efficient script. It's now my dail y cleanup script :D

Here's my YAML for Nautical Backup:
It's important to set at least these 2environments:
POST_BACKUP_EXEC: /config/backup_cleanup.sh
USE_DEST_DATE_FOLDER: true

services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    restart: unless-stopped

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /volume2/docker/nautical-backup/config:/config
      - /volume2/docker:/app/source:ro
      - /mnt/@usb/sdc1/docker_backups:/app/destination

    environment:
      TZ: Europe/Berlin
      BACKUP_ON_START: false
      CRON_SCHEDULE: 0 2 * * *
      ADDITIONAL_FOLDERS: nautical-backup
      ADDITIONAL_FOLDERS_USE_DEST_DATE_FOLDER: true
      USE_DEST_DATE_FOLDER: true
      REPORT_FILE_LOG_LEVEL: INFO
      POST_BACKUP_EXEC: /config/backup_cleanup.sh

save backup_cleanup.sh in the nautical backup docker /config/ (in my case in /volume2/docker/nautical-backup/config/) and use following bash script:

#!/bin/bash

# --- User Configuration ---
KEEP_DAYS=6
MIN_BACKUPS=6
CLEANUP_ENABLED=true
DEST="/app/destination"

# --- Operational Variables ---
TODAY=$(date +%Y-%m-%d)

# Ultra-compact helper using Python for high-res timestamps
# This guarantees 6-digit microseconds (matching Nautical Backup exactly)
log_msg() {
    local ts=$(python3 -c 'import datetime; print(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f"))')
    echo "$ts - $1"
}

# 1. Execute cleanup logic and capture output
LOG_DATA=$(
    log_msg "INFO: Start Cleanup Script (delete $KEEP_DAYS Days but keep at least $MIN_BACKUPS Backups)"

    if [ -d "$DEST" ]; then
        # List and sort folders chronologically
        FOLDERS=$(ls -1d $DEST/20??-??-?? 2>/dev/null | sort)
        TOTAL=$(echo "$FOLDERS" | grep -c "20")
        CUTOFF=$(date -d "@$(($(date +%s) - $KEEP_DAYS * 86400))" +%Y-%m-%d)
        
        DELETED=0
        REMAINING=$TOTAL

        for d in $FOLDERS; do
            NAME=${d##*/}
            if [[ "$NAME" < "$CUTOFF" ]] && [ $REMAINING -gt $MIN_BACKUPS ]; then
                if [ "$CLEANUP_ENABLED" = "true" ]; then
                    log_msg "INFO: Delete $NAME"
                    rm -rf "$d"
                    DELETED=$((DELETED + 1))
                fi
                REMAINING=$((REMAINING - 1))
            fi
        done
        log_msg "INFO: Deleted: $DELETED Backup(s) | $REMAINING Backups left"
    else
        log_msg "ERROR: Path $DEST not found!"
    fi
)

# 2. Append to today's report file
REPORT_FILE=$(ls "$DEST/Backup Report - $TODAY"* 2>/dev/null | head -n 1)

if [ -n "$REPORT_FILE" ]; then
    echo "$LOG_DATA" >> "$REPORT_FILE"
fi

We only use now the POST_BACKUP_EXEC command to run the cleanup script directly after a backup run. All Cleanups are saved into the nautical backup log file, for example:

2026-01-26 02:10:30.535592 - INFO: Stopping audiobookshelf...
2026-01-26 02:10:30.962187 - INFO: Backing up audiobookshelf...
2026-01-26 02:10:34.763755 - INFO: Starting audiobookshelf...
2026-01-26 02:10:35.412853 - INFO: Backup of audiobookshelf complete!
2026-01-26 02:10:35.420051 - INFO: Running EXEC command: /config/backup_cleanup.sh
2026-01-26 02:10:35.562558 - INFO: Start Cleanup Script (delete 6 Days but keep at least 6 Backups)
2026-01-26 02:10:35.807700 - INFO: Delete 2026-01-19
2026-01-26 02:11:13.833577 - INFO: Deleted: 1 Backup(s) | 7 Backups left
2026-01-26 02:11:13.719186 - INFO: Completed in 10m 33s
2026-01-26 02:11:13.719323 - INFO: Success. 10 containers backed up! 1 skipped.

@Minituff maybe we could add this directly into your docker, if it looks stable. The script does only check / verify folders with 20??-??-?? Syntax and ignore all other stuff in the directory. Environment could be "CLEANUP_DEST_DATE_FOLDER: yes"" or smth else like that