Containers skipped

[tetra-archos]

I get the following in my log file:

Backup Report - 2024-09-19 03:00:01.122325
2024-09-19 03:00:01.139351 - INFO: Starting backup...
2024-09-19 03:00:01.189656 - INFO: Processing 6 containers...
2024-09-19 03:00:01.236685 - INFO: Stopping nautical-backup...
2024-09-19 03:00:01.423824 - INFO: Shutting down API...

My compose file is:

services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    restart: unless-stopped
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /opt/nautical-backup/config:/config
      - /opt/:/app/source
      - /media/files/Settings/Backups/Docker/Iris:/app/destination
      - /opt/nautical-backup/ntfy.sh:/ntfy.sh # Mount ntfy script
    environment:
      - CRON_SCHEDULE=0 3 * * *
      - SKIP_CONTRAINERS=nautical-backup
      - TZ=America/New_York
      - POST_BACKUP_EXEC=/ntfy.sh

and all that is in my backup directory after running is my log file. The ntfy.sh is just a script pushing the output to ntfy.

[Minituff]

Hi,

It seems like Nautical is attempting to backup itself. I have some steps in place to prevent this but it seems like that isn't working.

Could you add the following label to your Nautical config:

"nautical-backup.enable=false"
"nautical-backup.stop-before-backup=false"

If this fixes the problem, then I need to troubleshoot further as to why Nautical didn't skip itself.

Could you also set the console LOG_LEVEL to TRACE and share what you find in the Docker console (not the backup report).

Also, great use of the nfty script. I love that service!

[tetra-archos]

nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 2 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | INFO: Connected to database at '/config/nautical-db.json'
nautical-backup    | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 2 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 2 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup    | INFO: Connected to database at '/config/nautical-db.json'
nautical-backup    | DEBUG: Verifying source directory '/app/source'...
nautical-backup    | TRACE: Source directory '/app/source' READ access verified
nautical-backup    | DEBUG: Verifying Nautical destination directory '/app/destination'...
nautical-backup    | TRACE: Destination directory '/app/destination' READ/WRITE access verified
nautical-backup    | INFO: Starting backup...
nautical-backup    | DEBUG: Verifying destination directory '/app/destination'...
nautical-backup    | TRACE: Destination directory '/app/destination' READ/WRITE access verified
nautical-backup    | INFO: Processing 6 containers...
nautical-backup    | DEBUG: Containers: nautical-backup, diun-diun-1, uptime-kuma, komodo_core_1, komodo_ferretdb_1, komodo_periphery_1
nautical-backup    | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'minituff/nautical-backup'.
nautical-backup    | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'nautical-backup'.
nautical-backup    | DEBUG: Skipping nautical-backup based on label
nautical-backup    | DEBUG: diun-diun-1 - Source directory '/app/source/diun-diun-1' does not exist. Skipping
nautical-backup    | DEBUG: Container diun-diun-1 was not stopped. No need to start.
nautical-backup    | INFO: Stopping uptime-kuma...
nautical-backup    | DEBUG: Verifying destination directory '/app/destination'...
nautical-backup    | TRACE: Destination directory '/app/destination' READ/WRITE access verified
nautical-backup    | ERROR: Destination directory '/app/destination/uptime-kuma' does not exist
nautical-backup    | INFO: Backing up uptime-kuma...
nautical-backup    | DEBUG: RUNNING: 'rsync -raq  /app/source/uptime-kuma/ /app/destination/uptime-kuma/'
nautical-backup    | INFO: Starting uptime-kuma...
nautical-backup    | INFO: Backup of uptime-kuma complete!
nautical-backup    | DEBUG: komodo_core_1 - Source directory '/app/source/komodo_core_1' does not exist. Skipping
nautical-backup    | DEBUG: Container komodo_core_1 was not stopped. No need to start.
nautical-backup    | DEBUG: komodo_ferretdb_1 - Source directory '/app/source/komodo_ferretdb_1' does not exist. Skipping
nautical-backup    | DEBUG: Container komodo_ferretdb_1 was not stopped. No need to start.
nautical-backup    | DEBUG: komodo_periphery_1 - Source directory '/app/source/komodo_periphery_1' does not exist. Skipping
nautical-backup    | DEBUG: Container komodo_periphery_1 was not stopped. No need to start.
nautical-backup    | DEBUG: Running POST_BACKUP_EXEC
nautical-backup    | TRACE: Setting environment variable NB_EXEC_COMMAND to /ntfy.sh
nautical-backup    | TRACE: Setting environment variable NB_EXEC_ATTACHED_TO_CONTAINER to False
nautical-backup    | TRACE: Setting environment variable NB_EXEC_CONTAINER_NAME to None
nautical-backup    | TRACE: Setting environment variable NB_EXEC_CONTAINER_ID to None
nautical-backup    | TRACE: Setting environment variable NB_EXEC_BEFORE_DURING_OR_AFTER to AFTER
nautical-backup    | INFO: Running EXEC command: /ntfy.sh
nautical-backup    | WARN: Exec command error: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
nautical-backup    |                                  Dload  Upload   Total   Spent    Left  Speed
100   332  100   226  100   106   1458    684 --:--:-- --:--:-- --:--:--  2155
nautical-backup    | DEBUG: Exec command output: {"id":"Xqg24Z07XI3E","time":1726815606,"expires":1726858806,"event":"message","topic":"nautical-backups","message":"Iris nautical-backup failed with status 127:\\n/ntfy.sh: line 8: nautical-backup-command: command not found"}
nautical-backup    | DEBUG: Containers completed: uptime-kuma
nautical-backup    | DEBUG: Containers skipped: nautical-backup, komodo_periphery_1, komodo_ferretdb_1, diun-diun-1, komodo_core_1
nautical-backup    | INFO: Completed in 00m 05s
nautical-backup    | INFO: Success. 1 containers backed up! 5 skipped.

Backup Report - 2024-09-20 03:00:01.290184
2024-09-20 03:00:01.302373 - INFO: Starting backup...
2024-09-20 03:00:01.347862 - INFO: Processing 6 containers...
2024-09-20 03:00:01.391589 - INFO: Stopping uptime-kuma...
2024-09-20 03:00:05.725986 - ERROR: Destination directory '/app/destination/uptime-kuma' does not exist
2024-09-20 03:00:05.749564 - INFO: Backing up uptime-kuma...
2024-09-20 03:00:06.247634 - INFO: Starting uptime-kuma...
2024-09-20 03:00:06.566118 - INFO: Backup of uptime-kuma complete!
2024-09-20 03:00:06.593174 - INFO: Running EXEC command: /ntfy.sh
2024-09-20 03:00:06.769319 - WARN: Exec command error: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   332  100   226  100   106   1458    684 --:--:-- --:--:-- --:--:--  2155
2024-09-20 03:00:06.791666 - INFO: Completed in 00m 05s
2024-09-20 03:00:06.806711 - INFO: Success. 1 containers backed up! 5 skipped.
2024-09-20 07:42:12.298277 - INFO: Shutting down API...
2024-09-20 07:42:19.267723 - INFO: Starting API on port 8069...

[Minituff]

Thanks for sending me the logs. I think I may have misunderstood your issue before.

Based on these logs, I can see that Nautical is not trying to back itself up

nautical-backup    | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'minituff/nautical-backup'.
nautical-backup    | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'nautical-backup'.
nautical-backup    | DEBUG: Skipping nautical-backup based on label

The last log, "Shutting Down", was this caused by you shutting down the container?

---

I think these logs are the most relevant to your issue.

nautical-backup    | DEBUG: Containers: nautical-backup, diun-diun-1, uptime-kuma, komodo_core_1, komodo_ferretdb_1, komodo_periphery_1
nautical-backup    | DEBUG: komodo_core_1 - Source directory '/app/source/komodo_core_1' does not exist. Skipping
nautical-backup    | DEBUG: komodo_ferretdb_1 - Source directory '/app/source/komodo_ferretdb_1' does not exist. Skipping
nautical-backup    | DEBUG: komodo_periphery_1 - Source directory '/app/source/komodo_periphery_1' does not exist. Skipping

Nautical is looking for a source folder that matches the container name. Since all your containers end in _1, this is likely the reason they are getting skipped.

You have a few options to remedy this (You only need to choose 1).

1. Use the OVERRIDE_SOURCE_DIR enviornment variable on the Nautical config.

# Nautical compose
OVERRIDE_SOURCE_DIR=komodo_core_1:komodo_core,komodo_ferretdb_1:komodo_ferretdb,komodo_periphery_1:komodo_periphery

2. Update the container_name property on each of the containers

Example:

services:
  komodo_core:
    container_name: komodo_core

3. Use the nautical-backup.override-source-dir label on each of your containers

services:
  komodo_core:
    container_name: komodo_core
    labels:
      - nautical-backup.override-source-dir=komodo_core

[tetra-archos]

So that was after I put the tags in you suggested to keep it from reading nautical, and I'm unsure why my it's reading a _1 on my container names, that isn't in my directory structure or docker-compose.yml files

[Minituff]

It looks like the -1 is coming from the project name of the Docker compose project?
Source; https://stackoverflow.com/questions/33045358/docker-compose-image-named-prefix-s-1-instead-of-s

I haven't encountered this before myself, but you may need to do some tinkering to either remove the -1 or add that to the nautical config so it can be recognized.

[IanWardell]

@Minituff I just wanted to chime in here and say I am also seeing this issue. I am seeing "-1" appended to all my containers which is resulting in skipped backups. I'll attach the logs at the bottom.

I use Portainer to manage some containers, these are created in stacks.
I believe portainer stores data in /data/compose/ and the normal /var/lib/docker/volumes

for example:

root@NAS:/home/machine/nautical-backups#  ls /var/lib/docker/volumes
metadata.db                      teslamate-deploy_teslamate-db
teslamate-deploy_mosquitto-conf  teslamate-deploy_teslamate-grafana-data
teslamate-deploy_mosquitto-data

and data/compose

root@NAS:/home/machine/nautical-backups# ls /data/compose/*
/data/compose/10:
openvpn_conf

/data/compose/11:
openvpn-data

/data/compose/13:
ccd_master  easyrsa_master

/data/compose/2:
app  proxy

/data/compose/3:
config  db

/data/compose/34:
import

/data/compose/35:
import

/data/compose/36:
import

/data/compose/37:
import

/data/compose/4:
config  db

docker-compose:

services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock           # Required for Docker interaction
      - /var/lib/docker/volumes:/app/source:ro              # Mount all Docker volumes as the source
      - /home/machine/nautical-backups/config:/config       # Configuration directory
      - /home/machine/nautical-backups/destination:/app/destination  # Backup destination
    environment:
      - TZ=America/New_York
      - BACKUP_ON_START=true
      - RUN_ONCE=true
      - CRON_SCHEDULE_ENABLED=false
      - LOG_LEVEL=TRACE

nautical container runtime logs:

DEBUG: REPORT_FILE: true
INFO: Nautical Backup Version: 2.10.4
DEBUG: Built for the platform: linux/amd64
DEBUG: Perparing enviornment variables...
DEBUG: Found defaults.env
DEBUG: CRON_SCHEDULE: 0 4 * * *
DEBUG: USE_DEST_DATE_FOLDER: false
DEBUG: DEST_DATE_PATH_FORMAT: date/container
DEBUG: DEST_DATE_FORMAT: %Y-%m-%d
DEBUG: USE_CONTAINER_BACKUP_DATE: true
DEBUG: USE_DEFAULT_RSYNC_ARGS: true
DEBUG: REQUIRE_LABEL: false
DEBUG: LABEL_PREFIX: nautical-backup
DEBUG: REPORT_FILE_LOG_LEVEL: INFO
DEBUG: REPORT_FILE_ON_BACKUP_ONLY: true
DEBUG: KEEP_SRC_DIR_NAME: true
DEBUG: EXIT_AFTER_INIT: false
DEBUG: LOG_RSYNC_COMMANDS: false
DEBUG: SOURCE_LOCATION: /app/source
DEBUG: DEST_LOCATION: /app/destination
DEBUG: TEST_MODE: -1
DEBUG: HTTP_REST_API_ENABLED: true
DEBUG: HTTP_REST_API_USERNAME: admin
DEBUG: HTTP_REST_API_PASSWORD: password
DEBUG: ADDITIONAL_FOLDERS_WHEN: before
DEBUG: ADDITIONAL_FOLDERS_USE_DEST_DATE_FOLDER: false
DEBUG: NAUTICAL_DB_PATH: /config
DEBUG: NAUTICAL_DB_NAME: nautical-db.json
TRACE: PRE_BACKUP_CURL: 
TRACE: POST_BACKUP_CURL: 
TRACE: RSYNC_CUSTOM_ARGS: 
TRACE: OVERRIDE_SOURCE_DIR: 
TRACE: OVERRIDE_DEST_DIR: 
TRACE: SELF_CONTAINER_ID: 
TRACE: SKIP_CONTAINERS: 
TRACE: SKIP_STOPPING: 
TRACE: ADDITIONAL_FOLDERS: 
DEBUG: SELF_CONTAINER_ID: a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a
INFO: Skipping CRON installation since CRON_SCHEDULE_ENABLED=false
DEBUG: Verifying source directory '/app/source'...
DEBUG: Verifying destination directory '/app/destination'...
INFO: Connected to database at '/config/nautical-db.json'
DEBUG: Installing nautical backup script...
TRACE: ln -s /app/backup.sh /usr/local/bin/nautical
TRACE: TEST_MODE: -1
INFO: Starting backup since BACKUP_ON_START is true
INFO: API listening on port 8069...
INFO: Note - BACKUP_ON_START logs are not available until all containers are processed, however the report file updates in real-time.
INFO: Initialization complete. Awaiting CRON schedule: 0 4 * * *
TRACE: Running: exec with-contenv python3 -m uvicorn app.api.main:app --host 0.0.0.0 --port 8069 --lifespan on --use-colors --log-level 'warning'
INFO: Connected to database at '/config/nautical-db.json'
DEBUG: Verifying source directory '/app/source'...
TRACE: Source directory '/app/source' READ access verified
DEBUG: Verifying Nautical destination directory '/app/destination'...
TRACE: Destination directory '/app/destination' READ/WRITE access verified
INFO: Starting backup...
DEBUG: Verifying destination directory '/app/destination'...
TRACE: Destination directory '/app/destination' READ/WRITE access verified
INFO: Processing 12 containers...
DEBUG: Containers: nautical-backup, zoneminder, zm_db, teslamate-deploy-teslamate-1, ovpn-admin-ovpn-admin-1, ovpn-admin-openvpn-1, portainer_portainer_1, nginx, teslamate-deploy-mosquitto-1, teslamate-deploy-database-1, teslamate-deploy-grafana-1, buildx_buildkit_default
TRACE: Skipping nautical-backup a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a because it's image matches 'minituff/nautical-backup'.
TRACE: Skipping nautical-backup a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a because it's image matches 'nautical-backup'.
TRACE: Skipping nautical-backup a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a because it's ID is the same as Nautical
DEBUG: zoneminder - Source directory '/app/source/zoneminder' does not exist. Skipping
DEBUG: Container zoneminder was not stopped. No need to start.
DEBUG: zm_db - Source directory '/app/source/zm_db' does not exist. Skipping
DEBUG: Container zm_db was not stopped. No need to start.
DEBUG: teslamate-deploy-teslamate-1 - Source directory '/app/source/teslamate-deploy-teslamate-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-teslamate-1 was not stopped. No need to start.
DEBUG: ovpn-admin-ovpn-admin-1 - Source directory '/app/source/ovpn-admin-ovpn-admin-1' does not exist. Skipping
DEBUG: Container ovpn-admin-ovpn-admin-1 was not stopped. No need to start.
DEBUG: ovpn-admin-openvpn-1 - Source directory '/app/source/ovpn-admin-openvpn-1' does not exist. Skipping
DEBUG: Container ovpn-admin-openvpn-1 was not stopped. No need to start.
DEBUG: portainer_portainer_1 - Source directory '/app/source/portainer_portainer_1' does not exist. Skipping
DEBUG: Container portainer_portainer_1 was not stopped. No need to start.
DEBUG: nginx - Source directory '/app/source/nginx' does not exist. Skipping
DEBUG: Container nginx was not stopped. No need to start.
DEBUG: teslamate-deploy-mosquitto-1 - Source directory '/app/source/teslamate-deploy-mosquitto-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-mosquitto-1 was not stopped. No need to start.
DEBUG: teslamate-deploy-database-1 - Source directory '/app/source/teslamate-deploy-database-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-database-1 was not stopped. No need to start.
DEBUG: teslamate-deploy-grafana-1 - Source directory '/app/source/teslamate-deploy-grafana-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-grafana-1 was not stopped. No need to start.
DEBUG: buildx_buildkit_default - Source directory '/app/source/buildx_buildkit_default' does not exist. Skipping
DEBUG: Container buildx_buildkit_default was not stopped. No need to start.
DEBUG: Containers completed: 
DEBUG: Containers skipped: ovpn-admin-openvpn-1, nginx, teslamate-deploy-teslamate-1, zm_db, buildx_buildkit_default, zoneminder, teslamate-deploy-mosquitto-1, ovpn-admin-ovpn-admin-1, portainer_portainer_1, nautical-backup, teslamate-deploy-database-1, teslamate-deploy-grafana-1
INFO: Completed in 00m 00s
INFO: Success. 0 containers backed up! 12 skipped.
INFO: RUN_ONCE is true. Exiting...

nautical config

root@NAS:/home/machine/nautical-backups# cat config/nautical-db.json
{
    "created_at": "2024-12-23 02:02:02",
    "backup_running": false,
    "containers_skipped": 12,
    "containers_completed": 0,
    "number_of_containers": 12,
    "errors": 0,
    "last_backup_seconds_taken": 0,
    "last_cron": "12/23/24 02:46",
    "completed": "0"
}

[Minituff]

Hi @IanWardell,

Are you able to share some of your container configs as well?

I'm wondering if you need to apply the container_name property to each of your containers.

[IanWardell]

Hi @IanWardell,

Are you able to share some of your container configs as well?

I'm wondering if you need to apply the container_name property to each of your containers.

I haven't added the container_naame property, but I can update them and see what happens.

I did try the OVERRIDE_SOURCE_DIR= method and it didn't work.

services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock           # Required for Docker interaction
      - /data/compose/:/app/source:ro              # Mount all Docker volumes as the source
      - /home/machine/nautical-backups/config:/config       # Configuration directory
      - /home/machine/nautical-backups/destination:/app/destination  # Backup destination
    environment:
      - TZ=America/New_York
      - BACKUP_ON_START=true
      - RUN_ONCE=true
      - CRON_SCHEDULE_ENABLED=false
      - LOG_LEVEL=TRACE
      - OVERRIDE_SOURCE_DIR=teslamate-deploy-teslamate-1:teslamate-deploy-teslamate
#DEBUG: Containers skipped: ovpn-admin-openvpn-1, nginx, teslamate-deploy-teslamate-1, zm_db, buildx_buildkit_default, zoneminder, teslamate-deploy-mosquitto-1, ovpn-admin-ovpn-admin-1, portainer_portainer_1, nautical-backup, teslamate-deploy-database-1, teslamate-deploy-grafana-1

here is an example docker-compose

version: "3"
services:
  teslamate:
    image: teslamate/teslamate:latest
    restart: always
    environment:
      - ENCRYPTION_KEY=[REDACTED]
      - DATABASE_USER=[REDACTED]
      - DATABASE_PASS=[REDACTED]
      - DATABASE_NAME=teslamate
      - DATABASE_HOST=database
      - MQTT_HOST=mosquitto
    ports:
      - "127.0.0.1:4001:4000"
    volumes:
      - ./import:/opt/app/import
    cap_drop:
      - all

  database:
    image: postgres:15
    restart: always
    environment:
      - POSTGRES_USER=[REDACTED]
      - POSTGRES_PASSWORD=[REDACTED]
      - POSTGRES_DB=teslamate
    volumes:
      - teslamate-db:/var/lib/postgresql/data

  grafana:
    image: teslamate/grafana:latest
    restart: always
    environment:
      - DATABASE_USER=[REDACTED]
      - DATABASE_PASS=[REDACTED]
      - DATABASE_NAME=teslamate
      - DATABASE_HOST=database
    ports:
      - 3000:3000
    volumes:
      - teslamate-grafana-data:/var/lib/grafana

  mosquitto:
    image: eclipse-mosquitto:2
    restart: always
    command: mosquitto -c /mosquitto-no-auth.conf
    # ports:
    #   - 1883:1883
    volumes:
      - mosquitto-conf:/mosquitto/config
      - mosquitto-data:/mosquitto/data

volumes:
  teslamate-db:
  teslamate-grafana-data:
  mosquitto-conf:
  mosquitto-data:

[IanWardell]

@Minituff

Can you verify what /source needs to be? And should I be listing a source for each docker-compose stack and container?

I have volumes in a few places ex:

• /var/lib/docker/volumes
• /data/compose/* [ ex 33/* or 3/* where each portainer stack puts the volumes and compose files]

rename done:

machine@NAS:~/NAS-Dockerfiles/portainer$ docker ps
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS                  PORTS                                                              NAMES
84c85ee747ab        portainer/portainer-ce                   "/portainer -H unix:…"   40 seconds ago      Up 38 seconds           8000/tcp, 9443/tcp, 0.0.0.0:9000->9000/tcp                         portainer_portainer
e479126d93f8        flant/ovpn-admin:latest                  "/app/ovpn-admin"        3 minutes ago       Up 3 minutes                                                                               ovpn-admin-ovpn-admin
6ae802cff2c4        openvpn:local                            "/etc/openvpn/setup/…"   3 minutes ago       Up 3 minutes            0.0.0.0:7777->1194/tcp, 127.0.0.1:8888->8080/tcp                   ovpn-admin-openvpn
ef8f2eebce75        postgres:15                              "docker-entrypoint.s…"   9 minutes ago       Up 8 minutes            5432/tcp                                                           teslamate-deploy-database
edb8a5d92143        teslamate/teslamate:latest               "tini -- /bin/sh /en…"   9 minutes ago       Up 8 minutes            127.0.0.1:4001->4000/tcp                                           teslamate-deploy-teslamate
8b413e1be965        teslamate/grafana:latest                 "/run.sh"                9 minutes ago       Up 8 minutes            0.0.0.0:3000->3000/tcp                                             teslamate-deploy-grafana
c837d918ab1e        eclipse-mosquitto:2                      "/docker-entrypoint.…"   9 minutes ago       Up 8 minutes            1883/tcp                                                           teslamate-deploy-mosquitto
19ff8942c4cf        quantumobject/docker-zoneminder:latest   "/sbin/my_init"          13 hours ago        Up 12 hours             6802/tcp, 9000/tcp, 0.0.0.0:9080->80/tcp                           zoneminder
87575c8a298a        mysql/mysql-server:5.7                   "/entrypoint.sh mysq…"   13 hours ago        Up 12 hours (healthy)   3306/tcp, 33060/tcp                                                zm_db
020f9bec71b6        nginx:latest                             "/docker-entrypoint.…"   11 months ago       Up 12 hours             0.0.0.0:4000->4000/tcp, 80/tcp, 0.0.0.0:8080-8081->8080-8081/tcp   nginx
5b5eeb9cffd9        moby/buildkit:buildx-stable-1            "buildkitd"              17 months ago       Up 12 hours                                                                                buildx_buildkit_default

debug output:

Lines
Actions

      

DEBUG: REPORT_FILE: true

INFO: Nautical Backup Version: 2.10.4

DEBUG: Built for the platform: linux/amd64

DEBUG: Perparing enviornment variables...

DEBUG: Found defaults.env

DEBUG: CRON_SCHEDULE: 0 4 * * *

DEBUG: USE_DEST_DATE_FOLDER: false

DEBUG: DEST_DATE_PATH_FORMAT: date/container

DEBUG: DEST_DATE_FORMAT: %Y-%m-%d

DEBUG: USE_CONTAINER_BACKUP_DATE: true

DEBUG: USE_DEFAULT_RSYNC_ARGS: true

DEBUG: REQUIRE_LABEL: false

DEBUG: LABEL_PREFIX: nautical-backup

DEBUG: REPORT_FILE_LOG_LEVEL: INFO

DEBUG: REPORT_FILE_ON_BACKUP_ONLY: true

DEBUG: KEEP_SRC_DIR_NAME: true

DEBUG: EXIT_AFTER_INIT: false

DEBUG: LOG_RSYNC_COMMANDS: false

DEBUG: SOURCE_LOCATION: /app/source

DEBUG: DEST_LOCATION: /app/destination

DEBUG: TEST_MODE: -1

DEBUG: HTTP_REST_API_ENABLED: true

DEBUG: HTTP_REST_API_USERNAME: admin

DEBUG: HTTP_REST_API_PASSWORD: password

DEBUG: ADDITIONAL_FOLDERS_WHEN: before

DEBUG: ADDITIONAL_FOLDERS_USE_DEST_DATE_FOLDER: false

DEBUG: NAUTICAL_DB_PATH: /config

DEBUG: NAUTICAL_DB_NAME: nautical-db.json

TRACE: PRE_BACKUP_CURL: 

TRACE: POST_BACKUP_CURL: 

TRACE: RSYNC_CUSTOM_ARGS: 

TRACE: OVERRIDE_SOURCE_DIR: 

TRACE: OVERRIDE_DEST_DIR: 

TRACE: SELF_CONTAINER_ID: 

TRACE: SKIP_CONTAINERS: 

TRACE: SKIP_STOPPING: 

TRACE: ADDITIONAL_FOLDERS: 

DEBUG: SELF_CONTAINER_ID: 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8

INFO: Skipping CRON installation since CRON_SCHEDULE_ENABLED=false

DEBUG: Verifying source directory '/app/source'...

DEBUG: Verifying destination directory '/app/destination'...

INFO: Connected to database at '/config/nautical-db.json'

DEBUG: Installing nautical backup script...

TRACE: ln -s /app/backup.sh /usr/local/bin/nautical

TRACE: TEST_MODE: -1

INFO: API listening on port 8069...

INFO: Starting backup since BACKUP_ON_START is true

INFO: Initialization complete. Awaiting CRON schedule: 0 4 * * *

TRACE: Running: exec with-contenv python3 -m uvicorn app.api.main:app --host 0.0.0.0 --port 8069 --lifespan on --use-colors --log-level 'warning'

INFO: Note - BACKUP_ON_START logs are not available until all containers are processed, however the report file updates in real-time.

INFO: Connected to database at '/config/nautical-db.json'

DEBUG: Verifying source directory '/app/source'...

TRACE: Source directory '/app/source' READ access verified

DEBUG: Verifying Nautical destination directory '/app/destination'...

TRACE: Destination directory '/app/destination' READ/WRITE access verified

INFO: Starting backup...

DEBUG: Verifying destination directory '/app/destination'...

TRACE: Destination directory '/app/destination' READ/WRITE access verified

INFO: Processing 12 containers...

DEBUG: Containers: nautical-backup, zoneminder, zm_db, portainer_portainer, ovpn-admin-ovpn-admin, ovpn-admin-openvpn, teslamate-deploy-database, teslamate-deploy-teslamate, teslamate-deploy-grafana, teslamate-deploy-mosquitto, nginx, buildx_buildkit_default

TRACE: Skipping nautical-backup 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8 because it's image matches 'minituff/nautical-backup'.

TRACE: Skipping nautical-backup 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8 because it's image matches 'nautical-backup'.

TRACE: Skipping nautical-backup 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8 because it's ID is the same as Nautical

DEBUG: zoneminder - Source directory '/app/source/zoneminder' does not exist. Skipping

DEBUG: Container zoneminder was not stopped. No need to start.

DEBUG: zm_db - Source directory '/app/source/zm_db' does not exist. Skipping

DEBUG: Container zm_db was not stopped. No need to start.

DEBUG: portainer_portainer - Source directory '/app/source/portainer_portainer' does not exist. Skipping

DEBUG: Container portainer_portainer was not stopped. No need to start.

DEBUG: ovpn-admin-ovpn-admin - Source directory '/app/source/ovpn-admin-ovpn-admin' does not exist. Skipping

DEBUG: Container ovpn-admin-ovpn-admin was not stopped. No need to start.

DEBUG: ovpn-admin-openvpn - Source directory '/app/source/ovpn-admin-openvpn' does not exist. Skipping

DEBUG: Container ovpn-admin-openvpn was not stopped. No need to start.

DEBUG: teslamate-deploy-database - Source directory '/app/source/teslamate-deploy-database' does not exist. Skipping

DEBUG: Container teslamate-deploy-database was not stopped. No need to start.

DEBUG: teslamate-deploy-teslamate - Source directory '/app/source/teslamate-deploy-teslamate' does not exist. Skipping

DEBUG: Container teslamate-deploy-teslamate was not stopped. No need to start.

DEBUG: teslamate-deploy-grafana - Source directory '/app/source/teslamate-deploy-grafana' does not exist. Skipping

DEBUG: Container teslamate-deploy-grafana was not stopped. No need to start.

DEBUG: teslamate-deploy-mosquitto - Source directory '/app/source/teslamate-deploy-mosquitto' does not exist. Skipping

DEBUG: Container teslamate-deploy-mosquitto was not stopped. No need to start.

DEBUG: nginx - Source directory '/app/source/nginx' does not exist. Skipping

DEBUG: Container nginx was not stopped. No need to start.

DEBUG: buildx_buildkit_default - Source directory '/app/source/buildx_buildkit_default' does not exist. Skipping

DEBUG: Container buildx_buildkit_default was not stopped. No need to start.

DEBUG: Containers completed: 

DEBUG: Containers skipped: teslamate-deploy-database, zm_db, ovpn-admin-ovpn-admin, ovpn-admin-openvpn, zoneminder, teslamate-deploy-mosquitto, nginx, nautical-backup, teslamate-deploy-teslamate, buildx_buildkit_default, portainer_portainer, teslamate-deploy-grafana

INFO: Completed in 00m 00s

INFO: Success. 0 containers backed up! 12 skipped.

INFO: RUN_ONCE is true. Exiting...

[Minituff]

Yeah it would be helpful if you could list out a few of your compose files. I am mostly looking to see how you mount your volumes to each of your apps.

Currently, Nautical doesnt really support native docker files--it's really been designed for bind mounts. So Nautical cant see the volumes in /var/lib/docker/volumes yet.

So lets take a look at 2 of your configs and how we can update them.

  database:
    image: postgres:15
    container_name: postgresql. # Added Name
    restart: always
    environment:
      - POSTGRES_USER=[REDACTED]
      - POSTGRES_PASSWORD=[REDACTED]
      - POSTGRES_DB=teslamate
    volumes:
      # - teslamate-db:/var/lib/postgresql/data  #OLD 
      - /home/machine/docker_volumes/postgresql:var/lib/postgresql/data #NEW

  grafana:
    image: teslamate/grafana:latest
    container_name: grafane # Added name
    restart: always
    environment:
      - DATABASE_USER=[REDACTED]
      - DATABASE_PASS=[REDACTED]
      - DATABASE_NAME=teslamate
      - DATABASE_HOST=database
    ports:
      - 3000:3000
    volumes:
      # - teslamate-grafana-data:/var/lib/grafana # OLD
      - /home/machine/docker_volumes/grafana:/var/lib/grafana # NEW

services:
  nautical-backup:
    image: minituff/nautical-backup:latest
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock           # Required for Docker interaction
      # - /data/compose/:/app/source:ro              # Mount all Docker volumes as the source (OLD)
      - /home/machine/docker_volumes/app/source:ro  # Mount all Docker volumes as the source (NEW)
      - /home/machine/nautical-backups/config:/config       # Configuration directory
      - /home/machine/nautical-backups/destination:/app/destination  # Backup destination
    environment:
      # Anything here

The goal is that the /app/source folder inside Nautical will look like this:

/app/souce:
  - grafana/
  - postgresql/
  - other containers .....

These paths would match the container names so nautical knows how to associate the containers.

I am working on Nautical version 3, which would actually read the Compose files and even look inside the /var/lib* folders for the container volumes, but right now that's not working quite yet.

[cirrusflyer]

I'm managing my stacks with Dockge. And I'm also seeing odd behavior with container names. For example, I have some named like "stirling-pdf-stirling-pdf-1" and "cloudflared-cloudflared-1" and "pocket-id-pocket-id-1". But not all are doing this. Almost seems like the older ones are so I wonder if it was a Dockge thing.

***Duh. The normal names have an explicit "container_name" setting in the compose file. That's all it was.

[cirrusflyer]

But it would be nice if Nautical could deal with these easily. What's the best solution at this point? For example, when using Dockge, the config and data reside in a folder "/opt/stacks/karakeep". The container names for this stack are:

karakeep-web-1
karakeep-chrome-1
karakeep-meilisearch-1