Added support for listening for an MQTT message to do an “on demand” sync.

[davidabacon]

OBJECTIVE

My changes are minor - just added a subroutine to listen for MQTT message (only if MQT_BROKER_URL is defined). Upon receipt of the message, a sync is fired.

Hopefully this could be of use to the wider community.

[jjb14778]

well i simple did this to make a trigged sync between multiple shared piholes as a service to make it work myself easy. This is based on my R&D work on it:

to start, KEEP it SIMPLE!!!

Perhaps build it in orbital-sync with a config setting that enables this build in trigger service that runs the triggersync service when using in a stack with a mgmt primary pihole (using internally shared log volume with a pihole container as shown in my dns-mgmt orbital pihole docker-compose.yml example below)

I'm able to compare the mgmt pilhole ftl.log read from a shared docker volume and use it to trigger the update of others if FTL.log changes. As FTL.log always changes when settings are changed on my mgt pihole.

R&D setup: One mgmt pihole in docker and the pihole-update.service localy on the docker host checking mgmt pihole docker mapped logfile volume location for the logfile directly. And another secondary pihole in docker that gets updated.

This trigger logfile check i use could be build in orbital-sync containr as well
what is needed further development than the stuff below:
config variables for execute commands within the trigger-service that triggers the action from a script on a needed orbital-sync mgmt config docker volume for config or sripts location. As i run my dns-mgmt host local i can simple trigger the sync container start, to trigger it from a container needs more work.

The obvious stuff, log file locations, docker container names, or custom start command for example to use ssh and docker start to run orbital triggered sync on a other docker host for example. Or fetch the porbital-sync container output and print it to the servicee log as well for example.

How to maybe integrate it in de orbital-sync container

Create a service config file (debian/ubuntu):

sudo -i mkdir /etc/pupdate
sudo -i nano /etc/pupdate/pihole-update.service
sudo -i chown -R 0644 /etc/pupdate/pihole-update.service
sudo -i ln -s /etc/pupdate/pihole-update.service /etc/systemd/system/pihole-update.service

pihole-update.service filecontent to save

[Unit]
Description="Pi-Hole Update Trigger"
After=multi-user.target
After=network-online.target
Wants=network-online.target

[Service]
Type=idle
Restart=always
ExecStart=bash /etc/pupdate/check-pihole.sh

# Log location for the Pi-Hole Update Trigger service itself
# Needs updating to orbital container logoutput location
StandardOutput=file:/var/log/PiholeUpdateTrigger.log # needs updating to orbital container logoutput
StandardError=file:/var/log/PiholeUpdateTrigger-error.log
Restart=always

[Install]
WantedBy=multi-user.target

Create the Pi-Hole Update Trigger script used in the service.

Create a folder /etc/pupdate and the check-pihole.sh service-script

sudo -i nano /etc/pupdate/check-pihole.sh 
sudo -i chmod -R 0764 /etc/pupdate/check-pihole.sh

/etc/pupdate/check-pihole.sh filecontent to save

#start endless loop
while true
do
      # FTL log file location (mapped shared stack docker volume from docker-compose where both containers in the stack can 
      # read the FTL.log file )
      # R&D i use my docker volume location to de FTL.log in the mgmt pihole container. This is how to do it in stack.
      FILE="/var/log/pihole/FTL.log"

      # Update last FTL logfile line to update.new 
      # As FTL last logfile line always changes when pihole settings get saved, something is written in the FTL.log).
      # update last FTL log line to update.new (for next loop compare).
      tail "$FILE" -n1 > /tmp/update.new

      # read last FTL log file line saved from last check.
      STRING=$(cat /tmp/update.chk)
      
      # read last FTL log file line udpdated from FTL log.
      SUB=$(cat /tmp/update.new)
 
# start internal check compare loop
	while [[ "$STRING" == "$SUB" ]]
         # update last FTL log line to update.new (for next loop compare).
         do
      		# update last FTL log line to update.new (for next loop compare).
		tail "$FILE" -n1 > /tmp/update.new

      		# reset vars for savety
		STRING=""
      		SUB=""

      		# Update last vars from files to reset check loop.
		STRING=$(cat /tmp/update.chk)
      		SUB=$(cat /tmp/update.new)

                # time sleeping until the logfile compare loop check starts in seconds.
      		sleep 3;
	  done
		 
    # Else , if the last logfile lines compared where not similar, trigger commands and reset loop via updating files
			
	        echo "starting orbital sync"
	        #resetting the sync loop after triggering or not.
		# update last FTL log line to update.chk as last loop run compare value.
		tail "$FILE" -n1 > /tmp/update.chk

		# update last FTL log line to update.new (for next loop compare as well).
                tail "$FILE" -n1 > /tmp/update.new
                
		#executing the trigger commands below to run docker container <name> of your orbital-sync runonce container
                echo "triggered update of dns-pri.services.local"
                docker start dns-mgmt-sync-pri-1 -i # Use a Docker vfariable SECONDARY_HOSTS_1_TRIGGERCMD.

                #change to MQTT message to do an “on demand” sync execute  afer the monitoed pihole (mgt pihole) changes.
                #perhaps with additional needed container variables to make it work and configurable.
                #use this against a triggerable orbital-sync container running MQTT triger listening mode. 
                
               # add more for extra sync containers, to sync them sequentially one after another.
               #time for the next pihole sync starts after triggering in seconds.
      		#sleep 3;
                #echo "triggered update of pihole-pri.services.local"
                #docker start dns-mgmt-sync-sec-1 -i
done
echo "End" 
# close the service script endless loop only exits if its false, so it will never get to echo End.

Register the Pi-Hole trigger service in the container to make it work! (debian/ubuntu).

Enable service

sudo -i systemctl enable pihole-update.service
# triggered service start with the propossed orbital-sync sync trigger container var PRIMARY_HOST_TRIGGER_SVC.
# make the var unusable combined with the run_once var for example.
#sudo -i systemctl start pihole-update.service 

Propossed pihole orbital-sync mgmt stack example

docker-compose.yml

version: "3"

volumes:
  log-pihole: { driver: local }
  etc-pihole: { driver: local }
  etc-dnsmasq.d: { driver: local }

#networks: # ipv6
#  default: # ipv6
#    driver: host # ipv6
#    driver_opts: # ipv6
#      com.docker.network.enable_ipv6: "true" # ipv6

services:
  dns-mgmt:
    container_name: dns-mgmt
    hostname: dns-mgmt.services.local
    image: pihole/pihole:latest
    ports:
      #- "[::]:53:53/tcp" # ipv6
      #- "[::]:53:53/udp" # ipv6
      - "xxx.xxx.xxx.xxx:53:53/tcp"
      - "xxx.xxx.xxx.xxx:53:53/udp"
      - "xxx.xxx.xxx.xxx:67:67/udp" #Pi-hole as your DHCP server      
      - "xxx.xxx.xxx.xxx:80:80/tcp"
    environment:
      SKIPGRAVITYONBOOT: '1'
      QUERY_LOGGING: 'true'
      CUSTOM_CACHE_SIZE: 5000
      QUERY_LOGGING: 'false'
      PIHOLE_DNS_: '1.1.1.1;1.0.0.1;8.8.8.8;8.8.4.4'
      #PIHOLE_DNS_: '1.1.1.1;1.0.0.1;8.8.8.8;8.8.4.4;2001:4860:4860:0:0:0:0:8888;2606:4700:4700::1111' # ipv6
      DNS_BOGUS_PRIV: 'true'
      DNS_FQDN_REQUIRED: 'true'
      DNSMASQ_LISTENING: 'all'
      CORS_HOSTS: 'services.local'
      REV_SERVER: 'true'
      REV_SERVER_DOMAIN: 'services.local'
      REV_SERVER_TARGET: 'xxx.xxx.xxx.xxx' #internal service.local dnsdomain forward server ip.
      REV_SERVER_CIDR: 'xxx.xxx.xxx.xxx/xx' #CIDR subnet and range 10.0.0.0/8 for example.
      PIHOLE_DOMAIN: 'services.local'
      DNSSEC: 'false'
      IPv6: 'false'
      #IPv6: 'true' # ipv6
      TZ: 'Europe/Amsterdam'
      WEBPASSWORD: ''
    volumes:
      - 'log-pihole:/var/log/pihole' #volume location where pi-hole trigger service can work with the FLTlog.
      - 'etc-pihole:/etc/pihole'
      - 'etc-dnsmasq.d:/etc/dnsmasq.d'
    cap_add:
      - NET_ADMIN # Required if you are using Pi-hole as your DHCP server, else not needed
    restart: always

#new trigger docker feature usage example, with this stuff build in activated with the example below.
#  sync-mgmt:
#    image: mattwebbio/orbital-sync:latest
#    depends_on:
#      - dns-mgmt
#    extra_hosts:
#      - "dns-mgmt.services.local:xxx.xxx.xxx.xxx" #container dnshostname
#      - "dns-pri.services.local:xxx.xxx.xxx.xxx" #container dnshostname
#      #- "dns-sec.services.local:xxx.xxx.xxx.xxx" #container dnshostname
#    environment:
#      RUN_ONCE: 'false'
#      TZ: 'Europe/Amsterdam'
#      PRIMARY_HOST_TRIGGER_SVC: 'true'
#      PRIMARY_HOST_CONTAINER_NAME: dns-mgmt
#      SECONDARY_HOSTS_1_TRIGGERCMD: 'docker start dns-mgmt-sync-pri-1'
#      #SECONDARY_HOSTS_2_TRIGGERCMD: 'docker start dns-mgmt-sync-sec-1'
#      INTERVAL_TRIGGERNEXT_TIMEOUT: 30 #seconds between sync targets so you minimalise downtime on dns resolution.
#    volumes:
#      - 'log-pihole:/var/log/pihole' #volume location where pi-hole trigger service can work with the FLTlog on the shared stack volume.      
 
  sync-pri:
    image: mattwebbio/orbital-sync:latest
    depends_on:
      - dns-mgmt
    extra_hosts:
      - "dns-mgmt.services.local:xxx.xxx.xxx.xxx" #container dnshostname
      - "dns-pri.services.local:xxx.xxx.xxx.xxx" #container dnshostname
    environment:
      RUN_ONCE: 'true'
      TZ: 'Europe/Amsterdam'
      PRIMARY_HOST_BASE_URL: 'http://dns-mgmt' #use stack internal pihole service name resolvable without fqdn.
      PRIMARY_HOST_PASSWORD: ''
      SECONDARY_HOSTS_1_BASE_URL: 'http://dns-pri.services.local'
      SECONDARY_HOSTS_1_PASSWORD: ''
      INTERVAL_MINUTES: 1
      SYNC_V5_WHITELIST: 'true'
      SYNC_V5_REGEX_WHITELIST: 'true'
      SYNC_V5_BLACKLIST: 'true'
      SYNC_V5_REGEX_LIST: 'true'
      SYNC_V5_AD_LIST: 'true'
      SYNC_V5_CLIENT: 'true'
      SYNC_V5_GROUP: 'true'
      SYNC_V5_AUDIT_LOG: 'true'
      SYNC_V5_STATIC_DHCP_LEASES: 'true'
      SYNC_V5_LOCAL_DNS_RECORDS: 'true'
      SYNC_V5_LOCAL_CNAME_RECORDS: 'true'
      SYNC_V5_FLUSH_TABLES: 'true'
      
#  sync-sec: #sequentialy so eliminate pihole shared service availability.
#    image: mattwebbio/orbital-sync:latest
#    depends_on:
#      - dns-mgmt
#    extra_hosts:
#      - "dns-mgmt.services.local:xxx.xxx.xxx.xxx" #container internal dnshostname
#      - "dns-sec.services.local:xxx.xxx.xxx.xxx" #container internal dnshostname
#    environment:
#      RUN_ONCE: 'true'    
#      TZ: 'Europe/Amsterdam'
#      PRIMARY_HOST_BASE_URL: 'http://dns-mgmt' #use stack internal pihole service name resolvable without fqdn.
#      PRIMARY_HOST_PASSWORD: ''
#      SECONDARY_HOSTS_1_BASE_URL: 'http://dns-sec.services.local'
#      SECONDARY_HOSTS_1_PASSWORD: ''
#      INTERVAL_MINUTES: 1
#      SYNC_V5_WHITELIST: 'true'
#      SYNC_V5_REGEX_WHITELIST: 'true'
#      SYNC_V5_BLACKLIST: 'true'
#      SYNC_V5_REGEX_LIST: 'true'
#      SYNC_V5_AD_LIST: 'true'
#      SYNC_V5_CLIENT: 'true'
#      SYNC_V5_GROUP: 'true'
#      SYNC_V5_AUDIT_LOG: 'true'
#      SYNC_V5_STATIC_DHCP_LEASES: 'true'
#      SYNC_V5_LOCAL_DNS_RECORDS: 'true'
#      SYNC_V5_LOCAL_CNAME_RECORDS: 'true'
#      SYNC_V5_FLUSH_TABLES: 'true'

docker-compose.yml dns-pri stack example.

version: "3"
volumes:
  etc-pihole: { driver: local }
  etc-dnsmasq.d: { driver: local }

#networks: # ipv6
#  default: # ipv6
#    driver: host # ipv6
#    driver_opts: # ipv6
#      com.docker.network.enable_ipv6: "true" # ipv6

services:
  pihole:
    container_name: dns-pri
    hostname: dns-pri.services.local
    image: pihole/pihole:latest
    ports:
      #- "[::]:53:53/tcp" # ipv6
      #- "[::]:53:53/udp" # ipv6
      - "xxx.xxx.xxx.xxx:53:53/tcp"
      - "xxx.xxx.xxx.xxx:53:53/udp"
      - "xxx.xxx.xxx.xxx:67:67/udp" #Pi-hole as your DHCP server      
      - "xxx.xxx.xxx.xxx:80:80/tcp"
    environment:
      SKIPGRAVITYONBOOT: '1'
      QUERY_LOGGING: 'true'
      CUSTOM_CACHE_SIZE: 5000
      QUERY_LOGGING: 'false'
      PIHOLE_DNS_: '1.1.1.1;1.0.0.1;8.8.8.8;8.8.4.4'
      #PIHOLE_DNS_: '1.1.1.1;1.0.0.1;8.8.8.8;8.8.4.4;2001:4860:4860:0:0:0:0:8888;2606:4700:4700::1111' # ipv6
      DNS_BOGUS_PRIV: 'true'
      DNS_FQDN_REQUIRED: 'true'
      DNSMASQ_LISTENING: 'all'
      CORS_HOSTS: 'services.local'
      REV_SERVER: 'true'
      REV_SERVER_DOMAIN: 'services.local'
      REV_SERVER_TARGET: 'xxx.xxx.xxx.xxx' #internal service.local dnsdomain forward server ip.
      REV_SERVER_CIDR: 'xxx.xxx.xxx.xxx/xx' #CIDR subnet and range 10.0.0.0/8 for example.
      PIHOLE_DOMAIN: 'services.local'
      DNSSEC: 'false'
      IPv6: 'false'
      #IPv6: 'true' # ipv6
      TZ: 'Europe/Amsterdam'
      WEBPASSWORD: ''
    volumes:
      - 'etc-pihole:/etc/pihole'
      - 'etc-dnsmasq.d:/etc/dnsmasq.d'
    cap_add:
      - NET_ADMIN # Required if you are using Pi-hole as your DHCP server, else not needed
    restart: always

Cheerz 🍺🦝🦝🦝

do whatever you want with it. i use my own stuff until you build it better ;)
after you fix: #535 (comment)
i have minimum time to spend on this, just liked to share my simple out of the box working build idea.

Maybe just discard all above and seek contact and integration with the pi-hole docker project offficially and integrate orbital-sync triggering via reading out changes from the Pihole API for example.

Maybe update orbital-sync to orbital-mgmt, make what you currently have into a combined solution so eterprises can use pihole docker containers HA disposable and manageble when combinted with orbital-mgmt for the mgmt service.

The pi-hole API has features for rebooting and system mgmt already. Simple integrate them, maybe build a simple mgmt webui from the pihole ui. where to add piholes, manage triggers, configs cental. Or just integrate pihole and the orbital idea all togheter. Make them bloodey usable and efficient Pi-Holes remotly managable and usable. 😍 As for a really big part you already did.