diff --git a/.gitignore b/.gitignore index 620b1cd..6d8781a 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ gravity-sync.log +gravity-sync.cron gravity.db.last gravity-sync.conf backup/*.last diff --git a/README.md b/README.md index 9820252..7947742 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ For more information visit [https://vmstan.com/gravity-sync/](https://vmstan.com/gravity-sync/) -![Pull execution](https://user-images.githubusercontent.com/3002053/82719284-c396d880-9c6e-11ea-859c-b316357a495c.png) +![Pull execution](https://user-images.githubusercontent.com/3002053/82774990-f88c6200-9e0b-11ea-97e5-23c8b38f32e3.png) ## Background @@ -12,7 +12,7 @@ The script assumes you have one "primary" PH as the place you make all your conf It will **not** overwrite device specific settings such as local network configuration, admin/API passwords/keys, upstream DNS resolvers, etc. It will also **not** keep DHCP settings or device leases synchronized. -## Prereqs +## Prerequisites You will need to designate one Pi-Hole as primary and one as secondary. The designation is purely at your discretion and depends on your desired use case: @@ -68,9 +68,9 @@ Download the latest release from [GitHub](https://github.com/vmstan/gravity-sync ``` cd ~ -wget https://github.com/vmstan/gravity-sync/archive/v1.2.5.zip -unzip v1.2.5.zip -mv ~/gravity-sync-1.2.5 ~/gravity-sync +wget https://github.com/vmstan/gravity-sync/archive/v1.3.0.zip +unzip v1.3.0.zip +mv ~/gravity-sync-1.3.0 ~/gravity-sync cd gravity-sync ``` @@ -108,7 +108,7 @@ The script, when functioning in `pull` mode, will not prompt for user input afte ./gravity-sync.sh pull ``` -If the execution completes, you will now have overwritten your running gravity.db on the secondary PH after creating a copy (`gravity.db.backup`) in the `backup` directory located with your script. The script will also keep a copy of the last sync'd gravity.db from the master, in the `backup` folder called `gravity.db.last` should you need it for some reason. +If the execution completes, you will now have overwritten your running gravity.db on the secondary PH after creating a copy (`gravity.db.backup`) in the `backup` directory located with your script. The script will also keep a copy of the last sync'd gravity.db from the master, in the `backup` folder called `gravity.db.pull` should you need it for some reason. Finally, a file called `gravity-sync.log` will be created in the `gravity-sync` folder along side the script, with the date the script was last executed appended to the bottom. Over time I intend for this logging function to become more helpful. @@ -144,7 +144,7 @@ I've automated my synchronization using Crontab. If you'd like to keep this a ma ``` crontab -e -*/30 * * * * /home/USER/gravity-sync/gravity-sync.sh pull >/dev/null 2>&1 +*/30 * * * * /bin/bash /home/USER/gravity-sync/gravity-sync.sh pull > /home/USER/gravity-sync/gravity-sync.cron ``` Now, make another small adjustment to your primary settings. Now just wait until the annointed hour, and see if your changes have been synchronized. If so, profit! diff --git a/gravity-sync.conf.example b/gravity-sync.conf.example index ec8bfe1..d2bce9f 100644 --- a/gravity-sync.conf.example +++ b/gravity-sync.conf.example @@ -1,3 +1,10 @@ -# REQUIRED SETTINGS -REMOTE_HOST='192.168.1.10' # your master Pihole IP or DNS -REMOTE_USER='pi' # user account with sudo rights on master Pihole \ No newline at end of file +# REQUIRED SETTINGS ###################### +# Pi-hole Primary Node in IP or DNS +REMOTE_HOST='192.168.1.10' + +# User on Primary PH with SUDO rights +REMOTE_USER='pi' + +# Password for REMOTE_USER account +# Leave blank to use key-pair SSH auth +REMOTE_PASS='' \ No newline at end of file diff --git a/gravity-sync.sh b/gravity-sync.sh index f5a4abc..a077eff 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -1,7 +1,8 @@ #!/bin/bash # GRAVITY SYNC BY VMSTAN ##################### -VERSION='1.2.5' +PROGRAM='Gravity Sync' +VERSION='1.3.0' # Must execute from a location in the home folder of the user who own's it (ex: /home/pi/gravity-sync) # Configure certificate based SSH authentication between the Pi-hole HA nodes - it does not use passwords @@ -11,7 +12,7 @@ VERSION='1.2.5' # REQUIRED SETTINGS ########################## -# You MUST define REMOTE_HOST and REMOTE_USER in a file called 'gravity-sync.conf' +# You MUST define REMOTE_HOST and REMOTE_USER in a file called 'gravity-sync.conf' OK # You can copy the 'gravity-sync.conf.example' file in the script directory to get started # STANDARD VARIABLES ######################### @@ -20,12 +21,17 @@ VERSION='1.2.5' LOCAL_FOLDR='gravity-sync' # must exist in running user home folder CONFIG_FILE='gravity-sync.conf' # must exist as explained above SYNCING_LOG='gravity-sync.log' # will be created in above folder +CRONJOB_LOG='gravity-sync.cron' BACKUP_FOLD='backup' # PH Folder/File Locations PIHOLE_DIR='/etc/pihole' # default install directory GRAVITY_FI='gravity.db' # this should not change +############################################## +### DO NOT CHANGE ANYTHING BELOW THIS LINE ### +############################################## + # Script Colors RED='\033[0;91m' GREEN='\033[0;92m' @@ -34,22 +40,33 @@ YELLOW='\033[0;93m' PURPLE='\033[0;95m' NC='\033[0m' -############################################## -### DO NOT CHANGE ANYTHING BELOW THIS LINE ### -############################################## +# Message Codes +FAIL="[${RED}FAIL${NC}]" +WARN="[${PURPLE}WARN${NC}]" +GOOD="[${GREEN}GOOD${NC}]" +STAT="[${CYAN}EXEC${NC}]" +INFO="[${YELLOW}INFO${NC}]" # FUNCTION DEFINITIONS ####################### # Import Settings function import_gs { - echo -e "[${CYAN}STAT${NC}] Importing ${CONFIG_FILE} Settings" + MESSAGE="Importing ${CONFIG_FILE} Settings" + echo -e "${STAT} $MESSAGE" if [ -f $HOME/${LOCAL_FOLDR}/${CONFIG_FILE} ] then source $HOME/${LOCAL_FOLDR}/${CONFIG_FILE} - echo -e "[${GREEN}GOOD${NC}] Using ${REMOTE_USER}@${REMOTE_HOST}" + error_validate + + MESSAGE="Using ${REMOTE_USER}@${REMOTE_HOST}" + echo -e "${INFO} ${MESSAGE}" else - echo -e "[${RED}FAIL${NC}] Required ${CONFIG_FILE} Missing" - echo -e "Please review installation documentation for more information" + + echo -e "${FAIL} ${MESSAGE}" + + MESSAGE="${CONFIG_FILE} Missing" + echo -e "${INFO} ${MESSAGE}" + exit_nochange fi } @@ -58,58 +75,123 @@ function import_gs { function update_gs { TASKTYPE='UPDATE' logs_export # dumps log prior to execution because script stops after successful pull - echo -e "[${PURPLE}WARN${NC}] Requires GitHub Installation" + + MESSAGE="Requires GitHub Installation" + echo -e "${INFO} ${MESSAGE}" git reset --hard git pull + + exit +} + +# Developer Build Update +function beta_gs { + TASKTYPE='BETA' + logs_export # dumps log prior to execution because script stops after successful pull + + MESSAGE="Requires GitHub Installation" + echo -e "${INFO} ${MESSAGE}" + git reset --hard + git pull + git checkout origin/development + exit } # Pull Function function pull_gs { TASKTYPE='PULL' - echo -e "[${CYAN}STAT${NC}] Pulling ${GRAVITY_FI} from ${REMOTE_HOST}" + + echo -e "${INFO} ${TASKTYPE} Requested" + md5_compare + + MESSAGE="Pulling ${GRAVITY_FI} from ${REMOTE_HOST}" + echo -e "${STAT} ${MESSAGE}" rsync -v -e 'ssh -p 22' ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.pull - echo -e "[${CYAN}STAT${NC}] Backing Up Current ${GRAVITY_FI} on $HOSTNAME" + error_validate + + MESSAGE="Backing Up ${GRAVITY_FI} on $HOSTNAME" + echo -e "${STAT} ${MESSAGE}" cp -v ${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.backup - echo -e "[${CYAN}STAT${NC}] Replacing ${GRAVITY_FI} on $HOSTNAME" + error_validate + + MESSAGE="Replacing ${GRAVITY_FI} on $HOSTNAME" + echo -e "${STAT} ${MESSAGE}" sudo cp -v $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.pull ${PIHOLE_DIR}/${GRAVITY_FI} + error_validate + + MESSAGE="Setting Permissions on ${GRAVITY_FI}" + echo -e "${STAT} ${MESSAGE}" sudo chmod 644 ${PIHOLE_DIR}/${GRAVITY_FI} + error_validate + + MESSAGE="Setting Ownership on ${GRAVITY_FI}" + echo -e "${STAT} ${MESSAGE}" sudo chown pihole:pihole ${PIHOLE_DIR}/${GRAVITY_FI} - echo -e "${GRAVITY_FI} ownership and file permissions reset" - echo -e "[${CYAN}STAT${NC}] Reloading FTLDNS Configuration" + error_validate + + sleep 3 + + MESSAGE="Updating FTLDNS Configuration" + echo -e "${STAT} ${MESSAGE}" pihole restartdns reloadlists + error_validate + + MESSAGE="Reloading FTLDNS Services" + echo -e "${STAT} ${MESSAGE}" pihole restartdns - logs_export + error_validate + + logs_export exit_withchange } # Push Function function push_gs { TASKTYPE='PUSH' - echo -e "[${PURPLE}WARN${NC}] DATA LOSS IS POSSIBLE" - echo -e "The standard use of this script is to ${YELLOW}PULL${NC} data from the" - echo -e "primary PH server to the secondary. By issuing a ${YELLOW}PUSH${NC}, we" - echo -e "will instead overwrite the ${GRAVITY_FI} on ${YELLOW}${REMOTE_HOST}${NC}" - echo -e "with ${YELLOW}$HOSTNAME${NC} server data. A copy of the remote ${GRAVITY_FI}" - echo -e "will be saved to this server at:" - echo -e "${YELLOW}$HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.push${NC}" - echo -e "" - echo -e "Are you sure you want to overwrite the primary node configuration on ${REMOTE_HOST}?" + + echo -e "${INFO} ${TASKTYPE} Requested" + md5_compare + + echo -e "${WARN} Are you sure you want to overwrite the primary node configuration on ${REMOTE_HOST}?" select yn in "Yes" "No"; do case $yn in Yes ) - # echo "Replacing gravity.db on primary" - echo -e "[${CYAN}STAT${NC}] Backing Up ${GRAVITY_FI} from ${REMOTE_HOST}" + + MESSAGE="Backing Up ${GRAVITY_FI} from ${REMOTE_HOST}" + echo -e "${STAT} ${MESSAGE}" rsync -v -e 'ssh -p 22' ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.push - echo -e "[${CYAN}STAT${NC}] Pushing ${GRAVITY_FI} to ${REMOTE_HOST}" + error_validate + + MESSAGE="Pushing ${GRAVITY_FI} to ${REMOTE_HOST}" + echo -e "${STAT} ${MESSAGE}" rsync --rsync-path="sudo rsync" -v -e 'ssh -p 22' ${PIHOLE_DIR}/${GRAVITY_FI} ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} - echo -e "[${CYAN}STAT${NC}] Applying Permissions to Remote ${GRAVITY_FI}" + error_validate + + MESSAGE="Setting Permissions on ${GRAVITY_FI}" + echo -e "${STAT} ${MESSAGE}" ssh ${REMOTE_USER}@${REMOTE_HOST} "sudo chmod 644 ${PIHOLE_DIR}/${GRAVITY_FI}" + error_validate + + MESSAGE="Setting Ownership on ${GRAVITY_FI}" + echo -e "${STAT} ${MESSAGE}" ssh ${REMOTE_USER}@${REMOTE_HOST} "sudo chown pihole:pihole ${PIHOLE_DIR}/${GRAVITY_FI}" - echo -e "[${CYAN}STAT${NC}] Reloading FTLDNS Configuration" + error_validate + + sleep 3 + + MESSAGE="Updating FTLDNS Configuration" + echo -e "${STAT} ${MESSAGE}" ssh ${REMOTE_USER}@${REMOTE_HOST} 'pihole restartdns reloadlists' + error_validate + + MESSAGE="Reloading FTLDNS Services" + echo -e "${STAT} ${MESSAGE}" + ssh ${REMOTE_USER}@${REMOTE_HOST} 'pihole restartdns' - logs_export + error_validate + + logs_export exit_withchange ;; @@ -123,15 +205,28 @@ function push_gs { # Logging Functions ## Check Log Function function logs_gs { - echo -e "Recent ${YELLOW}PULL${NC} attempts" + echo -e "========================================================" + echo -e "Recent Complete ${YELLOW}PULL${NC} Executions" tail -n 10 ${SYNCING_LOG} | grep PULL - echo -e "Recent ${YELLOW}UPDATE${NC} attempts" + echo -e "Recent Complete ${YELLOW}UPDATE${NC} Executions" tail -n 10 ${SYNCING_LOG} | grep UPDATE - echo -e "Recent ${YELLOW}PUSH${NC} attempts" + echo -e "Recent Complete ${YELLOW}PUSH${NC} Executions" tail -n 10 ${SYNCING_LOG} | grep PUSH + echo -e "========================================================" exit_nochange } +## Check Last Crontab +function logs_crontab { + echo -e "========================================================" + echo -e "========================================================" + echo -e "" + cat ${CRONJOB_LOG} + echo -e "" + echo -e "========================================================" + echo -e "========================================================" +} + ## Log Out function logs_export { echo -e "[${CYAN}STAT${NC}] Logging Timestamps to ${SYNCING_LOG}" @@ -142,32 +237,58 @@ function logs_export { # Validate Functions ## Validate GS Folders function validate_gs_folders { - if [ -d $HOME/${LOCAL_FOLDR} ] - then - echo -e "[${GREEN}GOOD${NC}] Required $HOME/${LOCAL_FOLDR} Located" - else - echo -e "[${RED}FAIL${NC}] Required $HOME/${LOCAL_FOLDR} Missing" - exit_nochange - fi + MESSAGE="Locating $HOME/${LOCAL_FOLDR}" + echo -e "${STAT} ${MESSAGE}" + if [ -d $HOME/${LOCAL_FOLDR} ] + then + echo -e "${GOOD} ${MESSAGE}" + else + echo -e "${FAIL} ${MESSAGE}" + exit_nochange + fi - if [ -d $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} ] - then - echo -e "[${GREEN}GOOD${NC}] Required $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} Located" - else - echo -e "[${RED}FAIL${NC}] Required $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} Missing" - exit_nochange - fi + MESSAGE="Locating $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}" + echo -e "${STAT} ${MESSAGE}" + if [ -d $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} ] + then + echo -e "${GOOD} ${MESSAGE}" + else + echo -e "${FAIL} ${MESSAGE}" + exit_nochange + fi } ## Validate PH Folders function validate_ph_folders { - if [ -d ${PIHOLE_DIR} ] - then - echo -e "[${GREEN}GOOD${NC}] Required ${PIHOLE_DIR} Located" - else - echo -e "[${RED}FAIL${NC}] Required ${PIHOLE_DIR} Missing" - exit_nochange - fi + MESSAGE="Locating ${PIHOLE_DIR}" + echo -e "${STAT} ${MESSAGE}" + if [ -d ${PIHOLE_DIR} ] + then + echo -e "${GOOD} ${MESSAGE}" + else + echo -e "${FAIL} ${MESSAGE}" + exit_nochange + fi +} + +## Validate SSHPASS +function validate_os_sshpass { + if hash sshpass 2>/dev/null + then + if test -z "$REMOTE_PASS" + then + sshpassword='' + MESSAGE="Using SSH Key-Pair Authentication" + else + sshpassword="sshpass -p ${REMOTE_PASS} " + MESSAGE="Using SSH Password Authentication" + fi + else + sshpassword='' + MESSAGE="Using SSH Key-Pair Authentication" + fi + + echo -e "$INFO $MESSAGE" } # List GS Arguments @@ -178,11 +299,16 @@ function list_gs_arguments { echo -e "Replication Options:" echo -e " ${YELLOW}pull${NC} Sync the ${GRAVITY_FI} configuration on primary PH to this server" echo -e " ${YELLOW}push${NC} Force any changes made on this server back to the primary PH" + echo -e " ${YELLOW}compare${NC} Check to see if there is any variance between primary and secondary" echo -e "" - echo -e "Debugging Options:" + echo -e "Update Options:" echo -e " ${YELLOW}update${NC} Use GitHub to update this script to the latest version available" + echo -e " ${YELLOW}beta${NC} Use GitHub to update this script to the latest beta version available" + echo -e "" + echo -e "Debug Options:" echo -e " ${YELLOW}version${NC} Display the version of the current installed script" echo -e " ${YELLOW}logs${NC} Show recent successful jobs" + echo -e " ${YELLOW}cron${NC} Display output of last crontab execution" echo -e "" exit_nochange } @@ -190,79 +316,184 @@ function list_gs_arguments { # Exit Codes ## No Changes Made function exit_nochange { - echo -e "[${CYAN}STAT${NC}] Exiting Without Making Changes" - exit + echo -e "${INFO} ${PROGRAM} ${TASKTYPE} Aborting" + exit 0 } ## Changes Made function exit_withchange { - echo -e "[${CYAN}STAT${NC}] ${GRAVITY_FI} ${TASKTYPE} Completed" - exit + echo -e "${INFO} ${PROGRAM} ${TASKTYPE} Completed" + exit 0 +} + +# Error Validation +function error_validate { + if [ "$?" != "0" ]; then + echo -e "${FAIL} ${MESSAGE}" + exit 1 + else + echo -e "${GOOD} ${MESSAGE}" + fi +} + +# Output Version +function show_version { + echo -e "${INFO} ${PROGRAM} ${VERSION}" +} + +# Look for Changes +function md5_compare { + echo -e "${INFO} Comparing ${GRAVITY_FI} Changes" + + MESSAGE="Analyzing Remote ${GRAVITY_FI}" + echo -e "${STAT} ${MESSAGE}" + primaryMD5=$(ssh ${REMOTE_USER}@${REMOTE_HOST} 'md5sum /etc/pihole/gravity.db') + error_validate + + MESSAGE="Analyzing Local ${GRAVITY_FI}" + echo -e "${STAT} ${MESSAGE}" + secondMD5=$(md5sum ${PIHOLE_DIR}/${GRAVITY_FI}) + error_validate + + if [ "$primaryMD5" == "$secondMD5" ] + then + echo -e "${INFO} No Changes in ${GRAVITY_FI}" + exit_nochange + else + echo -e "${INFO} Changes Detected in ${GRAVITY_FI}" + fi + } # SCRIPT EXECUTION ########################### - -echo -e "[${CYAN}STAT${NC}] Evaluating Script Arguments" + + MESSAGE="Evaluating Script Arguments" + echo -e "${STAT} ${MESSAGE}" case $# in 0) - echo -e "[${RED}FAIL${NC}] Missing Required Arguments" + echo -e "${FAIL} ${MESSAGE}" list_gs_arguments ;; 1) case $1 in pull) - echo -e "[${GREEN}GOOD${NC}] Pull Requested" - import_gs - - echo -e "[${CYAN}STAT${NC}] Validating Folder Configuration" + echo -e "${GOOD} ${MESSAGE}" + + import_gs + + MESSAGE="Validating Folder Configuration" + echo -e "${INFO} ${MESSAGE}" validate_gs_folders validate_ph_folders + validate_os_sshpass pull_gs exit ;; push) - echo -e "[${GREEN}GOOD${NC}] Push Requested" - import_gs + echo -e "${GOOD} ${MESSAGE}" + + import_gs - echo -e "[${CYAN}STAT${NC}] Validating Folder Configuration" + echo -e "${INFO} Validating Folder Configuration" validate_gs_folders validate_ph_folders + validate_os_sshpass push_gs exit ;; version) - echo -e "[ ${PURPLE}INFO${NC} ] Gravity Sync ${VERSION}" + TASKTYPE='VERSION' + show_version exit_nochange ;; update) - echo -e "[${GREEN}GOOD${NC}] Update Requested" + # TASKTYPE='UPDATE' + echo -e "${GOOD} ${MESSAGE}" + + echo -e "${INFO} Update Requested" update_gs exit_nochange ;; + + beta) + # TASKTYPE='BETA' + echo -e "${GOOD} ${MESSAGE}" + + echo -e "${INFO} Beta Update Requested" + beta_gs + exit_nochange + ;; logs) - echo -e "[${GREEN}GOOD${NC}] Logs Requested" + TASKTYPE='LOGS' + MESSAGE="Logs Requested" + echo -e "${GOOD} ${MESSAGE}" logs_gs ;; + + compare) + TASKTYPE='COMPARE' + + echo -e "${GOOD} ${MESSAGE}" + import_gs + + echo -e "${INFO} Validating Folder Configuration" + validate_gs_folders + validate_ph_folders + validate_os_sshpass + + md5_compare + ;; + + cron) + TASKTYPE='CRON' + echo -e "${GOOD} ${MESSAGE}" + + CRONPATH="$HOME/${LOCAL_FOLDR}/${CRONJOB_LOG}" + + MESSAGE="Replaying Last Cronjob" + echo -e "${STAT} ${MESSAGE}" + + if [ -f ${CRONPATH} ] + then + if [ -s ${CRONPATH} ] + echo -e "${GOOD} ${MESSAGE}" + logs_crontab + exit_nochange + then + echo -e "${FAIL} ${MESSAGE}" + echo -e "${INFO} ${CRONPATH} appears empty" + exit_nochange + fi + else + echo -e "${FAIL} ${MESSAGE}" + echo -e "${YELLOW}${CRONPATH}${NC} cannot be located" + exit_nochange + fi + + ;; *) - echo -e "[${RED}FAIL${NC}] '${YELLOW}$1${NC}' is Invalid Argument" + MESSAGE="'${YELLOW}$1${NC}' is an Invalid Argument" + echo -e "${FAIL} ${MESSAGE}" list_gs_arguments + exit_nochange ;; esac ;; *) - echo -e "[${RED}FAIL${NC}] Too Many Arguments" - list_gs_arguments - exit_nochange + MESSAGE="Too Many Arguments" + echo -e "${FAIL} ${MESSAGE}" + list_gs_arguments + exit_nochange ;; esac \ No newline at end of file