BackupScripts

This page documents a couple of backup shell scripts that I am using to implement my backup solution.

Copy script

rsync

The script that creates a copy of a data set using rsync is

htb-mkbackupcopy.sh

I have made the script somewhat generic and added it to herzbube's toolbox, my personal collection of shell scripts. Here's a direct link (GitHub) to the latest version of the script.

The script uses rsync to copy a source folder to a destination folder. The source folder can be on a remote machine, the destination folder is either on a local filesystem, on a Samba share or on a disk image (which in turn can be located on a Samba share). Here's the script's usage text:

pi@raspberrypi1:~$ htb-mkbackupcopy.sh -h
htb-mkbackupcopy.sh [-h] [-i <pattern>] [-e <pattern>] [-s <samba-spec>] [-d <disk-image-path>] [[user@]host:]source destination
 -h: Print this usage text
 -i <pattern>: An include pattern to be passed to rsync.
   This parameter can be specified multiple times. The
   order in which include/exclude patterns are specified
   is important.
 -e <pattern>: An exclude pattern to be passed to rsync.
   This parameter can be specified multiple times. The
   order in which include/exclude patterns are specified
   is important.
 [-s <samba-spec>]: Specification of a Samba share to
   mount. The specification conforms to the usual syntax
   used by "mount -t smbfs" (see man "mount_smbfs").
 [-d <disk-image-path>]: The path to a disk image to
   mount. If -s is also specified, the path specified
   here must exist on the Samba share after it is
   mounted. The disk image can be anything that can be
   processed by hdiutil.
 [[user@]host:]source: Specification of source folder.
   Specification of remote source folder conforms to
   the usual rsync / SSH syntax.
 destination: Specification of destination folder. If
  -s is specified, the path specified here must exist
  on the Samba share after it is mounted. If -d is
  specified (regardless of whether -s is also specified),
  the path specified here must exist on the disk image
  after it is mounted.

Exit codes:
 0: No error
 2: Aborted by signal (e.g. Ctrl+C)
 3: Error during initialisation
 4: Error while checking arguments
 5: Error during main program

sftp

A second script that can create a copy of a data set using sftp is

htb-sftp-get.sh

As with the first copy script, I have made this second script somewhat generic and added it to herzbube's toolbox, my personal collection of shell scripts. Here's a direct link (GitHub) to the latest version of the script.

The script uses sftp and sshpass (only if passwordless authentication is not possible) to copy a source folder to a destination folder. The source folder must be on a remote machine, the destination folder must be on a local filesystem. Here's the script's usage text:

pi@raspberrypi1:~$ htb-sftp-get.sh -h
htb-sftp-get.sh [-h] [-d <distinct-regex>] [-l <local-file-name-regex>] [-f <password-file-path>] [-e <password-envvar-name>] [user@]host:source-folder-path destination-folder-path
  -h: Print this usage text
 [-d <distinct-regex>]: Specification of a regular expression
   to extract that part from the names of the remote files
   which can be used to identify distinct sets of files from
   which only the one with the most recent modification
   timestamp should be transferred. If no regex is specified,
   then no such optimization occurs and all files in the
   remote folder are transferred. The regex must be an sed
   extended regex that contains at least one group. If more
   than one group is in the regex, only the first one is
   used. Back references are used to extract the file name
   part matched by the group. For instance, if the files
   foo-2025-03-23.zip, foo-2025-03-24.zip,
   foobar-2025-03-23.zip and foobar-2025-03-24.zip exist,
   then the regex "^([^-]+-).*$" can be used to match the
   distinct parts "foo-" and "foobar-". Note that the
   character "-" must be included in the distinction group,
   to avoid problems with the distinction logic which would
   otherwise occur because "foo" is a substring of "foobar".
   Only the files foo-2025-03-24.zip and bar-2025-03-24.zip
   will be transferred (based on the modification timestamp,
   not the date in the file name!).
 [-l <local-file-name-regex>]: Specification of a regular
   expression to extract parts from the names of the remote
   files to use to form the local file name. If this option
   is not specified, then transferred files are stored
   locally under the same name they have remotely. This
   option is only valid if -d is also specified. The regex
   must be an sed extended regex that contains at least two
   groups. If more than two groups are in the regex, only
   the first two are used. Back references are used to
   extract the file name parts matched by the groups. In
   the example given for the -d option, if the regex
   "^([^-]+).*(.zip)$" were used this would cause the local
   file names to be foo.zip and bar.zip.
 [-f <password-file-path>]: Path to a file that contains
   the password to use for authentication. If neither this
   nor -e is specified, then passwordless authentication
   is assumed. If specified this option is forwarded to
   sshpass.
 [-e <password-envvar-name>]: Name of an environment
   variable that contains the password to use for
   authentication. If neither this nor -f is specified,
   then passwordless authentication is assumed. If specified
   this option is forwarded to sshpass.
 [user@]host:source-path: Specification of remote host and
   remote source path, optionally with the name of the user
   to authenticate as. The specification conforms to the
   SFTP syntax. Read the SFTP man page for details of
   different SFTP behaviour when you specify a file or
   a folder. Specifically, the options -d and -l only work
   as expected if the remote source path is a folder.
 destination-folder-path: Specification of destination
   folder. The folder is created if it does not exist.

Exit codes:
 0: No error
 2: Aborted by signal (e.g. Ctrl+C)
 3: Error during initialisation
 4: Error while checking arguments
 5: Error during main program

Snapshot script

The script that creates a snapshot of a data set is

htb-mkbackupsnapshot.sh

As with the copy script, I have made the snapshot script somewhat generic and added it to herzbube's toolbox. Here's a direct link (GitHub) to the latest version of the script.

The script uses bup to create a snapshot of a specified source folder. Both the source folder and the bup repository must be somewhere on a local file system. Here's the script's usage text:

pi@raspberrypi1:~$ htb-mkbackupsnapshot.sh -h
htb-mkbackupsnapshot.sh [-h] source bup-repository
 -h: Print this usage text
 source: Specification of folder to snapshot.
 bup-repository: Specification of the bup repository
   folder.

Exit codes:
 0: No error
 2: Aborted by signal (e.g. Ctrl+C)
 3: Error during initialisation
 4: Error while checking arguments
 5: Error during main program