upgrade to v0.6.10

* update README

.gitignore

@@ -0,0 +1,3 @@
+*.html
+*.bak
+*~

Makefile

@@ -1,7 +1,7 @@
 # dsnap-sync
 # https://github.com/rzerres/dsnap-sync
 # Copyright (C) 2016, 2017 James W. Barnett
-# Copyright (C) 2017 - 2019 Ralf Zerres
+# Copyright (C) 2017 - 2021 Ralf Zerres
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by

PKGBUILD

@@ -1,8 +1,8 @@
-# Maintainer: Ralf Zerres <ralf.zerres.de at gmail dot com>
+# Maintainer: Ralf Zerres <ralf.zerres at mail dot de>
 pkgname=dsnap-sync
-pkgver=0.6.0
+pkgver=0.6.8
 pkgrel=1
-pkgdesc="Use snapper snapshots to backup to external drive"
+pkgdesc="BTRFS centric backups tool supporting snapper history for snapshot backups"
 arch=(any)
 url="https://github.com/rzerres/dsnap-sync"
 license=('GPL')

README.md

@@ -1,12 +1,12 @@
 <!-- dsnap-sync README.md -->
-<!-- version: 0.5.9 -->
+<!-- version: 0.6.10 -->
 
 # dsnap-sync
 
 <p align="center">
   <span>English</span> |
-  <!-- a href="lang/spanish#dsnap-sync">Spanish</a> | -->
-  <a href="lang/german#dsnap-sync">Deutsch</a>
+  <!-- a href="lang/spanish">Spanish</a> | -->
+  <a href="lang/german">Deutsch</a>
 </p>
 
 ## About
@@ -184,6 +184,7 @@ Please use your host software package manager.
 		  --label-running <desc>  snapper description tagging active jobs. Default: "dsnap-sync in progress"
 		  --label-synced <desc>   snapper description tagging last synced jobs.
 								  Default: "dsnap-sync last incremental"
+          --calculate-btrfs-size  Enable calculation of sync-size for given snapshots
 		  --color                 Enable colored output messages
 	  -c, --config <config>       Specify the snapper configuration to use. Otherwise will perform for each snapper
 								  configuration. You can select multiple configurations
@@ -191,6 +192,7 @@ Please use your host software package manager.
 		  --config-postfix <name> Specify a postfix that will be appended to the destination snapper config name.
 		  --dry-run               perform a trial run (no changes are written).
 		  --mediapool             Specify the name of the tape MediaPool
+          --no-btrfs-quota        don't consume btrfs-quota to estimate snapshot size
 	  -n, --noconfirm             Do not ask for confirmation for each configuration. Will still prompt for backup
 		  --nonotify              Disable graphical notification (via dbus)
 		  --nopv                  Disable graphical progress output (disable pv)
@@ -304,6 +306,62 @@ FUSE). Read and write access can be managed using common OS tools.
 An open-source implementation can be found at
 [LinearTapeFileSystem](https://github.com/LinearTapeFileSystem/ltfs).
 
+## Restore
+
+### From Snapshot
+
+When `dsnap-sync` did save the data using a native btrfs method
+(`btrfs-snapshot` or `btrfs-clone`), data will be stored in a snapper compatible
+directory structure on the target.
+
+The structure may look like:
+
+* └ backups
+   * └ @\<server-name\>
+	 * └ \<subvol-name\>
+	   *  └ \<subvol-id\>
+		 * ├ \.snapshot\>
+		 * └ info.xml
+
+To restore the data, you may either use `snapper` methods or copy back
+the needed files using standard OS tools.
+
+  * using snapper
+	You may use `snapper diff <id1>..<id2> [files]` to compare the differences
+	between the given suvol-id's.
+	You may use `snapper rollback <id> ` to restore all data from the given \<id\>
+
+  * using OS tools
+	Use your favorit file-manager and change into the directory storing your
+	target backup subvol-id. The snapshots are stored in **read-only** mode.
+	Data are presented flat file and can be copied to any desired destination.
+
+### From Tape
+
+When `dsnap-sync` did save the data with method `btrfs-archive`, you will find
+the corresponding data in a snapper compatible directory structure on the tape.
+
+The structure may look like:
+
+ * └ backups
+   * └ @\<server-name\>
+	 * └ archive-\<subvol-name\>
+	   *  └ \<subvol-id\>
+		 * ├ \<subvol-id\>_full.btrfs
+		 * └ info.xml
+
+The file `info.xml` provides the metadata corresponding to the snapshot.
+The data of the snapshot is stored in the file `<subvol-id>_full.btrfs`.
+This file has to be decrypted with btrfs tool `btrfs-send` to a btrfs
+restore directory:
+
+	cd /target_btrfs_path
+	cp /path_to_tape_root/backups/@<server-name>/archive-<subvol-name>/<subvol-id>_full.btrfs .
+	cat <subvol-id>_full.btrfs  | btrfs receive -v  .
+	rm <subvol-id>_full.btrfs
+
+Please consult btrfs-send man-page for further info.
+
 ## Contributing
 
 Help is very welcome! Feel free to fork and issue a pull request to add
@@ -331,4 +389,4 @@ This work is licensed under a [Creative Common License 4.0][License-CC_BY]
 ![Creative Common Logo][Logo-CC_BY]
 
 © 2016, 2017  James W. Barnett;
-© 2017 - 2018 Ralf Zerres
+© 2017 - 2025 Ralf Zerres

TODO.md

@@ -4,7 +4,7 @@
 
 - dsnap-sync: restore btrfs-streams from archive backups
   * find last full
-  * iterate oval available incrementals
+  * iterate over available incremental snapshots
 - dsnap-sync: restore btrfs snapshot from snapshot backups
   * $ssh btrfs send `<snapshot_path>/<snapshot-id>/snapshot_ro` | btrfs receive `/<btrfs-restore-dir>/`
 	(`recieved_uuid` attribte of `<btrfs-restore-dir>/snapshot_ro` will be imported from `<snapshot_path>/<snapshot-id>/snapshot_ro`)

bin/tape-admin

@@ -22,7 +22,7 @@
 # Helper routines for tape handling
 
 progname="${0##*/}"
-version="0.0.14"
+version="0.0.15"
 
 # global variables
 color=0
@@ -901,8 +901,8 @@ ltfs_reformat () {
 	    fi
 	    # wiping: put tape in an unformatted state
 	    ltfs_wipe
-	    # RET = 8 -> error?
 	    if [ $? -eq 0 ]; then
+		# format: create the ltfs partitions
 		ltfs_format $volume_name_active $volume_serial
 		if [ $? -eq 0 ]; then
 		    ltfs_mount
@@ -1091,7 +1091,7 @@ mount_tape () {
 
     ltfs_is_mounted
     if test $? -eq 0; then
-        # get date from last written tape in given mediapool
+	# if no explicit tape should be mounted, get date from last written tape in given mediapool
 	if [ "${#volume_name}" -eq 0 ]; then
 	    get_lastwrite ${mediapool_name}
 	    if [ $? != 0 ]; then
@@ -1128,7 +1128,7 @@ mount_tape () {
 	# check if given tape is poolmember of selected pool
 	get_poolmember ${mediapool_name} ${volume_name}
 	if [ $? -eq 0 ]; then
-	    # volume_name is member of given mediapool
+	    # check the mediapolicy for given volume_name
 	    get_mediapolicy ${mediapool_name} ${volume_name_active}
 	    if [ ${#volume_mediapolicy} -gt 0  ]; then
 		if [ ${volume_mediapolicy} = "append" ] ; then
@@ -1147,6 +1147,8 @@ mount_tape () {
 			      printf "${MAGENTA}Valid tape, retensiondate has exposed: ${GREEN}%s${NO_COLOR}\n" \
 				     "${volume_retensiondate}"
 			  fi
+			  #ltfs_format ${volume_name_active}
+			  ltfs_reformat ${volume_name_active}
 			  return 0
 		      fi
 		      if [ $? -eq 1 ]; then
@@ -1188,7 +1190,7 @@ mount_tape () {
 				     "${volume_name}"
 			  fi
 			  ltfs_wipe
-			  #ltfs_format ${volume_name} ${volume_serial}
+			  #ltfs_reformat ${volume_name_active}
 			  if [ $? -eq 0 ]; then
 			      return 0
 			  fi
@@ -1298,7 +1300,7 @@ mount_tape () {
 		RET=$?
 		if [ $RET -eq 2 ]; then
 		    if [ $verbose -ge 1 ]; then
-			printf "${MAGENTA}RetensionDate has exposed, mount the tape\n"
+			printf "${MAGENTA}RetensionDate has exposed, mount the tape${NO_COLOR}\n"
 		    fi
 		    ltfs_mount
 		    return 0
@@ -1326,7 +1328,7 @@ mount_tape () {
 		if [ $? -eq 2 ]; then
 		    # retensiondate has exposed: wipe given tape
 		    ltfs_wipe
-		    #ltfs_format ${volume_name} ${volume_serial}
+		    ltfs_format ${volume_name} ${volume_serial}
 		    if [ $? -eq 0 ]; then
 			return 0
 		    fi
@@ -1386,7 +1388,7 @@ mtx_getlabel () {
 	    volume_name_active=$(perl -ne '
 		/Data Transfer Element (\d+):Full \(Storage Element (\d+) Loaded\)(:VolumeTag =\s*(.+))?/ && print "$4\n";' ${TMPFILE})
 	    volume_name_active=$(echo $volume_name_active | sed -e 's/ *$//g')
-	    if [ $verbose -ge 2 ]; then
+	    if [ $verbose -ge 2 ] && [ ${#volume_name_active} -gt 0 ]; then
 		printf "${MAGENTA}Tape in slot ${GREEN}%s${MAGENTA} has Label: ${GREEN}%s${NO_COLOR}\n" \
 		       "${slot_source}" "${volume_name_active}"
 	    fi
@@ -2405,76 +2407,70 @@ case $cmd in
 	fi
 	;;
     ltfs-is-mounted)
+	if ! $quiet; then
+	    printf "${MAGENTA}LTFS tape mount checke\n"
+	fi
 	ltfs_is_mounted
 	if test $? -gt 0; then
 	    exit 1
-	else
-	    if ! $quiet; then
-		printf "${MAGENTA}LTFS Tape is-mounted: ${GREEN}true${NO_COLOR}\n"
-	    fi
 	fi
 	;;
     ltfs-mount)
+	if ! $quiet; then
+	    printf "${MAGENTA}LTFS mount Tape\n"
+	fi
 	ltfs_mount
 	if test $? -gt 0; then
 	    exit 1
-	else
-	    if ! $quiet; then
-		printf "${MAGENTA}LTFS Tape mount: ${GREEN}true${NO_COLOR}\n"
-	    fi
 	fi
        ;;
     ltfs-reformat)
-	ltfs_reformat "${volume_name}"
-	if test $? -gt 0; then
-	    return 1
-	else
 	if ! $quiet; then
 	    printf "${MAGENTA}LTFS Tape reformat: ${GREEN}%s${NO_COLOR}\n" \
 		   "${volume_name}"
 	fi
+	ltfs_reformat "${volume_name}"
+	if test $? -gt 0; then
+	    return 1
 	fi
 	;;
     ltfs-umount)
-	ltfs_umount
-	if test $? -gt 0; then
-	    exit 1
-	else
 	if ! $quiet; then
 	    printf "${MAGENTA}LTFS Tape unmount: ${GREEN}true${NO_COLOR}\n"
 	fi
+	ltfs_umount
+	if test $? -gt 0; then
+	    exit 1
 	fi
 	;;
     media-change)
+	if ! $quiet; then
+	    if  [ ${#volume_name} -ge 1 ]; then
+		printf "${MAGENTA}Media change in pool: ${GREEN}%s${NO_COLOR} (Volume-Name: ${GREEN}%s${NO_COLOR})\n" \
+		       "${mediapool_name}" "${volume_name}"
+	    else
+		printf "${MAGENTA}Media change in pool: ${GREEN}%s${NO_COLOR}\n" \
+		       "${mediapool_name}"
+	    fi
+	fi
 	media_change $mediapool_name $volume_name
 	if test $? -gt 0; then
 	    return 1
-	else
-	    if ! $quiet; then
-		if  [ ${#volume_name} -ge 1 ]; then
-		    printf "${MAGENTA}Media changed in pool: ${GREEN}%s${NO_COLOR} (Volume-Name: ${GREEN}%s${NO_COLOR})\n" \
-			   "${mediapool_name}" "${volume_name}"
-		else
-		    printf "${MAGENTA}Media changed in pool: ${GREEN}%s${NO_COLOR}\n" \
-			   "${mediapool_name}"
-		fi
-	    fi
 	fi
 	;;
     mount)
-        mount_tape "${mediapool_name}" "${volume_name}"
-	if test $? -gt 0; then
-	    exit 1
-	else
 	if ! $quiet; then
 	    if test ${#volume_name} -ge 1; then
-		    printf "${MAGENTA}Tape ${GREEN}'%s'${MAGENTA} for ${GREEN}'%s'${MAGENTA} mounted${NO_COLOR}\n" \
+		printf "${MAGENTA}Mount tape ${GREEN}'%s'${MAGENTA} for ${GREEN}'%s'${NO_COLOR}\n" \
 		       "${volume_name}" "${mediapool_name}"
 	    else
-		    printf "${MAGENTA}Next tape for mediapool ${GREEN}'%s'${MAGENTA} mounted${NO_COLOR}\n" \
+		printf "${MAGENTA}Mount next tape for mediapool ${GREEN}'%s'${NO_COLOR}\n" \
 		       "${mediapool_name}"
 	    fi
 	fi
+	mount_tape "${mediapool_name}" "${volume_name}"
+	if test $? -gt 0; then
+	    exit 1
 	fi
 	;;
     mtx-exchange)

debian/changelog

@@ -1,37 +0,0 @@
-dsnap-sync (0.6.4) unstable; urgency=low
-
-  [Ralf Zerres]
-  * branch: master
-
- -- Ralf Zerres <rzerres@networkx.de>  Thu, 14 Mar 2019 09:20:02 +0100
-
-dsnap-sync (0.6.0) unstable; urgency=low
-
-  [Ralf Zerres]
-  * branch: master
-  * multi-config: handling of multiple configurations as a group
-  * btrfs-archive: full and incremental stream files
-  * tape: handling of LTFS tapes
-  * ionice: optional use in target streams
-
- -- Ralf Zerres <rzerres@networkx.de>  Sat, 21 Sep 2018 20:00:00 +0200
-
-dsnap-sync (0.5.9) unstable; urgency=low
-
-  [Ralf Zerres]
-  * branch: wip-d2d
-  * initial tape-admin wrapper
-  * automounter support for target devices
-  * bugfixes
-  * btrfs quota to read valid snapshot size / change size
-  * color handling in  verbosity-levels
-
- -- Ralf Zerres <rzerres@networkx.de>  Sat, 25 Aug 2018 20:00:00 +0200
-
-dsnap-sync (0.5.3) unstable; urgency=low
-
-  [Ralf Zerres]
-  * initial package
-  * adapt compilation to support ubuntu bionic
-
- -- Ralf Zerres <rzerres@networkx.de>  Fri, 1 Jun 2018 21:00:01 +0000

debian/compat

@@ -1 +0,0 @@
-10

debian/control

@@ -1,25 +0,0 @@
-Source: dsnap-sync
-Section: admin
-Priority: optional
-Maintainer: Networkx GmbH <support@networkx.de>
-Build-Depends: debhelper (>= 10), dh-exec
-Uploaders: Ralf Zerres <ralf.zerres@networkx.de> 
-Standards-Version: 4.1.4.1
-Homepage: https://github.com/rzerres/dsnap-sync.git
-
-Package: dsnap-sync
-Architecture: amd64
-Conflicts: snap-sync
-Depends: dash, snapper, btrfs-progs, systemd
-Suggests: pv, libnotify-bin, mtx, jq, ltfs
-Description: Backup and synchronize btrfs filesystems
- dsnap-sync is designed to backup btrfs formated filesystems. It takes 
- advantage of the specific snapshots functionality btrfs offers and i
- combines it with managemnet functionality of snapper.
- . 
- dsnap-sync creates backups as btrfs-snapshots on a selectable target 
- device. Plug in and mount any btrfs-formatted device to your system. 
- Supported devices may be either local USB drives, but can be as well 
- remote accessible RAID drives. If possible the backup process will send
- incremental snapshots to the target drive. If the snapshot will be 
- stored on a remote host, it is secured with ssh.

debian/dsnap-sync.install

@@ -1,3 +0,0 @@
-bin/
-etc/
-usr/

debian/rules

@@ -1,22 +0,0 @@
-#!/usr/bin/make -f
-# debian/rules
-# -*- mode: makefile; coding: utf-8 -*-
-
-export DH_VERBOSE=1
-#export DESTROOT=$(CURDIR)/debian/dsnap-sync
-export DESTDIR=$(CURDIR)/debian/dsnap-sync
-
-%:
-#	dh $@ --sourcedirectory=src
-	dh $@
-
-#override_dh_auto_configure:
-#	dh_auto_configure --sourcedirectory=src -- --prefix=/usr
-
-override_dh_auto_install:
-	dh_auto_install -- prefix=/usr
-
-#override_dh_install:
-#	dh_install 
-#	dh_missing --fail-missing
-

lang/german/README.md

@@ -1,8 +1,14 @@
 <!-- dsnap-sync README.md -->
-<!-- version: 0.5.9 -->
+<!-- version: 0.6.9 -->
 
 # dsnap-sync
 
+<p align="center">
+  <span>English</span> |
+  <a href="../..">Englisch</a>
+  <!-- a href="../spanish">Spanisch</a> | -->
+</p>
+
 ## Über
 
 `dsnap-sync` ist konzipiert, um Backups für btrfs formatierte Dateisysteme
@@ -184,6 +190,7 @@ Software Paket Manager.
 		  --label-running <desc>  snapper description tagging active jobs. Default: "dsnap-sync in progress"
 		  --label-synced <desc>   snapper description tagging last synced jobs.
 								  Default: "dsnap-sync last incremental"
+          --calculate-btrfs-size  Enable calculation of sync-size for given snapshots
 		  --color                 Enable colored output messages
 	  -c, --config <config>       Specify the snapper configuration to use. Otherwise will perform for each snapper
 								  configuration. You can select multiple configurations
@@ -326,6 +333,66 @@ dann mit den gewohnten Betriebssystem Tools. Eine Open-Source Implementierung
 finden Sie z.B. unter
 [LinearTapeFileSystem](https://github.com/LinearTapeFileSystem/ltfs).
 
+## Rücksicherung (restore)
+
+### Dateisystem (target)
+
+Wurden Daten mit einer nativen btrfs Methode (`btrfs-snapshot`,
+`btrfs-clone`) gesichert, befinden sie sich in einer `snapper` kompatiblen
+Verzeichnisstruktur auf dem gewählten Target.
+
+Diese Struktur könnte so aussehen:
+
+* └ backups
+   * └ @\<server-name\>
+	 * └ \<subvol-name\>
+	   *  └ \<subvol-id\>
+		 * ├ \.snapshot\>
+		 * └ info.xml
+
+Um Daten zurückzusichern, können Sie entweder native `snapper` Methoden verwenden,
+oder auf Betriebssystem Programme zurückgreifen.
+
+  * native snapper Methoden
+
+	Verwenden Sie `snapper diff <id1>..<id2> [Dateien]`, um Unterschiede in den
+	gewählten subvol-id's aufzuzeigen.
+	Verwenden Sie `snapper rollback <id>`, um alle Daten der Sicherung (\<subvol-id\>)
+	an den Ursprungsort zurückzusichern.
+
+  * Betriebssystem Programme
+
+	Verwenden Sie Ihren bevorzugten  Datei-Manager und wechseln Sie in das
+	Verzeichnis, in dem sich die Daten des gewünschten Backups (\<subvol-id\>)
+	befinden. Die Daten wurden als **read-only** Snapshot gesichert.
+	Sie können von hier aus in jedes gewünsche Zielverzeichnis kopiert werden.
+
+### Band-Sicherung (tape)
+
+Wurden Daten mit der Methode `btrfs-archive` gesichert, speichert `dsnap-sync`
+diese in einer `snapper` kompatiblen Datei-Struktur auf dem Band.
+
+Dies Struktur könnte so aussehen:
+
+ * └ backups
+   * └ @\<server-name\>
+	 * └ archive-\<subvol-name\>
+	   *  └ \<subvol-id\>
+		 * ├ \<subvol-id\>_full.btrfs
+		 * └ info.xml
+
+In der Datei `info.xml` werden die Metadaten zum Snapshot gespeichert.
+Die Bewegungsdaten des snapshots befinden sich in der Datei `\<subvol-id\>_full.btrfs`.
+Diese Datei muss mit dem btrfs programm `btrfs-send` auf ein btrfs Verzeichnis zurück
+geschrieben werden:
+
+	cd /target_btrfs_path
+	cp /path_to_tape_root/backups/@<server-name>/archive-<subvol-name>/<subvol-id>_full.btrfs .
+	cat <subvol-id>_full.btrfs  | btrfs receive -v  .
+	rm <subvol-id>_full.btrfs
+
+Die man-page zu `btrfs-send` enthält weiterführende Informationen.
+
 ## Mitarbeit
 
 Hilfe ist sehr willkommen! Gerne könnt Ihr das Projekt forken und PR's einreichen,
@@ -357,4 +424,4 @@ Diese Arbeit ist unter der [Creative Common License 4.0][License-CC_BY] lizensie
 ![Creative Common Logo][Logo-CC_BY]
 
 © 2016, 2017  James W. Barnett;
-© 2017 - 2018 Ralf Zerres
+© 2017 - 2023 Ralf Zerres

usr/share/doc/dsnap-sync/Examples.md

@@ -1,5 +1,25 @@
 # Examples
 
+## Snapper configs cache
+
+To integrate already existing snapshop subfolders located on a given
+filesystem, you need to add the snapper config name to the config
+file. Then restart the snapper service to reread the config list.
+
+```
+[root@dwsbackup dsnap-sync]# cat /etc/conf.d/snapper
+## Path: System/Snapper
+
+## Type:        string
+## Default:     ""
+# List of snapper configurations.
+
+#SNAPPER_CONFIGS="root data.dwssrv1 home.dwssrv1 machines.dwssrv1 root.dwssrv1 libvirt.dwssrv1 data2.dwssrv1 lxd.dwssrv1 archiv2.dwssrv1 archiv3.dwssrv1 root.dwssrv2"
+SNAPPER_CONFIGS="root data.dwssrv1"
+
+[root@dwsbackup dsnap-snyc]# systemctl restart snapperd
+```
+
 ## Automounter
 
 `dsnap-sync` will take advantage of systemd automount units to incorporate external,