rsnapshot

由 datahunter 在五, 24/02/2012 - 23:34 發表

最後更新: 2022-10-18

介紹

所有被備份的目錄及 LV 要以 /Folder/ 形式表示,

即一定要有 "/" 作尾!!

參數只能用 tab 分隔, 不可以用空格 !!

Home Page: http://www.rsnapshot.org/

安裝

Rocky 8

yum install rsnapshot

人手安裝

Download: http://www.rsnapshot.org/downloads/rsnapshot-1.3.1.tar.gz

./configure --prefix=/usr --sysconfdir=/etc

# --prefix=PREFIX

# --sysconfdir=DIR

make install

which rsnapshot

/usr/bin/rsnapshot

mv /etc/rsnapshot.conf.default /etc/rsnapshot.conf

設定

rsnapshot.conf

# 這兩個設定是為了在 USB Disk 的. USB Disk mount mount 到 /backup
# 當 "/backup/backup/" 不存在時, 那就不會行 backup (no_create_root)
# Backup 時會建立 ".sync" folder
snapshot_root   /backup/backup/
no_create_root  1

# 當 daily 不是 7 時, weekly 要少心行 !!
interval        daily   7        <--- 它定義了1個week有幾多個day !!! 最後一個 daily.n 會成為 weekly.0
interval        weekly  4        <--- 當沒有最後的 daily.n 時, 如果照行 rsnapshot weekly, 那 weekly.0 消失 !!

# 排除一些非必要的東西
exclude         _X
exclude         _bak/
exclude         _tmp/
exclude         logs/*log*
exclude         .vscode-server
exclude         .cache

# delete just before rsnapshot exits. The default is 0 (off)
use_lazy_deletes    1

說明

use_lazy_deletes

The oldest directory(alpha.6) -> [interval_name].delete -> delete just before rsnapshot exits

Enabling this means that snapshots get taken sooner (since the delete doesn't come first)

.. /usr/bin/rsnapshot alpha: started
.. echo 301419 > /var/run/rsnapshot.pid
.. mv /backup/alpha.6/ /backup/_delete.301419/
..
.. /usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded ...
.. touch /backup/alpha.0/
.. rm -f /var/run/rsnapshot.pid
.. /usr/bin/rm -rf /backup/_delete.301419
.. /usr/bin/rsnapshot alpha: completed successfully

設定要備份什麼

# Local Folder Backup
backup   /vz/private/                        vps_backup/

執行 "rsnapshot sync" 會有以下情況:

Local Folder Backup:

/vz/private/  --->  /backup/.sync/vps_backup/vz/private   //  放到目錄內

檢查設定

rsnapshot configtest

Syntax OK

Backup By CLI

# 新版使用了 alpha, beta, gamma, delta 取代 hourly, daily, weekly, monthly

rsnapshot hourly | daily | weekly | monthly

當有設定 "sync_first 1" 時要

rsnapshot sync && rsnapshot alpha

沒有用sync_first的情況

log

/usr/bin/cp -al /backup/alpha.0 /backup/alpha.1
/usr/bin/rsync -a --delete --numeric-ids --relative \
  --delete-excluded \
  /etc/ /backup/alpha.0/localhost/

使用sync_first的情況

log

rsnapshot sync

/usr/bin/rsync -a --delete --numeric-ids --relative \
  --delete-excluded --exclude=logs/*.log \
  /etc/ /backup/.sync/localhost/

rsnapshot alpha

..

LVM Backup 原理

Config

# LVM Backup Configure #
linux_lvm_snapshotsize  30G
linux_lvm_snapshotname  rsnapshot
linux_lvm_vgpath        /dev

# 這 folder 一定要人手先建立它
linux_lvm_mountpath     /mnt/rsnapshot

# 它們的 "#" 要拿走
# 要測試 CLI 是否存在. ls -l /usr/sbin/{lvcreate,lvremove}
linux_lvm_cmd_lvcreate  /usr/sbin/lvcreate
linux_lvm_cmd_lvremove  /usr/sbin/lvremove
linux_lvm_cmd_mount     /usr/bin/mount
linux_lvm_cmd_umount    /usr/bin/umount

# Backup
backup  /home/data/     localhost/
backup  lvm://vg1/lv1/  lv1/

"lvm://vg1/lv1" 會使用 "linux_lvm_vgpath", 所以 "vg1/lv1" 對應了 /dev/vg1/lv1

Notes

* lvm://path/to/lv/ 最尾的 "/" 必須加上

/usr/bin/rsnapshot sync: ERROR: Could not understand LVM source "lvm://vg1/lv1" in linux_lvm_parseurl()

* "lv1/" 必須指定 Folder, 不可以使用 "localhost/".
否則會影響其他 Folder backup (e.g. /home/data/)

... chdir(/mnt/rsnapshot)
... /usr/bin/rsync -a --delete ... --exclude=_bak ./ /backup/BackupData/.sync/localhost/

* linux_lvm_snapshotsize 的 unit 有 M, G

* linux_lvm_cmd_* 的設定

ls -l /usr/sbin/{lvcreate,lvremove}

lrwxrwxrwx. 1 root root 3 Feb  3 14:05 /usr/sbin/lvcreate -> lvm
lrwxrwxrwx. 1 root root 3 Feb  3 14:05 /usr/sbin/lvremove -> lvm

ls -l /sbin/{lvcreate,lvremove}

lrwxrwxrwx. 1 root root 3 Feb  3 14:05 /sbin/lvcreate -> lvm
lrwxrwxrwx. 1 root root 3 Feb  3 14:05 /sbin/lvremove -> lvm

stat /{,usr}/sbin/lvm | grep -e File -e Inode

  File: //sbin/lvm
Device: fd03h/64771d    Inode: 51190255    Links: 1
  File: /usr/sbin/lvm
Device: fd03h/64771d    Inode: 51190255    Links: 1

rsnapshot rsync 相當於

1. 建立 snapshot

/sbin/lvcreate --snapshot --size 30G --name rsnapshot /dev/lxc/vps_mail

2. mount snapshot 到 "linux_lvm_mountpath"

/bin/mount /dev/lxc/rsnapshot /mnt/rsnapshot

3. 之後 rsync data

chdir(/mnt/rsnapshot/)

/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded . /backup/.sync/vps/

相當於用 "rsync_long_args" 去 rsync

rsync_long_args --delete --numeric-ids --relative --delete-excluded

LVM 與 XFS

mount.sh

#! /bin/bash
# This is a wrapper for mount to pass it the nouuid option.
# This is required to enable rsnapshot to mount an xfs lvm snapshot.

if [ $# -ne 2 ]; then echo "require 2 args" && exit 1; fi

/bin/mount -o nouuid,ro "$1" "$2"

Daily Backup 真正佔用的空間(du)

rsnapshot du

70G     /backup/.sync
199M    /backup/daily.0/          # 與 .sync 有多少不同
1.9G    /backup/daily.1/
2.6G    /backup/daily.2/
2.5G    /backup/daily.3/
2.3G    /backup/daily.4/
2.7G    /backup/daily.5/
2.2G    /backup/daily.6/
3.1G    /backup/daily.7/
1.8G    /backup/daily.8/
89G     total

找出兩天 Backup 的不同之處 (可以用來 verify backkup)

rsnapshot diff daily.4 daily.0 # daily.4 daily.0 不用分次序, 因為一定係新(數字細)比舊(數字大)

Comparing daily.4 to daily.0
Between daily.4 and daily.0:
  335841 were added, taking 53300465177 bytes;  # 在新 backup 才出現
  3 were removed, saving 179750 bytes;          # 在新 backup 不見了

Useful Options

-v verbose # 看到多了或小了什麼 File

rsnapshot -v diff daily.0 daily.1 | grep ^'-'

+ /mnt/backup/daily.0/localhost/data/backuptime.txt
- /mnt/backup/daily.1/localhost/data/backuptime.txt
...

To check the differences between two directories

rsnapshot -v diff daily.0/localhost/etc daily.1/localhost/etc

rsnapshot-diff -vi /mnt/backup/daily.0/localhost/etc \
    /mnt/backup/daily.1/localhost/etc

Comparing /mnt/backup/daily.1/localhost/etc to /mnt/backup/daily.0/localhost/etc
+ /mnt/backup/daily.0/localhost/etc/shadow-
+ /mnt/backup/daily.0/localhost/etc/backuptime.txt
+ /mnt/backup/daily.0/localhost/etc/adjtime
+ /mnt/backup/daily.0/localhost/etc/shadow
- /mnt/backup/daily.1/localhost/etc/shadow-
- /mnt/backup/daily.1/localhost/etc/backuptime.txt
- /mnt/backup/daily.1/localhost/etc/adjtime
- /mnt/backup/daily.1/localhost/etc/shadow
+ /mnt/backup/daily.0/localhost/etc/blkid/blkid.tab.old
+ /mnt/backup/daily.0/localhost/etc/blkid/blkid.tab
- /mnt/backup/daily.1/localhost/etc/blkid/blkid.tab
Between /mnt/backup/daily.1/localhost/etc and /mnt/backup/daily.0/localhost/etc:
  6 were added, taking 444257 bytes;
  5 were removed, saving 443496 bytes;

Other Options

-i ignore symlinks, directories, and special files in verbose output

Backup remote data (rsync 方式)

Server 設定

/etc/hosts

192.168.88.176  c1

cmd:

mkdir /etc/bak_client_pw

chmod 700 /etc/bak_client_pw

echo "YOUR_PW" > /etc/bak_client_pw/c1

chmod 400 /etc/bak_client_pw/c1

rsnapshot.conf

把 remote 要 backup 的東西以 "rsync over ssh" 方式拉過來

# ssh Backup Server 的設定

backup   root@client1:/data/      client1/

# modules 的設定

backup   rsync://client2/data/    client2/

* 注意尾要有 "/"

rsync 有 Password 保護的情況

# + sign usage <-- default replacing args

Usage:

... +rsync_long_args=--exclude=/var/spool/ ...

e.g.

backup rsync://backup@c1/home/ c1/home/   +rsync_long_args=--password-file=/etc/bak_client_pw/c1

backup 後的目錄結構

root@ubuntu:/backup# tree .sync

.sync
└── c1
    └── home
        └── tester

心得(tips)

心得1: 沒有足夠空間的情況

在以下設定時:

retain          hourly  6
retain          daily   7

當沒有足夠空間做 hourly backup 時, 那行 daily backup 就會出錯

# 由於 backup 係由 hourly.0 數起, 所以 hourly.5 = 6

... /backup/hourly.5 not present (yet), nothing to copy
... rm -f /var/run/rsnapshot.pid

心得2: link_dest

rsync 的 "--link-dest" 功能

hardlink to files in DIR when unchanged

# best way to support special files (FIFOs, etc)
# The default is 0 (off)
link-dest    1

Cron Job

/root/scripts/bakscript.sh

#!/bin/bash
# Version: 1

#### Setting
Disk=/dev/sdb1
BakDiskMountPoint=/mnt/backup
Admin=admin@domain
Server=Full-Domain-Name

#### Code
ChkFile=$BakDiskMountPoint/usbdisk.txt

function notify2admin(){
        if [ $? -ne 0 ]; then
                echo "mount disk fail"
                echo "$Server" | mail -r root@$Server -s "Backup Fail" $Admin
                exit 1
        fi
}

if [ ! -f $ChkFile ]; then
        echo "mount backup disk"
        mount -o noatime $Disk $BakDiskMountPoint
        notify2admin
fi

echo "Start Backup - `date`"
/usr/bin/rsnapshot daily
notify2admin
echo "Done         - `date`"

Troubleshoot

[Q1] 行 rsnapshot sync 見到

rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot daily
----------------------------------------------------------------------------
ERROR: rsync_cleanup_after_native_cp_al() only works on directories
ERROR: Error! cp_al("/backup/backup/.sync", "/backup/backup/daily.0")

[A1]解決

cmd_cp          /bin/cp

cmd_cp 解說

If you are using Linux, you should uncomment cmd_cp

With GNU cp, rsnapshot can take care of both normal files and special files

(such as FIFOs, sockets, and block/character devices) in one pass

If cmd_cp is disabled, rsnapshot will use its own built-in function, native_cp_al() to backup up regular files and directories.

This will then be followed up by a separate call to rsync, to move the special files over (assuming there are any).

常用 exclude

# Apache Logs
exclude         logs/*.log
exclude         logs/*.gz
# For docker
exclude         logs/httpd/*
exclude         logs/php-fpm/*
# For DEV
exclude         .vscode-server
# For Backup
exclude         _bak
exclude         _X