最後更新: 2023-05-14

介紹

Backup utility for libvirt, using the changed block tracking features.libvirt

Program: python

Backup job operation via libvirt api

• libvirt module
• libnbd bindings

Homepage: https://github.com/abbbi/virtnbdbackup/

要求: Libvirt & Qemu & Qcow2(v3) 支援

Backup Format

Stream(default)

The backup file consists of metadata about offsets and lengths of zeroed or allocated contents of the virtual machines disk.

Backup Operation

copy, full, inc, diff

copy

no checkpoint is created for further incremental backups, existing checkpoints will be left untouched.

full

a new checkpoint named virtnbdbackup will be created,
all existent checkpoints from prior backups matching this name will be removed

inc

Perform incremental backup, based on the last full or incremental backup.
A checkpoint for each incremental backup is created and saved.

diff

Perform differential backup: saves the current delta to the last incremental or full backup.

 

---

KVM VM 要求

 

qcow images

qcow2 v3     # bitmaps 功能

qcow2 has two versions

v2 (aka compat=0.10)
v3 (aka compat=1.1)

# V2 -> V3

qemu-img convert -p -f qcow2 -O qcow2 -o compat=v3 win7.orig.qcow2 win7.qcow2

qemu-img info win7.qcow2

image: win7.qcow2
file format: qcow2
virtual size: 40 GiB (42949672960 bytes)
disk size: 39.4 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false

P.S.

The compression is read-only. It means that if a compressed sector is rewritten, then it is rewritten as uncompressed data.

get smaller image => Image conversion

Compressed 好處

reading and uncompressing a compressed file is usually faster than just reading uncompressed data.
CPUs are fast and powerful, while disk I/O is slow (even on a PCI-e Gen 4 NVME drive).

 

Libvirt versions <= 7.6.0 (Debian Bullseye, Ubuntu 20.x) KVM Settings

<qemu:capabilities>
   <qemu:add capability='incremental-backup'/>
</qemu:capabilities>

 

---

安裝

 

apt-get install python3-lxml \
python3-lz4 \
python3-paramiko \
python3-tqdm \
python3-typing-extensions \
python3-libnbd libnbd-bin nbdkit nbdkit-plugin-python

wget https://github.com/abbbi/virtnbdbackup/releases/download/v1.9.23/virtnbd...

dpkg -i virtnbdbackup_1.9.23-1_all.deb

 

---

Usage

 

建立儲存 Backup 的 Folder

mkdir /backup/usb-bak-dsk/admin_bak

tree /backup/usb-bak-dsk/admin_bak/

/backup/usb-bak-dsk/admin_bak/
├── admin.cpt
├── ...
├── backup.full.05142023225827.log
├── checkpoints
│   └── virtnbdbackup.0.xml
├── vda.full.data
├── vda.virtnbdbackup.0.qcow.json
└── vmconfig.virtnbdbackup.0.xml

進行 Backup

By default virtnbdbackup will exclude all disks with format raw, passthrough disks (LVM)

virtnbdbackup -d VMname -l full \
    -o /backup/usb-bak-dsk/admin_bak

Exclude Block Device

• exclude certain disks            "-x vdc"
• only backup specific disks     "-i vda"

# list VM block device

virsh domblklist VMname

 Target   Source
------------------------------------------
 vda      /ssd/admin/win7.qcow2
 vdc      /dev/mapper/vg3t-vm_admin_data
 vdd      /home/kvm/admin/swap.raw
 hdc      -

# 只 backup vda

virtnbdbackup -d admin -l full -i vda \
    -o /backup/usb-bak-dsk/admin_bak

Output

... root checkpoint - backup [MainThread]: Saving checkpoint config to: [/backup/usb-bak-dsk/admin_bak/checkpoints/virtnbdbackup.0.xml]
... nbd client - printVersion [vda]: libnbd version: 1.10.5
... nbd client - connect [vda]: Waiting until NBD server at [nbd+unix:///vda?socket=/var/tmp/virtnbdbackup.1111399] is up.
... nbd client - _getBlockInfo [vda]: Using Maximum Block size supported by NBD server: [33554432]
... nbd client - connect [vda]: Connection to NBD backend succeeded.
... root virtnbdbackup - backupDisk [vda]: Got 3098 extents to backup.
... root virtnbdbackup - backupDisk [vda]: 42949672960 bytes disk size
... root virtnbdbackup - backupDisk [vda]: 42346741760 bytes of data extents to backup
... root virtnbdbackup - getWriter [vda]: Write data to target file: [/backup/usb-bak-dsk/admin_bak/vda.full.data.partial].
... root virtnbdbackup - backupDisk [vda]: Creating thin provisioned stream backup image
saving disk vda:   9%|███                             | 4.00G/42.3G [00:09<02:55, 219MB/s]

Backup 後的 Image Info

...
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
                [0]: auto
            name: virtnbdbackup.0
            granularity: 65536

 

Apparmor Error - Failed to bind socket

... [internal error: unable to execute QEMU command 'nbd-server-start':
 Failed to bind socket to /var/tmp/virtnbdbackup.1110600: Permission denied]

修改 apparmor 設定

在以下設定檔加入設定

• /etc/apparmor.d/local/usr.lib.libvirt.virt-aa-helper
• /etc/apparmor.d/local/abstractions/libvirt-qemu
• /etc/apparmor.d/local/usr.sbin.libvirtd

/var/tmp/virtnbdbackup.* rw,
/var/tmp/backup.* rw,

service apparmor reload

 

Estimating backup size

"-p" which will query the virtual machine checkpoint information for the current size:

virtnbdbackup -d admin -l diff -i vda \
    -o /backup/usb-bak-dsk/admin_bak -p

Out

... root virtnbdbackup - main [MainThread]: Estimated checkpoint backup size: [98107392] Bytes

 

Get Backup Info

virtnbdrestore -i /tmp/backupset/ -o dump

 

---

Restore

 

Complete restore

virtnbdrestore -i /tmp/backupset/ -o /tmp/restore

Process only specific disks during restore

virtnbdrestore -i /tmp/backupset/ -o /tmp/restore -d sda

 

Point in time recovery

# "--until" allows to perform a point in time restore up to the desired checkpoint.

virtnbdrestore -i /tmp/backupset/ -o /tmp/restore --until virtnbdbackup.2

 

 

---

Tips

 

Enable compression for the stream format via lz4 algorithm by using the "--compress" option.

 

libvirt 行 VM 時看 qcow2 info

qemu-img info -U win7.qcow2

--force-share (-U)
    If  specified, qemu-img will open the image in shared mode, allowing other QEMU processes to open it in write
    mode. For example, this can be used to get the image information (with 'info' subcommand) when the  image  is
    used  by  a  running guest.  Note that this could produce inconsistent results because of concurrent metadata
    changes, etc. This option is only allowed when opening images in read-only mode.

 

---

技術

 

Dirty Bitmaps

It are in-memory objects that track writes to block devices.

qcow2 supports persistent bitmaps.

The size of the segment that is tracked is the granularity of the bitmap.
If the granularity of a bitmap is 64K, each "1" bit means that a 64K region
 as a whole may have changed in some way, possibly by as little as one byte.

CLI: qemu-img bitmap ...

• --add to create BITMAP, enabled to record future edits.
• --remove to remove BITMAP.
• --clear to clear BITMAP.
• --enable to change BITMAP to start recording future edits.
• --disable to change BITMAP to stop recording future edits.
• --merge to merge the contents of the SOURCE bitmap into BITMAP.

Bitmaps that are frozen cannot be deleted.

Deleting the bitmap does not impact any other bitmaps attached to the same node, nor does it affect any backups already created from this node.