最後更新: 2017-07-13

介紹

 

dattobd (Datto Block Driver)

The primary intended use case of Dattobd is for backing up live Linux systems.

Dattobd is an open source Linux kernel module for point-in-time live snapshotting

It can be loaded onto a running Linux machine (without a reboot)

After the first snapshot, the driver tracks incremental changes to the block device and

therefore can be used to efficiently update existing backups by copying only the blocks that have changed.

supports most common filesystems

 - ext2,3,4
 - xfs

 

---

Download

 

HomePage:

https://github.com/datto/dattobd

 * 會用 dkms 去 compile ko (dkms-dattobd)

 * 要有 kernel Source

CentOS (Datto's repositories)

yum localinstall https://cpkg.datto.com/datto-rpm/repoconfig/datto-el-rpm-release-$(rpm -E %rhel)-latest.noarch.rpm

yum install dkms-dattobd dattobd-utils

dkms-dattobd Package

 - /etc/kernel/postinst.d/50-dattobd
 - /etc/modules-load.d/dattobd.conf

# load

modprobe dattobd

# checking

cat /proc/datto-info

{
        "version": "0.9.16",
        "devices": [

        ]
}

 

---

general flow

 

take a snapshot, copy it and move the snapshot into 'incremental' mode.

Usage

# Create a snapshot:

# create a snapshot of the root volume at /dev/datto0 with a backing COW file at /.datto
# This file must exist on the volume that will be snapshotted.

dbdctl setup-snapshot /dev/sda1 /.datto 0

# Backup

dd if=/dev/datto0 of=/backups/sda1-bkp bs=1M

# move the snapshot (/dev/datto0) to incremental mode.
# the driver will only track the addresses of blocks that have changed (without the data itself)
# This will remove the associated snapshot device.

dbdctl transition-to-incremental 0

# Move the incremental back to snapshot mode:
# The /.datto file now contains a list of the blocks that have changed since our initial snapshot.
# It is important to not use the same file name that we specified in step 2 for this command.
# Otherwise, we would overwrite our list of changed blocks.

dbdctl transition-to-snapshot /.datto1 0

# Copy the changes:
# 1. a snapshot
# 2. the list of changed blocks
# 3. original backup image

update-img /dev/datto0 /.datto /backups/sda1-bkp

 

---

Driver Status

 

cat /proc/datto-info

minor:

The minor number of the snapshot (for identification purposes).

cow_file:

The path to the cow file relative to the mountpoint of the block device.

block_device:

The block device being tracked by this device.

max_cache: ( Default: 300 Mbyte )

The maximum amount of memory that may be used to cache metadata for this device (in bytes).

seq_id:

The sequence id of the snapshot. This number starts at 1 for new snapshots and is incremented on each transition to snapshot.

"state"

0 = dormant incremental
1 = dormant snapshot
2 = active incremental
3 = active snapshot
4 = unverified incremental
5 = unverified snapshot

 

---

dbdctl

 

dbdctl is the userspace tool used to manage the dattobd kernel module.

-c cache-size     Specify how big the in-memory data cache can grow to (in MB). Defaults to 300 MB.

-f fallocate        Specify the maximum size of the COW file on disk.

destroy

# stop tracking /dev/sda1, remove the associated /dev/datto4
# unlinking the associated COW file.

dbdctl destroy <minor>

reload-incremental

# Reloads a block device that was in incremental mode.

dbdctl reload-incremental [-c <cache size>] <block device> <cow file> <minor>

# This will act the same as reload-snapshot,
# but for a device that was left in incremental mode.

reload-snapshot

# Reloads a snapshot.

# This command is meant to be run before the block device is mounted,
after a reboot or after the driver is unloaded.

It notifies the kernel driver to expect the block device specified to come back online.

This command requires that the snapshot was cleanly unmounted in snapshot mode beforehand.

If this is not the case, the snapshot will be put into the failure state once it attempts to come online.

The minor number will be used as a reference number for all other dbdctl commands.

dbdctl reload-snapshot [-c <cache size>] <block device> <cow file> <minor>

 

---

dattobd Inner Workings

 

When first initialized for a block device, the kernel module creates a COW datastore file on the drive.

The kernel module intercepts all writes at the block level,

and copies the data that is about to be changed into the snapshot store before letting the write complete.

Block: A chunk of contiguous sectors. In our case, 4096 bytes, or 8 sectors.

Section: The basic unit of size that the COW manager works with. 4096 sectors long.

The COW snapshot file can be in one of three states:

    Dormant
    Active
    Unverified

State

Active state

the snapshot device is fully initialized and all in-memory data structures are active. A snapshot will stay in active mode as long as the block device is mounted.

Dormant state

When the block device is unmounted, whether by user action or during normal shutdown of the computer, the block device will transition to "dormant" state. In a nutshell, "dormant" state is a state in which the block device can be safely unmounted by the operating system. When an unmount system call is received for a block device that the dattobd module is tracking, the kernel module first writes all its in-memory data structures for that snapshot device to the COW file for later use, closes all open file descriptors to that file, and shuts down all processing threads for that block device.
(allows snapshots and tracking to persist across unmounts.)

"unverified" state

During the early boot process, the kernel module can be initialized with all of the drives' data structures before the tracked drives themselves are mounted by the OS. This allows the kernel module to immediately start tracking drives as they are mounted during system startup.

mode

In snapshot mode

the on-disk COW file maintains an on-disk block cache.
This cache takes up roughly 10% of the total size of the drive

In incremental mode

only the COW index is kept on disk. This frees up disk space for the rest of the filesystem to use. Writes are tracked in-memory and periodically synced to the index file as needed. During this mode, the driver does not present a readable snapshot device and simply tracks which blocks have changed in the COW index.

Reloading Data During Boot

When the system is coming up during a reboot or power-on event, the dattobd module takes control of the block devices during early boot, in the initramfs stage. The reload process is much the same as the initialization process, with the key difference that the metadata is reloaded from the header instead of being generated.

 

---

Toubleshoot

 

問題1: dbdctl destroy $NUM 見到

[ 6511.219250] datto: device specified is busy: -16
[ 6511.220517] datto: error during transition to incremental ioctl handler: -16

lsof /dev/datto0 見唔到有人用

原因1: mapper 係見唔到的

dmsetup remove "/dev/mapper"

 

---

 

附加檔案	大小
dattobd_scripts.zip	2 KB