urbackup

最後更新: 2017-07-12

介紹

 

Home Page: https://www.urbackup.org/

 * incremental image and file backups
 * Conistent backups of used files on Windows and Linux.
 * file level deduplication
 * Webinterface that shows the status of the clients
 * restore via restore CD/USB stick)

目錄

  • Install
  • Linux - Client

 


Install

 

# Ubuntu

Install via
PPA: http://launchpad.net/~uroni/+archive/urbackup

e.g.
sudo add-apt-repository ppa:uroni/urbackup
sudo apt update
sudo apt install urbackup-server

# CentOS 7

cd /etc/yum.repos.d/
wget http://download.opensuse.org/repositories/home:uroni/CentOS_7/home:uroni...
yum install urbackup-server

# Configure

# Debian - Configure File: /etc/default/urbackupsrv

# Centos - /etc/sysconfig/urbackup-server

# Firewall

Web: 55414
FastCGI: 55413
internet mode: 55415

firewall-cmd --permanent \
--add-rich-rule='rule family="ipv4" source address="192.168.88.0/24" port protocol="tcp" port="55414" accept'

firewall-cmd --permanent \
--add-rich-rule='rule family="ipv4" source address="192.168.88.0/24" port protocol="tcp" port="55415" accept'

firewall-cmd --reload

# Storage

mkdir /mnt/urbackup_data

chown urbackup. /mnt/urbackup_data

chmod 770 /mnt/urbackup_data

# Service

systemctl restart urbackup-server

# checking

lsof -i tcp
 


Linux - Client

 

 

UrBackupClientLinux2.1.16.sh <-- Makeself

urbackupclientctl set-settings \
 -k internet_mode_enabled -v true \
 -k internet_server -v 192.168.88.114 -k internet_server_port -v 55415 \
 -k computername -v "s08r2" -k internet_authkey -v ????

INTERNET_ONLY

[ ! -e /etc/default/urbackupclient ] || sed -i 's/INTERNET_ONLY=false/INTERNET_ONLY=true/' /etc/default/urbackupclient

[ ! -e /etc/sysconfig/urbackupclient ] || sed -i 's/INTERNET_ONLY=false/INTERNET_ONLY=true/' /etc/sysconfig/urbackupclient

# Centos 7

systemctl restart urbackupclientbackend

# Checking

lsof -i tcp | grep urbackup

urbackupc  2337 root    9u  IPv4  33941      0t0  TCP *:35623 (LISTEN)
urbackupc  2337 root   10u  IPv4  46734      0t0  TCP *:35621 (LISTEN)

# Config Path

/usr/local/var/urbackup

 - backup_client.db               # SQLite 3.x database

 - pw.txt

 - pw_change.txt

...

/usr/local/var/urbackup/data

 - settings.cfg

/usr/local/etc/urbackup

# snapshot cfg

dattobd

/usr/local/etc/urbackup/snapshot.cfg

create_filesystem_snapshot=/usr/local/share/urbackup/dattobd_create_filesystem_snapshot
remove_filesystem_snapshot=/usr/local/share/urbackup/dattobd_remove_filesystem_snapshot

no_filesystem_snapshot

/usr/local/etc/urbackup/no_filesystem_snapshot

# reinstall

rm -rf /usr/local/var/urbackup /usr/local/etc/urbackup

# Firewall

 * all incoming

  • 35621/TCP     Sending files during file backups (file server)
  • 35622/UDP     broadcasts for discovery
  • 35623/TCP     Commands and image backups

# Checking

netstat -nlp | grep urbackup

tcp        0      0 0.0.0.0:35621               0.0.0.0:*                   LISTEN      2828/urbackupclient
tcp        0      0 0.0.0.0:35623               0.0.0.0:*                   LISTEN      2828/urbackupclient
udp        0      0 0.0.0.0:35622               0.0.0.0:*                               2828/urbackupclient

# 當 "INTERNET_ONLY=true" 時

netstat -nlp | grep urbackup

tcp        0      0 127.0.0.1:35623             0.0.0.0:*                   LISTEN      2871/urbackupclient

 


Client Discovery

 

same subnet

If server and clients are in the same subnet the server will automatically discover the clients and then start backing them up

1. The UrBackup server broadcasts a UDP message every 50 seconds

2. On receiving such a broadcast message the client answers back with its fully qualified domain name.

different subnet

"add new client" on the status page

Discover new client via IP/hostname hint

The server will then additionally send an UDP message directly to that entered IP or

resolved host name allowing switches to forward the message across subnet boundaries.

The core client process listens on port 35622 UDP for UDP broadcast messages from the server and on receiving one sends a message with its name back to the server.

It listens on port 35623 TCP for commands from the client interface process
and the server

35621/TCP for file requests from the server.

he core client process is responsible for building a list of all files in the directories to be backed up. (filelist.ub)

The server downloads the file list from the client and starts the backup by downloading changed or new files from the build in client file server. The image backup is done using only the command port.

To speed up the directory list creation directories to be backed up are constantly watched via the Windows Change Journal. The Windows Change Journal can only be used for whole partitions. Thus the first time a directory on a volume is added the UrBackup core client process reads all the directory entries on the new volume into the client database file in ’urbackup/backup_client.db’. After a volume is successfully indexed the database is constantly updated to be in sync with the file system. The updating is done every 10 seconds or if a file list is requested.

 


Client security

 

UrBackup Client only answers commands if the server or the interface process supply it with credentials. The server credential is saved in ’/var/ lib/ urbackup/ server_ident.key’. If it does not exist the server will randomly generate it the first time it runs. The server identity is also confirmed by private/public key authentication. If not present the server will generate a private and public ECDSA key in ’server_ident_ecdsa409k1.priv’ and ’server_ident_ecdsa409k1.pub’.

pw.txt:

    Getting the current status
    Get the paths which are backed up during file backups
    Get the incremental file backup interval
    Start backups
    Pause backups

pw_change.txt

    Change the paths which are backed up during file backups
    Get all settings
    Change all settings
    Get log entries/logs
    Accept a new server

If you want to manually add a server to ’server_idents.txt’ you need to remove the preceding ’#I’ and ’#’ at the end of the contents of ’server_ident.key’. After installation the ’server_idents.txt’ does not exist and the client core process accepts(and adds) the first server it sees (with the public key of the server). After that no other servers with different credentials are accepted and you need to add their credentials either manually, or via clicking on the popup box, once the client has detected the new server. This prevents others from accessing files you want to be backed up in public places.
If you want to have several servers to be able to do backups of a client you have two options. Either you manually supply the server credentials to the client (by copying them into ’server_idents.txt’) or you give all servers the same credentials by copying the same ’server_ident.key’, ’server_ident_ecdsa409k1.p riv’ and ’server_ident_ecdsa409k1.pub’ to all servers.

 


Per Client Setting

 

The settings configured on the client will overwrite the settings configured here.

If you want to change this behaviour do not allow the client to change settings.

General -> Permission

 * Allow client-side changing of settings

 


Transfer security

 

The transfer of data from client to server is unencrypted on the local network

Internet mode:

The data is encrypted and authenticated using AES-GCM authentication via server identity key and

ECDSA private/public key authentication is done.

 


File system - Btrfs

 

支援 block-level deduplication on incremental file backups.

If UrBackup detects a btrfs file system it uses a special snaphotting file backup mode.

It saves every file backup of every client in a separate btrfs sub-volume.

When creating an incremental file backup UrBackup then creates a snapshot of the last file backup and removes, adds and changes only the files required to update the snapshot. This is much faster than the normal method, where UrBackup links (hard link) every file in the new incremental file backups to the file in the last one. It also uses less metadata (information about files, i.e., directory entries). If a new/changed file is detected as the same as a file of another client or the same as in another backup, UrBackup uses cross device reflinks to save the data in this file only once on the file system.

Using btrfs also allows UrBackup to backup files changed between incremental backups in a way that only changed data in the file is stored. This greatly decreases the storage amount needed for backups, especially for large database files (such as e.g. the Outlook archive file).

 

urbackup_ snapshot_helper

In order to create and remove btrfs snapshots UrBackup installs a setuid executable urbackup_ snapshot_helper

enable btrfs mode

mkdir /etc/urbackup

echo "/mnt/urbackup_data" > /etc/urbackup/backupfolder

# test

urbackup_snapshot_helper test

Testing for btrfs...
Create subvolume '/mnt/urbackup_data/testA54hj5luZtlorr494/A'
Create a snapshot of '/mnt/urbackup_data/testA54hj5luZtlorr494/A' in '/mnt/urbackup_data/testA54hj5luZtlorr494/B'
Delete subvolume (commit): '/mnt/urbackup_data/testA54hj5luZtlorr494/A'
Delete subvolume (commit): '/mnt/urbackup_data/testA54hj5luZtlorr494/B'
BTRFS TEST OK

btrfs sub list .

ID 270 gen 123 top level 5 path s08r2/170712-1012

Global soft file system quota

You should set a generously low soft file system quota if using btrfs,

because btrfs currently still has issues in out-of-space situations and may require manual intervention.

 

 


Internet clients

 

All connections are from client to server.

Enable By GUI

Setting -> General -> Internet

internet mode: 1
Internet server port: 55415

Internet client setting

echo "internet_server=example.com
internet_server_port=55415
internet_authkey=foobar
internet_mode_enabled=true" > /usr/local/var/urbackup/data/settings.cfg

 


Cleanup time window

 

During cleanups UrBackup will look at the used space of the file system the backup folder is on.

If the used space is higher than the global soft file system quota UrBackup will delete old backups if possible,

till the used space is below the quota.

This is when old backups and clients are deleted. Default: 1-7/3-4

Then you can stop the server run the cleanup separately by calling

urbackupsrv cleanup --amount x

Where x is the percent of space to free on the backup storage

e.g. "20G" or "10%"

If it should only delete old backups use "0%"

 


time window

 

1-7/3-4        on each day (1-Monday - 7-Sunday) between 3 am and 4 am.

 


Setting

 

General -> Server

Automatically backup UrBackup database:

If checked UrBackup will save a backup of its internal database in a subdirectory called ’urbackup’

in the backup storage path. This backup is done daily within the clean up time window.

i.e.

/mnt/urbackup_data/urbackup

 


urbackupsrv

 

DOC

man urbackup-server

verify-hashes

Verify file backup hashes

remove-unknown

Remove unknown files and directories from backup storage and fix symbolic links in backup storage

reset-pw

Reset web interface administrator password     

defrag-database

Rebuild UrBackup database

decompress-file

Decompress UrBackup compressed file

mount-vhd

Mount VHD file

assemble

Assemble VHD(Z) volumes into one disk VHD file

 


Transfer modes

 

raw

hashed

You do not need to use the hashed transfer mode if you backup via a Internet mode connection with enabled encryption, as the encryption layer already protects the integrity of the transmitted data.

Block differences

CRC32 and MD5 hash functions.

 


urbackupclientctl

 

start                # Start an incremental/full image/file backup

-i,  --incremental     # Start incremental backup
-f,  --full                 # Start full backup
-b,  --non-blocking  # Do not show backup progress and block till the backup is finished but return immediately after starting it

status              # Get current backup status

{
"capability_bits": 4096,
"finished_processes": [],
"internet_connected": true,
"internet_status": "connected",
"last_backup_time": 0,
"running_processes": [{
"action": "FULL",
"eta_ms": -1,
"percent_done": -1,
"process_id": 1,
"server_status_id": 19,
"speed_bpms": 0
}
],
"time_since_last_lan_connection": 73383823
}

connection 建立後

"servers": [{
"internet_connection": true,
"name": "192.168.88.114"
}
],

set-settings         # Set backup settings

browse                # Browse backups and files/folders in backups

reset-keep            # Reset keeping files during incremental backups

Backup Path

list-backupdirs                # List directories that are being backed up

add-backupdir                # Add new directory to backup set

add-backupdir -d /etc

-d <path>,  --path <path>       # (required)  Backup path
-n <name>,  --name <name>  #  Backup directory name

-s,  --require-snapshot             # Fail backup if snapshot of backup path cannot be created
-x,  --one-filesystem                # Do not cross filesystem boundary during backup
-f,  --no-follow-symlinks           # Do not follow symbolic links outside of backup path
-r,  --require-symlinks              # Fail backup if symbolic link targets do not exist

-o,  --optional                           # Do not fail backup if path does not exist

-k,  --keep                               # Keep deleted files and directories during incremental backups

remove-backupdir             # Remove directory from backup set

 

 


Excluded files

 

# File

*.mp3;*.avi;*.mkv;*.mp4;*.mpg;*.mpeg

# Directory

*/Temp/*

* separated by a semicolon (";")
* Use a backslash for Windows clients ("\") and
  a forward slash for Linux clients ("/") to separate folders.

 


Pre and Post backup scripts on client

 

On Linux the clients pre and post backups scripts are searched for

/etc/urbackup/

/usr/local/etc/urbackup/

# Check Script

mkdir /etc/urbackup

touch /etc/urbackup/prefilebackup

touch /etc/urbackup/postfilebackup

chmod +x /etc/urbackup/*

prefilebackup

Called before a file backup (before snapshot/shadowcopy creation).

* On failure (return code not zero)
 * Indexing fails and backup is not started

postfilebackup

Called if a file backup successfully finished

Other

- preimagebackup

- postimagebackup

 


Browse Backup

 

# show all backup ("id" 大的在頂)

urbackupclientctl browse

[{
"archived": 0,
"backuptime": 1500540095,
"id": 56,
"incremental": 0,
"size_bytes": 203162
}
,
.........
]

# show 某一 backup

-b <id>,  --backupid <id>

ie.

urbackupclientctl browse -b 56

output

[{
"access": 1500539740,
"creat": 0,
"dir": true,
"mod": 1500539730,
"name": "etc"
}
,{
"access": 1500540095,
"creat": 0,
"dir": true,
"mod": 1316778468,
"name": "home"
}
,{
"access": 1500539633,
"creat": 0,
"dir": true,
"mod": 1407743962,
"name": "root"
}
]

 


Restore backup

 

 * 不會問 yes/no 就開始

-b <id>,  --backupid <id>
     (required)  Backupid of backup from which to restore files/folders or
     "last" for last complete backup
    
-n,  --no-remove
     Do not remove files/directories not in backup
    
-d <path>,  --path <path>
     Path of folder/file to restore

# restore 去其他地方

# Map to local output path of folders/files to a different local path

-t <path>,  --map-to <path>  (accepted multiple times)

# Map from local output path of folders/files to a different local path

-m <path>,  --map-from <path>  (accepted multiple times)

i.e.

urbackupclientctl restore-start -b 28 -d /root -m / -t /mnt/restore

 


Archiving

 

 * UrBackup has the ability to automatically archive file backups.

 * Archived file backups cannot be deleted by the nightly or emergency clean up

Archival window

Hour;Day of month;Month;Day of week

# To archive a file backup on the first Friday of every month

*;*;*;5

# To archive a backup on the first of every month

*;1;*;*

 


Doc

 

https://www.urbackup.org/administration_manual.html

 

 

Creative Commons license icon Creative Commons license icon