dd

最後更新: 2019-11-13

 

目錄

  • gzip dd disk image
  • dd with "0" or "ff"
  • Options - bs
  • bs & count Q&A
  • skip (跳過)
  • Create urandom file
  • 修改 hex file 的某 location
  • 知到要做幾耐(status=progress)
  • 查看 dd 的 status
  • Window 上的 dd
  • IO Sync: direct, dsync, sync
  • Disk IO Test
  • dcfldd

 


gzip dd disk image

 

dd if=/dev/sda1 | gzip > /mnt/backup/sda1.img.gz

 


dd with "0" or "ff"

 

dd if=/dev/zero of=00.bin

dd if=/dev/zero | tr "\000" "\377" > ff.bin

 


Options - bs

 

read and write up to BYTES bytes at a time

# Unit

Default: bs=BYTES

K, M, G         = 1024

kB, MB, GB    = 1000

Speed test with different bs

# Default Setting: 94 MiB/s

dd if=/dev/sda | pv -i 3 > /dev/null

# 8M: 99 MiB/s

dd bs=8M if=/dev/sda | pv -i 3 > /dev/null

# 32M: 98 MiB/s

dd bs=32M if=/dev/sda | pv -i 3 > /dev/null

結論: bs 先速度沒有關係

 


bs & count Q&A

 

請問大家, 以下兩條指令的結果是不是一樣的呢 ??

# cmd_1

dd if=/dev/zero of=~/myfile1 bs=8 count=8

# cmd_2

dd if=/dev/zero of=~/myfile2 bs=64 count=1

相信大家都會答 "是的"

因為它都是會產生一個大小為 64 byte 的 myfile

那我又問, 以下兩指令的結果是不是一樣的呢 ??

# cmd_3

dd if=/dev/random of=~/myfile1 bs=8 count=8

# cmd_4

dd if=/dev/random of=~/myfile2 bs=32 count=2

答案 :

有機會一樣, 亦可能不一樣 = , =", 原因 ??

因為 /dev/random 的 output 是不保證足 64 byte, 所以就有 urandom 了

 


 

skip=N       # skip N ibs-sized blocks at start of input

seek=N      # skip N obs-sized blocks at start of output

bs=BYTES  # 同時設定了 obs-size 及 ibs-size

read and write up to BYTES bytes at a time (default: 512);

ie.

# dd 到一半要 stop 了, 查看 dd 到那裡

dd if=/dev/zero of=/dev/sdX bs=16M

kill -USR1 $(pgrep ^dd)

65743+0 records in
65743+0 records out
1102984511488 bytes (1.1 TB, 1.0 TiB) copied, 4964.41 s, 222 MB/s

# 再開始

1102984511488 / 65743 = 16 M

dd if=/dev/zero of=/dev/sdX bs=16M seek=65743

 


Create urandom file

 

dd if=/dev/urandom of=sample.txt bs=128M count=1

 


以 hex 修改 Binary file 的某位置

 

建立被修改的 test 檔

echo -n 0123456789 > test.txt

修改位置 0x8 的 1 byte

# 用 bs 及 count 去限制影響 (bs = read and write) !!

# bs=1 count=1 => 1 byte

# conv=notrunc => Do not truncate the output file (必須!!), 否則 x08 後的內容會被清空

# seek=8          => 跳過頭 8 個, 修改第 9 個, 由於由 0x00 計起, 所以修改緊 x08

dd if=/dev/urandom of=test.txt bs=1 count=1 conv=notrunc seek=8

查看結果

# 由於是用 urandom 做 source, 有機會係 non-printable 字, 所以要用 xxd / hexdump

xxd test.txt

00000000: 3031 3233 3435 3637 4c39                 01234567L9

當寫的位置比 file size 大時 ( ie. 16 )

echo -n 0123456789 > test.txt

dd if=/dev/urandom of=test.txt bs=1 count=1 conv=notrunc seek=16

xxd test.txt

00000000: 3031 3233 3435 3637 3839 0000 0000 0000  0123456789......
00000010: cf                                       .

當 "bs=x" 時

echo -n 0123456789 > test.txt

dd if=/dev/urandom of=test.txt bs=2 count=1 conv=notrunc seek=2

xxd test.txt

00000000: 3031 3233 a16b 3637 3839                 0123.k6789

修改成指定字符

# \x2a => "*"          # "2a" 係 "*" 的 Hex 值. A = \x41, a = \x61, 0 = \x30

echo -n 0123456789 > test.txt

printf '\x2a\x2a\x2a' | dd of=test.txt bs=1 count=1 seek=4 conv=notrunc

cat test.txt

# 跳過了頭 4 個
0123*56789

 


知到要做幾耐(status=progress)

 

# visual size/percentage progress indicator

status=LEVEL

  • 'none' suppresses everything but error messages,
  • 'noxfer' suppresses the final transfer statistics,
  • 'progress' shows periodic transfer statistics

i.e.

dd status=progress if=/dev/mapper/centos-root of=/dev/null

715182592 bytes (715 MB) copied, 3.006233 s, 238 MB/s

 


查看 dd 的 status

 

[Terminal 1]

dd if=/dev/mapper/centos-root of=/dev/null

[Terminal 2]

kill -USR1 $(pgrep ^dd)

[Terminal 1]

2310657+0 records in
2310657+0 records out
1183056384 bytes (1.2 GB) copied, 1.81717 s, 651 MB/s

 


Window 上的 dd

 

Home Page: http://www.chrysocome.net/dd

# Go to un-zip folder

cd C:\dd-0.5>

# 查看本機上的 Disk

dd --list

\\.\Volume{a4942ba8-deae-11e6-b2c7-e0cb4eeccb4b}\
  link to \\?\Device\HarddiskVolume7
  removeable media
  Mounted on \\.\e:

\\.\Volume{2c9987c2-cf93-11e3-b215-806e6f6e6963}\
  link to \\?\Device\CdRom0
  CD-ROM
  Mounted on \\.\o:

Virtual input devices
 /dev/zero   (null data)
 /dev/random (pseudo-random data)
 -           (standard input)

Virtual output devices
 -           (standard output)

 * dd 只對有 "Mounted on \\.\N:" 的地方作操作

# Clone (Partition0 --> DuinOS.img)

dd of=\\?\Device\Harddisk1\Partition0 if=c:\DuinOS.img bs=1M

# IO test

# wirte

ddrelease64.exe if=/dev/zero of=test.bin bs=1M count=10240 --progress

# read

ddrelease64.exe if=test.bin of=/dev/null bs=1M count=10240 --progress

 * 只能對 Partition 作操作

 


dcfldd

 

Enhanced version of dd for forensics and security

HomePage: http://dcfldd.sourceforge.net/

  • Hashing on-the-fly
    dcfldd can hash the input data as it is being transferred, helping to ensure data integrity.
  • Status output
    dcfldd can update the user of its progress in terms of the amount of data transferred and how much longer operation will take.
  • Flexible disk wipes
    dcfldd can be used to wipe disks quickly and with a known pattern if desired.
  • Image/wipe Verify
    dcfldd can verify that a target drive is a bit-for-bit match of the specified input file or pattern.

 


IO Sync: direct, dsync, sync

 

說明

direct (use direct I/O for data)

Direct I/O open mode O_DIRECT
= bypass kernel page cache
= no extra copies from userland to the kernel
=>  userland buffer -> Disk
=> saving the kernel work
     (no need to perform most buffer cache management operations)

 * 應用: applications do their own caching

 * Due to the lack of caching, sequential reads would have a heavier impact.

 * doesn't guarantee that data of a successful write() call will survive sudden power loss.
    "O_SYNC" 才有此功能

dsync (use synchronized I/O for data)

sync (likewise, but also for metadata)

direct 與 dsync/sync 的分別

dsync/sync 是一次寫一個 bs 大小的資料, 之後同步寫到磁碟後再繼續寫下一個 IO

 => This bypasses all caches including the hardware cache on the drive itself.

Usage

iflag=direct         # this bypasses kernel caching and substantially increases performance.

 


Disk IO Test

 

寫入 HDD

cd /ssd                                              # 去要 test 的 device

dd if=/dev/zero of=test.bin bs=1G count=50G oflag=direct

oflag=dsync    # direct: use direct I/O for data
                     # dsync: use synchronized I/O for data

查看情況

# dstat - versatile tool for generating system resource statistics

dstat -d -r -D sde

--dsk/sde-- ---io/sde--
 read  writ| read  writ
   7B  178k|0.00  0.61
   0   188M|   0   655
   0   177M|   0   622
   0   183M|   0   640
...

 


More

 

 

 

Creative Commons license icon Creative Commons license icon