![首頁 Logo Keith ]p !!](/themes/mytheme/logo.png){kind=logo}
最後更新: 2019-11-13
目錄
- gzip dd disk image
- dd with "0" or "ff"
- Options - bs
- bs & count Q&A
- skip & seek & bs
- Create urandom file
- 修改 hex file 的某 location
- 知到要做幾耐(status=progress)
- 查看 dd 的 status
- Window 上的 dd
- IO Sync: direct, dsync, sync
- Disk IO Test
- dcfldd
gzip dd disk image
dd if=/dev/sda1 | gzip > /mnt/backup/sda1.img.gz
dd with "0" or "ff"
dd if=/dev/zero of=00.bin
dd if=/dev/zero | tr "\000" "\377" > ff.bin
Options - bs
read and write up to BYTES bytes at a time
# Unit
Default: bs=BYTES
K, M, G = 1024
kB, MB, GB = 1000
Speed test with different bs
# Default Setting: 94 MiB/s
dd if=/dev/sda | pv -i 3 > /dev/null
# 8M: 99 MiB/s
dd bs=8M if=/dev/sda | pv -i 3 > /dev/null
# 32M: 98 MiB/s
dd bs=32M if=/dev/sda | pv -i 3 > /dev/null
結論: bs 先速度沒有關係
bs & count Q&A
請問大家, 以下兩條指令的結果是不是一樣的呢 ??
# cmd_1
dd if=/dev/zero of=~/myfile1 bs=8 count=8
# cmd_2
dd if=/dev/zero of=~/myfile2 bs=64 count=1
相信大家都會答 "是的"
因為它都是會產生一個大小為 64 byte 的 myfile
那我又問, 以下兩指令的結果是不是一樣的呢 ??
# cmd_3
dd if=/dev/random of=~/myfile1 bs=8 count=8
# cmd_4
dd if=/dev/random of=~/myfile2 bs=32 count=2
答案 :
有機會一樣, 亦可能不一樣 = , =", 原因 ??
因為 /dev/random 的 output 是不保證足 64 byte, 所以就有 urandom 了
skip & seek & bs
skip=N # Skip N ibs-sized blocks at start of input
seek=N # Skip N obs-sized blocks at start of output
bs=BYTES # 同時設定了 obs-size 及 ibs-size (default: 512)
# read and write up to BYTES bytes at a time
e.g.
應用1: dd 到一半要 stop 了, 查看 dd 到那裡
dd if=/dev/zero of=/dev/sdX bs=16M
kill -USR1 $(pgrep ^dd)
65743+0 records in 65743+0 records out 1102984511488 bytes (1.1 TB, 1.0 TiB) copied, 4964.41 s, 222 MB/s
# 再開始
1102984511488 / 65743 = 16M # bs size
dd if=/dev/zero of=/dev/sdX bs=16M seek=65743
Create urandom file
dd if=/dev/urandom of=sample.txt bs=128M count=1
以 hex 修改 Binary file 的某位置
建立被修改的 test 檔
echo -n 0123456789 > test.txt
修改位置 0x8 的 1 byte
# 用 bs 及 count 去限制影響 (bs = read and write) !!
# bs=1 count=1 => 1 byte
# conv=notrunc => Do not truncate the output file (必須!!), 否則 x08 後的內容會被清空
# seek=8 => 跳過頭 8 個, 修改第 9 個, 由於由 0x00 計起, 所以修改緊 x08
dd if=/dev/urandom of=test.txt bs=1 count=1 conv=notrunc seek=8
查看結果
# 由於是用 urandom 做 source, 有機會係 non-printable 字, 所以要用 xxd / hexdump
xxd test.txt
00000000: 3031 3233 3435 3637 4c39 01234567L9
當寫的位置比 file size 大時 ( ie. 16 )
echo -n 0123456789 > test.txt
dd if=/dev/urandom of=test.txt bs=1 count=1 conv=notrunc seek=16
xxd test.txt
00000000: 3031 3233 3435 3637 3839 0000 0000 0000 0123456789......
00000010: cf .當 "bs=x" 時
echo -n 0123456789 > test.txt
dd if=/dev/urandom of=test.txt bs=2 count=1 conv=notrunc seek=2
xxd test.txt
00000000: 3031 3233 a16b 3637 3839 0123.k6789
修改成指定字符
# \x2a => "*" # "2a" 係 "*" 的 Hex 值. A = \x41, a = \x61, 0 = \x30
echo -n 0123456789 > test.txt
printf '\x2a\x2a\x2a' | dd of=test.txt bs=1 count=1 seek=4 conv=notrunc
cat test.txt
# 跳過了頭 4 個 0123*56789
知到要做幾耐(status=progress)
# visual size/percentage progress indicator
status=LEVEL
- 'none' suppresses everything but error messages,
- 'noxfer' suppresses the final transfer statistics,
- 'progress' shows periodic transfer statistics
i.e.
dd status=progress if=/dev/mapper/centos-root of=/dev/null
715182592 bytes (715 MB) copied, 3.006233 s, 238 MB/s
查看 dd 的 status
[Terminal 1]
dd if=/dev/mapper/centos-root of=/dev/null
[Terminal 2]
kill -USR1 $(pgrep ^dd)
[Terminal 1]
2310657+0 records in 2310657+0 records out 1183056384 bytes (1.2 GB) copied, 1.81717 s, 651 MB/s
Window 上的 dd
Home Page: http://www.chrysocome.net/dd
# Go to un-zip folder
cd C:\dd-0.5>
# 查看本機上的 Disk
dd --list
\\.\Volume{a4942ba8-deae-11e6-b2c7-e0cb4eeccb4b}\
link to \\?\Device\HarddiskVolume7
removeable media
Mounted on \\.\e:
\\.\Volume{2c9987c2-cf93-11e3-b215-806e6f6e6963}\
link to \\?\Device\CdRom0
CD-ROM
Mounted on \\.\o:
Virtual input devices
/dev/zero (null data)
/dev/random (pseudo-random data)
- (standard input)
Virtual output devices
- (standard output)* dd 只對有 "Mounted on \\.\N:" 的地方作操作
# Clone (Partition0 --> DuinOS.img)
dd of=\\?\Device\Harddisk1\Partition0 if=c:\DuinOS.img bs=1M
# IO test
# wirte
ddrelease64.exe if=/dev/zero of=test.bin bs=1M count=10240 --progress
# read
ddrelease64.exe if=test.bin of=/dev/null bs=1M count=10240 --progress
* 只能對 Partition 作操作
dcfldd
Enhanced version of dd for forensics and security
HomePage: http://dcfldd.sourceforge.net/
-
Hashing on-the-fly
dcfldd can hash the input data as it is being transferred, helping to ensure data integrity. -
Status output
dcfldd can update the user of its progress in terms of the amount of data transferred and how much longer operation will take. -
Flexible disk wipes
dcfldd can be used to wipe disks quickly and with a known pattern if desired. -
Image/wipe Verify
dcfldd can verify that a target drive is a bit-for-bit match of the specified input file or pattern.
IO Sync: direct, dsync, sync
說明
direct (use direct I/O for data)
Direct I/O open mode O_DIRECT
= bypass kernel page cache
= no extra copies from userland to the kernel
=> userland buffer -> Disk
=> saving the kernel work
(no need to perform most buffer cache management operations)
* 應用: applications do their own caching
* Due to the lack of caching, sequential reads would have a heavier impact.
* doesn't guarantee that data of a successful write() call will survive sudden power loss.
"O_SYNC" 才有此功能
dsync (use synchronized I/O for data)
sync (likewise, but also for metadata)
direct 與 dsync/sync 的分別
dsync/sync 是一次寫一個 bs 大小的資料, 之後同步寫到磁碟後再繼續寫下一個 IO
=> This bypasses all caches including the hardware cache on the drive itself.
Usage
iflag=direct # this bypasses kernel caching and substantially increases performance.
Disk IO Test
寫入 HDD
cd /ssd # 去要 test 的 device
dd if=/dev/zero of=test.bin bs=1G count=50G oflag=direct
oflag=dsync # direct: use direct I/O for data
# dsync: use synchronized I/O for data
查看情況
# dstat - versatile tool for generating system resource statistics
dstat -d -r -D sde
--dsk/sde-- ---io/sde-- read writ| read writ 7B 178k|0.00 0.61 0 188M| 0 655 0 177M| 0 622 0 183M| 0 640 ...
More
