最後更新: 2020-08-25

介紹

Pipes and FIFOs (also known as named pipes) provide a unidirectional interprocess communication channel.

目錄

• Process Substitution - "p2 <(p1)"
• Pipe output to two different commands
• pipe with zip
• pipe with ssh
• named pipe
• mkfifo

 

---

Process Substitution

 

Syntax

• <(command)       # run command and make its output appear as a file (stdout -> file)
• >(command)       # feeds the output of a process into the stdin of another process (file -> stdin)

 * "<", ">" 與 ")", "(" 之間不可有空格

"p2 <(p1)" 測試

echo <(echo test)    # /dev/fd/63

cat <(echo test)      # test

原理

p1 的 stdout 放到臨時的檔案(/dev/fd/N)
這裡的臨時檔案是 "file descriptors"

"p2 <(p1)" 應用場景

A command does not support input from STDIN(|"|")

Process substitution feeds the output of a process (or processes) into the stdin of another process

">(X)" 的原理

The shell creates the pipe and passes a file name like "/dev/fd/3" to the command.

The number(3) is the file descriptor that the pipe is connected to.

Testing

echo >(true) <(true)

/dev/fd/63 /dev/fd/62

 

---

Pipe output to two different commands

 

一共有兩方法

• 方法1: process substitution ">(X)"
• 方法2: named pipe

方法1: process substitution ">(X)"

tee >(command1) >(command2) | command3

">" 與 "(" 之間不可有空格

Remark

tee

方法2: named pipe

基礎版

mkfifo pipe

cat pipe | (command 1) &

echo 'test' | tee pipe | (command 2)

多 command 進階版

tmp_dir=$(mktemp -d)
mkfifo "$tmp_dir/f1" "$tmp_dir/f2"
command1 <"$tmp_dir/f1" & pid1=$!
command2 <"$tmp_dir/f2" & pid2=$!
tee "$tmp_dir/f1" "$tmp_dir/f2" | command3
rm -rf "$tmp_dir"
wait $pid1 $pid2

應用例子: nc & md5

# Sender

time pv vda.qcow2 | tee >(md5sum > md5.txt) |  nc -w 2 127.0.0.1 6666

# Receiver

nc -v -l 6666 | tee >(md5sum > md5.txt) | pv > vda.qcow2

 

 

---

pipe with zip

 

echo testtesttest > a.txt

dd if=a.txt | gzip -c | gzip -d | dd of=b.txt
 

---

pipe with ssh

 

ie.

dd bs=1M if=/dev/sda1 | lz4 | \
  ssh -o 'Compression no' -c arcfour REMOTE \
  'lz4 -d | dd bs=1M of=/dev/sda1'

特點

• dd with "bs"
• lz4 compress
• ssh no compress & low encrytion

 

---

named pipe

 

A named pipe (also known as a FIFO for its behavior) is an extension to the traditional pipe concept on Unix.

It is one of the methods of inter-process communication (IPC).

A traditional pipe is "unnamed" and lasts only as long as the process.

A named pipe, however, can last as long as the system is up, beyond the life of the process.

 

---

mkfifo

 

mkfifo - make FIFOs (named pipes)

Opts

-m nnn     # 設定 permission. Default: prw-r--r-- ; i.e. -m 600 )

i.e.

mkfifo MyPipe

Example 1: Create temporary pipeline

# -u, --dry-run              do not create anything; merely print a name (unsafe)

tmppipe=$(mktemp -u)

mkfifo -m 600 "$tmppipe"

Example 2: gzip by pipe

A: 是有次數的, A 後才可以 B

mkfifo MyPipe

gzip -9 -c < MyPipe > out.gz &

B:

# 只可以行一次

cat file > MyPipe

Example 3

mkfifo MyPipe

# A read call to a FIFO is blocked until a data is available in the pipe

cat MyPipe> test.bin

dd if=/dev/zero of=MyPipe bs=1k count=1

# A write call is blocked until the data is read from another process.

dd if=/dev/zero of=MyPipe bs=1k count=1

cat MyPipe > test.bin

# The pipe has a 64k buffer (on linux) and, will block the writer when full

dd if=/dev/zero bs=1 | sleep 999

3, 4 秒後按 "Ctrl + C"

^C65537+0 records in
65536+0 records out
65536 bytes (66 kB, 64 KiB) copied, 2.47784 s, 26.4 kB/s

OR

# sending 1k chunks to blocked pipe until buffer full
M=0;
while true; do
    dd if=/dev/zero bs=1k count=1 2>/dev/null;
    M=$(($M+1));
    echo -en "\r$M KB" 1>&2;
done | sleep 999

 

 

---