pipe

 

 

 


Pipe output to two different commands

 

一共有兩方法

  • 方法1: process substitution ">(X)"
  • 方法2: named pipe

方法1: process substitution ">(X)"

tee >(command1) >(command2) | command3

">" 與 "(" 之間不可有空格

">(X)" 的原理

The shell creates the pipe and passes a file name like "/dev/fd/3" to the command.

The number(3) is the file descriptor that the pipe is connected to.

Remark

tee - read from standard input and write to standard output and files

方法2: named pipe

基礎版

mkfifo pipe

cat pipe | (command 1) &

echo 'test' | tee pipe | (command 2)

多 command 進階版

tmp_dir=$(mktemp -d)
mkfifo "$tmp_dir/f1" "$tmp_dir/f2"
command1 <"$tmp_dir/f1" & pid1=$!
command2 <"$tmp_dir/f2" & pid2=$!
tee "$tmp_dir/f1" "$tmp_dir/f2" | command3
rm -rf "$tmp_dir"
wait $pid1 $pid2

應用例子: nc & md5

# Sender

time pv vda.qcow2 | tee >(md5sum > md5.txt) |  nc -w 2 127.0.0.1 6666

# Receiver

nc -v -l 6666 | tee >(md5sum > md5.txt) | pv > vda.qcow2

 


pipe with zip

 

echo testtesttest > a.txt

dd if=a.txt | gzip -c | gzip -d | dd of=b.txt
 


pipe with ssh

 

dd bs=1M if=/dev/sda1 | lz4 | \
  ssh -o 'Compression no' -c arcfour REMOTE \
  'lz4 -d | dd bs=1M of=/dev/sda1'

特點

  • dd with "bs"
  • lz4 compress
  • ssh no compress & low encrytion

 


named pipe

 

A named pipe (also known as a FIFO for its behavior) is an extension to the traditional pipe concept on Unix.

It is one of the methods of inter-process communication (IPC).

A traditional pipe is "unnamed" and lasts only as long as the process.

A named pipe, however, can last as long as the system is up, beyond the life of the process.

 


mkfifo

# mkfifo - make FIFOs (named pipes)

# -m nnn ( Default: prw-r--r-- ; i.e. -m 600 )

i.e.

mkfifo MyPipe

Example 1

A:

mkfifo MyPipe

gzip -9 -c < MyPipe > out.gz &

B:

# 只可以行一次

cat file > MyPipe

Example 2

A:

dd if=/dev/zero of=1k bs=1k count=1

# A read call to a FIFO is blocked until a data is available in the pipe and a write call is blocked until the data is read from another process.

cat 1k > MyPipe

The pipe has a 64k buffer (on linux) and, will block the writer when full.