首頁 » network

conntrack

由 datahunter 在 四, 31/10/2013 - 15:25 發表

最後更新: 2019-04-24

 

目錄

  • NAT timeout
  • conntrack 資訊 (conntrack & count)
  • Tools: conntrack
  • Tools: iptstate
  • Conntrack 的出現
  • conntrack 的 Kernel module
  • ip_conntrack table full (conntrack timout)
  • Openwrt
  • Find connections per ip on an OpenWRT router
  • Troubleshoot
  • nf_conntrack_ftp
  • DOC

 

NAT timeout

 

RFC 5382 - NAT Behavioral Requirements for TCP

A NAT cannot determine whether the endpoints of a TCP connection are active,

it MAY abandon the session if it has been idle for some time.  

In such cases, the value of the "established connection idle-timeout" (7440 sec)

 

conntrack 資訊 (conntrack & count)

 

conntrack

* display the existing flows, their state and other information

Path:

  • /proc/net/nf_conntrack
  • /proc/net/ip_conntrack ( deprecated )

Example:

cat /proc/net/nf_ip_conntrack

ipv4     2 tcp      6 1782 ESTABLISHED 
  src=x.x.x.x dst=z.z.z.z sport=33136 dport=1025 packets=3 bytes=164
  src=z.z.z.z dst=y.y.y.y sport=1025 dport=33136 packets=2 bytes=158 [ASSURED] mark=0 use=2

client(x) -> router(y) -> server(z)

client telnet server 的 1025/tcp

flag:

  • [ASSURED]     請求和響應都有流量
  • [UNREPLIED]  沒收到響應

Column 說明

# 網絡協議名 協議號 Protocol_Name Protocol_Number  TTL  STATE
ipv4        2    tcp           6                300  ESTABLISHED

conntrack count

Path:

  • /proc/sys/net/netfilter/nf_conntrack_count
  • /proc/sys/net/ipv4/netfilter/ip_conntrack_count ( deprecated )

不斷 checking nat 數量

while true; do date; cat /proc/sys/net/netfilter/nf_conntrack_count; sleep 1; done

 

Tools: conntrack

 

Install

Ubuntu

apt-get install conntrack

Centos

yum install conntrack-tools  # C6

yum install conntrack          # C7

conntrack tables:

* conntrack (除了 NOTRACK target 外全部 connection 都在這裡)
* expect (RELATED connections to existing ones)
* unconfirmed (new entries, that are not yet inserted into the conntrack table)

Options:

-L, --dump
-G, --get                # Search for and show a particular (matching) entry
-D, --delete
-E, --event             # Real time event
-F, --flush               # Flush the whole given table
-C, --count             # Show the table counter
                             # (cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count)
-S, --stats              # Show the in-kernel connection tracking system statistics

cpu=0    found=0 invalid=79 ignore=254806 insert=0 insert_failed=0 drop=0 
         early_drop=0 error=76 search_restart=417
cpu=1    found=0 invalid=89 ignore=263587 insert=0 insert_failed=0 drop=0 
         early_drop=0 error=87 search_restart=572
...

List: -L

conntrack -L [table] [options]

parameters

  • -s, --orig-src IP_ADDRESS
  • -d, --orig-dst IP_ADDRESS
     
  • -p, --proto PROTO
  • --sport, --orig-port-src PORT
  • --dport, --orig-port-dst PORT
     
  • -n, --src-nat
  • -g, --dst-nat
  • -j, --any-nat

Example

conntrack -L

....
udp      17 28 src=192.168.88.61  dst=192.168.88.255 sport=46011 dport=1900 [UNREPLIED] 
               src=192.168.88.255 dst=192.168.88.61  sport=1900 dport=46011 mark=0 use=1
conntrack v1.4.4 (conntrack-tools): 19 flow entries have been shown.

udp <= Protocol

17 <= udp Protocol Numbers

28 <= how long this conntrack entry has to live (decremented regularly until we see more traffic)

When a connection has seen traffic in both directions, the conntrack entry will erase the [UNREPLIED] flag

( [UNREPLIED] -> [ASSURED] )

[ASSURED] flag tells us that this connection is assured and

that it will not be erased if we reach the maximum possible tracked connections.

conntrack -L -s 192.168.123.200

icmp     1 29 src=192.168.123.200 dst=192.168.123.12 type=8 code=0 id=1 
src=192.168.123.12 dst=192.168.123.200 type=0 code=0 id=1 mark=0 use=1
conntrack v1.0.0 (conntrack-tools): 1 flow entries have been shown.

conntrack -L -p tcp --dport 21

tcp      6 431994 ESTABLISHED src=192.168.88.177 dst=192.168.88.182 sport=36233 dport=21 
                              src=192.168.88.182 dst=192.168.88.177 sport=21 dport=36233 [ASSURED] 
                              mark=0 helper=ftp use=2

# 找出某段 IP 的 track

conntrack -L | awk '$4 ~ /^src=R.R/{print $4}'

 

Deleting Specific Entries from Conntrack: -D

root@home:~# conntrack -D -s 192.168.123.200

icmp     1 29 src=192.168.123.200 dst=192.168.123.12 type=8 code=0 id=1 
src=192.168.123.12 dst=192.168.123.200 type=0 code=0 id=1 mark=0 use=1
conntrack v1.0.0 (conntrack-tools): 1 flow entries have been deleted.

 

Tools: iptstate

 

iptstate - top-like interface to your netfilter connection-tracking table

apt-get install iptstate

Sort

  • b   Sort by next column
  • B   Sort by previous column
  • r   Toggle reverse sorting

x   Delete the currently highlighted state from netfilter

p   Toggle scrolling

                                                 IPTState - IPTables State Top
Version: 2.2.6        Sort: DstIP           b: change sorting   h: help
Source                     Destination         Prt  State       TTL
192.168.88.150:40634       n.n.n.n:2194        tcp  SYN_SENT    0:00:06
127.0.0.1:53984            127.0.0.1:123       udp              0:00:21

 

Conntrack 的出現

 

在 Centos 上

-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT

* 當 "#" 了第一行, 那 port 22 會連不到  !!

conntrack table full

log

Aug 20 10:16:37 kernel: nf_ct_sip: dropping packetIN= OUT=br0 SRC=w.w.w.w DST=192.168.88.150 
                LEN=462 TOS=0x00 PREC=0x00 TTL=119 ID=21433 PROTO=UDP SPT=5125 DPT=5060 LEN=442
Aug 20 10:16:41 kernel: net_ratelimit: 302020 callbacks suppressed
Aug 20 10:16:41 kernel: nf_conntrack: expectation table full

 

conntrack 的 Kernel module

 

lsmod | egrep 'conntrack'

main

nf_conntrack           46014 13 iptable_nat
nf_conntrack_ipv4       5188 11

nf_defrag_ipv4: defragment IPv4 packets before they reach Netfilter's connection tracking (nf_conntrack_ipv4 module)

helper 類

nf_conntrack_sip       16691  1 nf_nat_sip
nf_conntrack_h323      35054  1 nf_nat_h323
nf_conntrack_rtsp       4280  1 nf_nat_rtsp
nf_conntrack_ftp        5155  1 nf_nat_ftp

function 類

Xtables: connection tracking state match

xt_conntrack            2160 10

 * xt_?

xt_comment
xt_conntrack
...

 

ip_conntrack table full (conntrack timout)

 

當在 log 裡見到

Dec  6 21:37:47 vm kernel: ip_conntrack: table full, dropping packet.
Dec  6 21:37:49 vm last message repeated 9 times
Dec  6 21:39:06 vm kernel: printk: 1 messages suppressed.

那就是 TCP 的 Connection Full 了的情況

現在 track 的數量

  • cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count

IP 的使用情況:

  • sed -ne 's/^.*src=\([^ ]*\).*/\1/p' /proc/net/ip_conntrack | sort | uniq -c | sort -r | head

如果過多的話, 可以考慮設置較低的 timeout 時間

conntrack timout

  • cat /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_*

其中最值得注意的是 ip_conntrack_tcp_timeout_established, 因為它是 5 天的...

  • cat /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established

修改 timeout

sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=600

conntrack 的上限

  • cat /proc/sys/net/nf_conntrack_max
16384

memory usage (slabs)

ip_conntrack_max * 232 Bytes

修改上限:

# Default: nf_conntrack_buckets  x 4

sysctl -w net.nf_conntrack_max=10240       # 相當修改了 net.netfilter.nf_conntrack_max

/etc/sysctl.conf

net.ipv4.ip_conntrack_max=102400

 

nf_conntrack_buckets

 

# Size of hash table. Default: memory / 16384

# only writeable in the initial net namespace

# For systems with more than 4GB => 65536 (2^16)

sysctl net.netfilter.nf_conntrack_buckets

 

Openwrt

 

Install conntrack-tools

opkg update && opkg install conntrack-tools

Conntrack

OS: Openwrt 14.07 (Linux 3.10.49)

Kernel model: nf_conntrack.ko (lsmod | grep nf_conntrack)

載入了 model 後, 就有 tables

- /proc/net/nf_conntrack
- /proc/net/nf_conntrack_expect

[ASSURED] - 表示客戶端和路由器之間的連接已經確認
[UNREPLIED] - 表示服務器不響應客戶端

# 現在 conntrack 的用量

cat /proc/sys/net/netfilter/nf_conntrack_count

249

# conntrack 的上限

# 查看上限

cat /proc/sys/net/netfilter/nf_conntrack_max

16384

# 設置上限

# realtime

echo 300000 > /proc/sys/net/netfilter/nf_conntrack_max

# permanent

sysctl net.netfilter.nf_conntrack_max=300000

# nf_conntrack_expect_max

# the mechanism used to "expect" connections related to existing ones
# Expectations are generally used by connection tracking helpers (FTP, SIP, H.323)

其他設定

ls /proc/sys/net/netfilter

tcp

# tcp 3 way handshake

# nf_conntrack_tcp_timeout_syn_sent # 120s, iptables state: new

SYN_SENT           # S  <--  FW  <--SYN--  C

==========

# nf_conntrack_tcp_timeout_syn_recv # 60s, iptables state : established

SYN_RECV           # S  --SYN+ACK-->  FW  -->  C

==========

# nf_conntrack_tcp_timeout_established # 3600s, iptables state : established

ESTABLISHED        # S  <--  FW  <-- ACK--  C

* C-S Side 的 ESTABLISHED 與 iptables 的 ESTABLISHED 係有不同的

==========

nf_conntrack_tcp_max_retrans  (3)

# number of packets that can be retransmitted without received an (acceptable) ACK from the destination

udp

nf_conntrack_udp_timeout                    # cleint request 後等待 server 的 response (60)

nf_conntrack_udp_timeout_stream        # 當 server 有 response, 那就是 stream 了 (180)

options

nf_conntrack_acct                                  # 1, Enable connection tracking flow accounting. 64-bit

nf_conntrack_buckets                             # 1024, read-only

nf_conntrack_checksum                         

Verify checksum of incoming packets.

Packets with bad checksums are in INVALID state.

If this is enabled, such packets will not be considered for connection tracking.

nf_conntrack_log_invalid

Log invalid packets of a type specified by value.

0   - disable (default)
1   - log ICMP packets
6   - log TCP packets
17  - log UDP packets
255 - log packets of any protocol

nf_conntrack_helper (1)

Enable automatic conntrack helper assignment

nf_conntrack_skip_filter (1)

packets with the established state, completely bypass iptables filter table

nf_conntrack_tcp_be_liberal

nf_conntrack_tcp_loose (1)

# If it is set to zero, we disable picking up already established connections.

nf_conntrack_tcp_no_window_check

nf_conntrack_helper (1)

timeout

# Get timeout value

sysctl -a 2>/dev/null | grep conntrack.*timeout

 

nf_conntrack_icmp_timeout (30)

Default for ICMP timeout.

nf_conntrack_generic_timeout (600)

This refers to layer 4 unknown/unsupported protocols.

nf_conntrack_tcp_timeout_last_ack

nf_conntrack_tcp_timeout_unacknowledged (300)

nf_conntrack_tcp_timeout_fin_wait

nf_conntrack_tcp_timeout_max_retrans (300)

nf_conntrack_tcp_timeout_time_wait (120)

close

nf_conntrack_tcp_timeout_close

nf_conntrack_tcp_timeout_close_wait

 

Find connections per ip on an OpenWRT router

 

[2] Router 自身的 connection

netstat -ntu

[1] NAT 的 connection

sed -ne 's/^.*src=\([^ ]*\).*/\1/p' /proc/net/nf_conntrack | sort | uniq -c | sort

output

............
     19 192.168.123.1
     27 192.168.123.14
    260 192.168.123.41

Troubleshoot

 

[1Q]

[6717278.750255] nf_conntrack: falling back to vmalloc.
[6717278.760663] nf_conntrack: falling back to vmalloc.
[6717278.760797] ip_set: protocol 6

[1A]

reboot

 

nf_conntrack_ftp

 

設定及套用

# The systemd-modules-load.service daemon will read these files and load the modules.

/etc/modules-load.d/iptables.conf

nf_conntrack_ftp

/etc/modprobe.d/iptables.conf

options nf_conntrack_ftp ports=21

reboot

lsmod | grep ftp

P.S.

# Source address => Fix IP address
loose = 0 (default)

原理

# firewall-cmd default 有

"ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED"

或 iptables 有

$IPT -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT

說明

ESTABLISHED: A packet that is part of an existing connection.

RELATED: A packet that is requesting a new connection but is part of an existing connection.
               For example, FTP uses port 21 to establish a connection, but data is transferred on a different port

 

Doc

http://datahunter.org/iptables#conntrack

http://conntrack-tools.netfilter.org/conntrack.html

»
  • 瀏覽次數： 11970