最後更新: 2019-04-24



  • NAT timeout
  • conntrack 資訊 (conntrack & count)
  • Tools: conntrack
  • Tools: iptstate
  • Conntrack 的出現
  • conntrack 的 Kernel module
  • ip_conntrack table full (conntrack timout)
  • Openwrt
  • Find connections per ip on an OpenWRT router
  • Troubleshoot
  • nf_conntrack_ftp
  • DOC


NAT timeout


RFC 5382 - NAT Behavioral Requirements for TCP

A NAT cannot determine whether the endpoints of a TCP connection are active,

it MAY abandon the session if it has been idle for some time.  

In such cases, the value of the "established connection idle-timeout" (7440 sec)


conntrack 資訊 (conntrack & count)



* display the existing flows, their state and other information


  • /proc/net/nf_conntrack
  • /proc/net/ip_conntrack ( deprecated )


cat /proc/net/nf_ip_conntrack

ipv4     2 tcp      6 1782 ESTABLISHED 
  src=x.x.x.x dst=z.z.z.z sport=33136 dport=1025 packets=3 bytes=164
  src=z.z.z.z dst=y.y.y.y sport=1025 dport=33136 packets=2 bytes=158 [ASSURED] mark=0 use=2

client(x) -> router(y) -> server(z)

client telnet server 的 1025/tcp


  • [ASSURED]     請求和響應都有流量
  • [UNREPLIED]  沒收到響應

Column 說明

# 網絡協議名 協議號 Protocol_Name Protocol_Number  TTL  STATE
ipv4        2    tcp           6                300  ESTABLISHED

conntrack count


  • /proc/sys/net/netfilter/nf_conntrack_count
  • /proc/sys/net/ipv4/netfilter/ip_conntrack_count ( deprecated )

不斷 checking nat 數量

while true; do date; cat /proc/sys/net/netfilter/nf_conntrack_count; sleep 1; done


Tools: conntrack



# Ubuntu

apt-get install conntrack

# C7

yum install conntrack

conntrack tables:

* conntrack (除了 NOTRACK target 外全部 connection 都在這裡)
* expect (RELATED connections to existing ones)
* unconfirmed (new entries, that are not yet inserted into the conntrack table)


-L, --dump
-G, --get
                # Search for and show a particular (matching) entry
-D, --delete
-E, --event             # Real time event
-F, --flush               # Flush the whole given table
-C, --count             # Show the table counter
                             # (cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count)
-S, --stats              # Show the in-kernel connection tracking system statistics

cpu=0    found=0 invalid=79 ignore=254806 insert=0 insert_failed=0 drop=0 
         early_drop=0 error=76 search_restart=417
cpu=1    found=0 invalid=89 ignore=263587 insert=0 insert_failed=0 drop=0 
         early_drop=0 error=87 search_restart=572

List: -L

conntrack -L [table] [options]


  • -s, --orig-src IP_ADDRESS
  • -d, --orig-dst IP_ADDRESS
  • -p, --proto PROTO
  • --sport, --orig-port-src PORT
  • --dport, --orig-port-dst PORT
  • -n, --src-nat
  • -g, --dst-nat
  • -j, --any-nat


conntrack -L

udp      17 28 src=  dst= sport=46011 dport=1900 [UNREPLIED] 
               src= dst=  sport=1900 dport=46011 mark=0 use=1
conntrack v1.4.4 (conntrack-tools): 19 flow entries have been shown.

udp <= Protocol

17 <= udp Protocol Numbers

28 <= how long this conntrack entry has to live (decremented regularly until we see more traffic)

When a connection has seen traffic in both directions, the conntrack entry will erase the [UNREPLIED] flag


[ASSURED] flag tells us that this connection is assured and

that it will not be erased if we reach the maximum possible tracked connections.

conntrack -L -s

icmp     1 29 src= dst= type=8 code=0 id=1 
src= dst= type=0 code=0 id=1 mark=0 use=1
conntrack v1.0.0 (conntrack-tools): 1 flow entries have been shown.

conntrack -L -p tcp --dport 21

tcp      6 431994 ESTABLISHED src= dst= sport=36233 dport=21 
                              src= dst= sport=21 dport=36233 [ASSURED] 
                              mark=0 helper=ftp use=2

Deleting Specific Entries from Conntrack: -D

root@home:~# conntrack -D -s

icmp     1 29 src= dst= type=8 code=0 id=1 
src= dst= type=0 code=0 id=1 mark=0 use=1
conntrack v1.0.0 (conntrack-tools): 1 flow entries have been deleted.


Tools: iptstate


iptstate - top-like interface to your netfilter connection-tracking table

apt-get install iptstate


  • b   Sort by next column
  • B   Sort by previous column
  • r   Toggle reverse sorting

x   Delete the currently highlighted state from netfilter

p   Toggle scrolling

                                                 IPTState - IPTables State Top
Version: 2.2.6        Sort: DstIP           b: change sorting   h: help
Source                     Destination         Prt  State       TTL       n.n.n.n:2194        tcp  SYN_SENT    0:00:06         udp              0:00:21


Conntrack 的出現


在 Centos 上

-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT

* 當 "#" 了第一行, 那 port 22 會連不到  !!

conntrack table full


Aug 20 10:16:37 kernel: nf_ct_sip: dropping packetIN= OUT=br0 SRC=w.w.w.w DST= 
                LEN=462 TOS=0x00 PREC=0x00 TTL=119 ID=21433 PROTO=UDP SPT=5125 DPT=5060 LEN=442
Aug 20 10:16:41 kernel: net_ratelimit: 302020 callbacks suppressed
Aug 20 10:16:41 kernel: nf_conntrack: expectation table full


conntrack 的 Kernel module


lsmod | egrep 'conntrack'


nf_conntrack           46014 13 iptable_nat
nf_conntrack_ipv4       5188 11

nf_defrag_ipv4: defragment IPv4 packets before they reach Netfilter's connection tracking (nf_conntrack_ipv4 module)

helper 類

nf_conntrack_sip       16691  1 nf_nat_sip
nf_conntrack_h323      35054  1 nf_nat_h323
nf_conntrack_rtsp       4280  1 nf_nat_rtsp
nf_conntrack_ftp        5155  1 nf_nat_ftp


Xtables: connection tracking state match

xt_conntrack            2160 10

 * xt_?



ip_conntrack table full (conntrack timout)


當在 log 裡見到

Dec  6 21:37:47 vm kernel: ip_conntrack: table full, dropping packet.
Dec  6 21:37:49 vm last message repeated 9 times
Dec  6 21:39:06 vm kernel: printk: 1 messages suppressed.

那就是 TCP 的 Connection Full 了的情況

現在 track 的數量

  • cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count

IP 的使用情況:

  • sed -ne 's/^.*src=\([^ ]*\).*/\1/p' /proc/net/ip_conntrack | sort | uniq -c | sort -r | head

如果過多的話, 可以考慮設置較低的 timeout 時間

conntrack timout

  • cat /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_*

其中最值得注意的是 ip_conntrack_tcp_timeout_established, 因為它是 5 天的...

  • cat /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established

修改 timeout

sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=600

conntrack 的上限

  • cat /proc/sys/net/nf_conntrack_max

memory usage (slabs)

ip_conntrack_max * 232 Bytes


# Default: nf_conntrack_buckets  x 4

sysctl -w net.nf_conntrack_max=10240       # 相當修改了 net.netfilter.nf_conntrack_max






# Size of hash table. Default: memory / 16384

# only writeable in the initial net namespace

# For systems with more than 4GB => 65536 (2^16)

sysctl net.netfilter.nf_conntrack_buckets




Install conntrack-tools

opkg update && opkg install conntrack-tools


OS: Openwrt 14.07 (Linux 3.10.49)

Kernel model: nf_conntrack.ko (lsmod | grep nf_conntrack)

載入了 model 後, 就有 tables

- /proc/net/nf_conntrack
- /proc/net/nf_conntrack_expect

[ASSURED] - 表示客戶端和路由器之間的連接已經確認
[UNREPLIED] - 表示服務器不響應客戶端

# 現在 conntrack 的用量

cat /proc/sys/net/netfilter/nf_conntrack_count


# conntrack 的上限

# 查看上限

cat /proc/sys/net/netfilter/nf_conntrack_max


# 設置上限

# realtime

echo 300000 > /proc/sys/net/netfilter/nf_conntrack_max

# permanent

sysctl net.netfilter.nf_conntrack_max=300000

# nf_conntrack_expect_max

# the mechanism used to "expect" connections related to existing ones
# Expectations are generally used by connection tracking helpers (FTP, SIP, H.323)


ls /proc/sys/net/netfilter


# tcp 3 way handshake

# nf_conntrack_tcp_timeout_syn_sent # 120s, iptables state: new

SYN_SENT           # S  <--  FW  <--SYN--  C


# nf_conntrack_tcp_timeout_syn_recv # 60s, iptables state : established

SYN_RECV           # S  --SYN+ACK-->  FW  -->  C


# nf_conntrack_tcp_timeout_established # 3600s, iptables state : established

ESTABLISHED        # S  <--  FW  <-- ACK--  C

* C-S Side 的 ESTABLISHED 與 iptables 的 ESTABLISHED 係有不同的


nf_conntrack_tcp_max_retrans  (3)

# number of packets that can be retransmitted without received an (acceptable) ACK from the destination


nf_conntrack_udp_timeout                    # cleint request 後等待 server 的 response (60)

nf_conntrack_udp_timeout_stream        # 當 server 有 response, 那就是 stream 了 (180)


nf_conntrack_acct                                  # 1, Enable connection tracking flow accounting. 64-bit

nf_conntrack_buckets                             # 1024, read-only


Verify checksum of incoming packets.

Packets with bad checksums are in INVALID state.

If this is enabled, such packets will not be considered for connection tracking.


Log invalid packets of a type specified by value.

0   - disable (default)
1   - log ICMP packets
6   - log TCP packets
17  - log UDP packets
255 - log packets of any protocol

nf_conntrack_helper (1)

Enable automatic conntrack helper assignment

nf_conntrack_skip_filter (1)

packets with the established state, completely bypass iptables filter table


nf_conntrack_tcp_loose (1)

# If it is set to zero, we disable picking up already established connections.


nf_conntrack_helper (1)


# Get timeout value

sysctl -a 2>/dev/null | grep conntrack.*timeout


nf_conntrack_icmp_timeout (30)

Default for ICMP timeout.

nf_conntrack_generic_timeout (600)

This refers to layer 4 unknown/unsupported protocols.


nf_conntrack_tcp_timeout_unacknowledged (300)


nf_conntrack_tcp_timeout_max_retrans (300)

nf_conntrack_tcp_timeout_time_wait (120)





Find connections per ip on an OpenWRT router


[2] Router 自身的 connection

netstat -ntu

[1] NAT 的 connection

sed -ne 's/^.*src=\([^ ]*\).*/\1/p' /proc/net/nf_conntrack | sort | uniq -c | sort






[6717278.750255] nf_conntrack: falling back to vmalloc.
[6717278.760663] nf_conntrack: falling back to vmalloc.
[6717278.760797] ip_set: protocol 6







# The systemd-modules-load.service daemon will read these files and load the modules.




options nf_conntrack_ftp ports=21


lsmod | grep ftp


# Source address => Fix IP address
loose = 0 (default)


# firewall-cmd default 有

"ACCEPT     all  --              ctstate RELATED,ESTABLISHED"

或 iptables 有

$IPT -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT


ESTABLISHED: A packet that is part of an existing connection.

RELATED: A packet that is requesting a new connection but is part of an existing connection.
               For example, FTP uses port 21 to establish a connection, but data is transferred on a different port