SO_REUSEPORT

最後更新: 2015-09-24

介紹

 

This socket option allows multiple sockets to listen on the same IP address and port combination.

The kernel then load balances incoming connections across the sockets.

This can reduce lock contention between workers accepting new connections

* improve performance on multicore systems.

* kernel version 3.9 and later

 


Tengine with SO_REUSEPORT enabled (reuse_port on)

 

sysctl.conf

net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_tw_reuse  = 1
net.ipv4.tcp_max_syn_backlog = 65535

# 詳見
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 65535

Tengine configuration file:

  worker_processes auto;
  worker_cpu_affinity auto;
  worker_rlimit_nofile 65535;

  events {
    # number of simultaneous connections(a worker process)
    # this number includes all connections (proxy(in / out))
    # cannot exceed the current limit on the maximum number of open files ( worker_rlimit_nofile )
    worker_connections 65535;

    reuse_port on;
  }

  http {
    include mime.types;
    default_type application/octet-stream;
    access_log logs/access.log;
    keepalive_timeout 0;
    server {
      listen 81 backlog=65535;
      server_name localhost;
      location = /empty.gif {
        empty_gif;
      }
    }
  }

 


Other Options

 

Default:

accept_mutex on;

If accept_mutex is enabled, worker processes will accept new connections by turn.

Otherwise, all worker processes will be notified about new connections (off),

and if volume of new connections is low, some of the worker processes may just waste system resources.

OS may wake all processes waiting on accept() and select(), this is called thundering herd problem.

This is a problem if you have a lot of workers as in Apache (hundreds and more),

but this insensible if you have just several workers as nginx usually has.

Therefore turning accept_mutex off is as scheduling incoming connection by OS via select/kqueue/epoll/etc (but not accept()).

* the reuseport parameter also disables the accept_mutex directive for the socket

 


 

Creative Commons license icon Creative Commons license icon