10 - Tengine

最後更新: 2019-04-09

介紹

HomePage: http://tengine.taobao.org/

Based on the Nginx

Version

Tengine-2.2.3 => nginx-1.8.1

Tengine-2.1.2 => nginx-1.6.2

Taobao 及 Sogou 在用

功能

  • Dynamic module loading support (DSO). You don't have to recompile Tengine when adding new modules to it.
  • Sends unbuffered upload directly to HTTP and FastCGI backend servers, which saves disk I/Os.
  • More load balancing methods
     - consistent hashing
     - session persistence
     - upstream health check
     - resolving upstream domain names on the fly.
  • Input body filter support. It's quite handy to write Web Application Firewalls using this mechanism.
  • Dynamic scripting language (Lua) support, which is very efficient and makes it easy to extend core functionalities.
  • Multiple CSS or JavaScript requests can be combined into one request to reduce download time.
  • Removes unnecessary white spaces and comments to reduce the size of a page.
  • Proactive health checks of upstream servers can be performed.
  • The number of worker processes and CPU affinities can be set automatically.
  • Protects the server in case system load or memory use goes too high.
  • The number of worker processes and CPU affinities can be set automatically.
  • The limit_req module is enhanced with whitelist support and more conditions are allowed in a single location.

Version:

nginx -v

Tengine version: Tengine/1.5.2 (nginx/1.2.9)

tengine-1.5.2 - All features of nginx-1.2.9 are inherited

目錄

  • tengine 比 nginx 的額外 Opts
  • Install
  • Without something
  • 查看 complie 出黍既 nginx 有什麼 module
  • module 篇
  • DSO

 


Install

 

Get Source

方法1

mkdir /usr/src/tengine

cd /usr/src/tengine

git clone https://github.com/alibaba/tengine.git

cd tengine; git status

方法2

mkdir /usr/src/tengine

cd /usr/src/tengine

wget http://tengine.taobao.org/download/tengine-2.1.1.tar.gz

tar -zxf tengine-2.1.1.tar.gz

cd tengine-2.1.1/

Check 下有咩 compile opts

./configure --help

Centos 7 with Tengine 2.1.3:

yum groupinstall "Development Tools" -y

yum install epel-release -y

yum install pcre-devel gd-devel libxslt-devel geoip-devel -y

yum install openssl openssl-devel -y

yum install perl-devel perl-ExtUtils-Embed -y

Ubuntu Install

apt-get install gcc make libpcre3-dev libzip-dev libssl-dev

./configure 的設定 (ubuntu-12.04)

# By default, it will be installed to /usr/local/nginx

Ver: 2.2.3 (Size: M)

 

Ver: 2.1.3 (Size: 7.7M)

build.sh

#!/bin/bash

cd tengine

./configure \
 --prefix=/opt/tengine \
 --conf-path=/etc/tengine/nginx.conf \
 --error-log-path=/var/log/tengine/error.log \
 --http-log-path=/var/log/tengine/access.log \
 --pid-path=/var/run/tengine.pid \
 --user=tengine --group=tengine \
 --with-ld-opt="-Wl,-E" \
 --with-http_v2_module \
 --without-select_module \
 --without-poll_module \
 --without-http_fastcgi_module \
 --without-http_ssi_module \
 --without-http_uwsgi_module \
 --without-http_scgi_module \
 --without-http_memcached_module \
 --without-mail_smtp_module \
 --without-mail_imap_module \
 --without-mail_pop3_module \
 --without-procs \
 --without-http_browser_module \
 --without-http_user_agent_module

P.S. 1

# 要設定 source code 的位置
--with-jemalloc --with-jemalloc=/usr/src/jemalloc/jemalloc-stable-3

P.S. 2

別加 "--enable-mods-static=all", 因為會 enable 了所有 module

output

Configuration summary
  + using system PCRE library
  + using system OpenSSL library
  + md5: using OpenSSL library
  + sha1: using OpenSSL library
  + using system zlib library
  + jemalloc library is disabled

  nginx path prefix: "/opt/tengine"
  nginx binary file: "/opt/tengine/sbin/nginx"
  nginx configuration prefix: "/etc/tengine"
  nginx configuration file: "/etc/tengine/nginx.conf"
  nginx pid file: "/var/run/tengine.pid"
  nginx error log file: "/var/log/tengine/error.log"
  nginx http access log file: "/var/log/tengine/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

Ver: 1.5.2

build.sh

#!/bin/bash

cd tengine

./configure --prefix=/opt/tengine --conf-path=/etc/nginx/nginx.conf \
            --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log \
            --pid-path=/var/run/nginx.pid \
            --user=tengine --group=tengine \
            --enable-mods-static=all \
            --with-jemalloc --with-jemalloc=/usr/src/jemalloc-3.5.0 \
            --with-zlib --with-pcre --with-openssl \

一堆 without module

--without-syslog \
--without-dso \
--without-select_module \
--without-poll_module \
--without-http_fastcgi_module \
--without-http_ssi_module \
--without-http_uwsgi_module \
--without-http_scgi_module \
--without-http_memcached_module \
--without-mail_smtp_module \
--without-mail_imap_module \
--without-mail_pop3_module \
--without-procs \
--without-http_browser_module \
--without-http_user_agent_module

 


Troubleshoot

 

[1]

nginx -t

Can't load '/usr/local/lib64/perl5/auto/nginx/nginx.so'
for module nginx: /usr/local/lib64/perl5/auto/nginx/nginx.so:
undefined symbol: ngx_http_perl_handle_request at /usr/share/perl5/XSLoader.pm line 68.
at /usr/local/lib64/perl5/nginx.pm line 56.
Compilation failed in require.
BEGIN failed--compilation aborted.
nginx: [alert] perl_parse() failed: 255
nginx: configuration file /etc/tengine/nginx.conf test failed

加入

--with-ld-opt="-Wl,-E" \

 


tengine 比 nginx 的額外 Opts

 

# Show all compiled-in modules then exit.

the '-m' option

Tengine version: Tengine/2.1.1 (nginx/1.6.2)
loaded modules:
    ngx_core_module (static)
    ngx_errlog_module (static)
    ngx_conf_module (static)
    ngx_dso_module (static)
    ngx_syslog_module (static)
    ngx_events_module (static)

.........................

* 自行 compile 的話, 基本上全部都係 static ( size: 7.9M )

# Show all supported directives then exit.

the '-l' option

# Dump the contents of the configuration file, including the contents of 'include'd files.

the '-d' option

 


Without something

 

--without-http_footer_filter_module

--without-http_memcached_module

--without-http_scgi_module

# The Simple Common Gateway Interface (SCGI) is a protocol
# for applications to interface with HTTP servers, as an alternative to the CGI protocol.

--without-syslog

# tengine 才有的 opts

# OS 沒有 kqueue, epoll, rtsig 才會選用它

--without-select_module
-–without-poll_module

# mail 的功能

--without-mail_pop3_module
--without-mail_imap_module
--without-mail_smtp_module

--without-http_ssi_module

 


查看 complie 出來的 tengine 有什麼 module

 

nginx -V

loaded modules:
    ngx_core_module (static)
    ngx_errlog_module (static)
    ngx_conf_module (static)
    ngx_dso_module (static)
    ngx_syslog_module (static)
    ...........................

 



Module 篇

 

ngx_http_trim_filter_module

它會清了

# comments
# spaces, tabs, newlines

設定:

location / {
    trim on;
    # inline javascript and css
    trim_jscss on;
}

trim_types MIME types
Default: trim_types: text/html

procs:

provides a mechanism to support standalone processes

http://tengine.taobao.org/document/proc.html

useragent:

similar to NGINX’s native ngx_http_browser_module

ngx_http_spdy_module

# Server can detect whether SPDY or HTTP is used directly, and also it can negotiate with client via NPN, for example:

listen 443 ssl spdy_detect spdy;

* spdy_detect: without using a TLS extension (NPN)

the server will examine the first byte of one connection and determine whether the connection is SPDY or HTTP based on what it looks like
(0x80 or 0x00 for SPDY).

http_upstream_consistent_hash_module

consistent_hash

consistent_hash $remote_addr: mapping via client ip address

consistent_hash $request_uri: mapping via request-uri

"id" field

used as server flag. If id field is not set,
ip address and port are used to identify server.

i.e.

worker_processes  1;
http {
    upstream test {
        consistent_hash $request_uri;
        server 192.168.0.11:80 weight=5;
        server 192.168.0.12:80 weight=3;
        server 192.168.0.13:80 weight=2;
    }
}

 


Library

 

libxslt

Libxslt is the XSLT C library developed for the GNOME project.

XSLT itself is a an XML language to define transformation for XML.

Libxslt is based on libxml2 the XML C library developed for the GNOME project.

 


DSO

 

dso {
     load ngx_http_lua_module.so;
     load ngx_http_memcached_module.so;
}

# load [module_name] [module_path]

 - the absolute path

 - relative path to the default path (NGX_PREFIX/modules

How to compile a module

(1) manual:

./configure --with-http_sub_module=shared

make

make dso_install

(2) dso_tool:

# You can specify the destination directory you want install to by the --dst option.

./dso_tool --add-module=/home/dso/lua-nginx-module

 


Doc