uWSGI

最後更新: 2017-01-02

目錄

uwsgi 介紹
    Components & Package
    uwsgi Install & testing
nginx 設定
    file: uwsgi_params
    Cluster App Server
    Dynamic apps
    Static files
    uwsgi protocol magic variables
Automatically starting uWSGI on boot
    Default configure
    Start / Stop Script
parses uwsgi configure files
    Expanding variables
    Placeholders
    ‘@’ magic
    Magic variables
    Fallback configuration
    Configuration logic
Router
uwsgi Protocol
Signals for controlling uWSGI
Logging
webpy
進階使用
Router
Troubleshoot
Doc

 


Web Server connect Script 方式

 

cgi (one new process per request) [PUT/POST + Env Var =>  standard input]

mod_python (Apache module)

fcgi (FastCGI uses persistent processes to handle a series of requests)

運作: mod_fastcgi(Apache) acts as a FCGI client

wsgi (Web Server Gateway Interface){mod_wsgi}[Python - low-level interface ]

運作:

server -- env info +  callback function --> py
server  <-- HTTP headers + content -- py framework

http://datahunter.org/apache_with_python

uwsgi

A full stack for building hosting services (apps, proxies, pm, mon)

 


uWSGI 介紹

 

A full stack for building hosting services (apps, proxies, pm, mon)

Application servers (for various programming languages and protocols),

proxies, process managers and monitors are all implemented using a common api and a common configuration style.

HomePage: http://projects.unbit.it/uwsgi/

 


Components & Package

 

Components:

- Core
  + configuration
  + processes management
  + sockets creation
  + monitoring
  + logging
  + shared memory areas
  + ipc
  + cluster membership
- Request plugins
  (application server interfaces for various languages)(platforms: WSGI, PSGI, Rack, Lua WSAPI, CGI, PHP, Go ...)
- Gateways
  (implement load balancers, proxies and routers)
- Emperor
  (massive instances management and monitoring)
- Loop engines
  (events and concurrency, components can be run in preforking, threaded, asynchronous/evented)

Package:

main:

uwsgi                            # fast, self-healing application container server

uwsgi-plugin-python      # Python via uWSGI

other plugin:

uwsgi-plugin-cache
uwsgi-plugin-nagios
uwsgi-plugin-echo
uwsgi-plugin-admin

admin_plugin.so
/usr/bin/uwsgi_admin (pure C)
man uwsgi_admin

uwsgi-plugin-cache
...

 


uwsgi Install & testing

 

uwsgi Install:

apt-get install uwsgi uwsgi-plugin-python

yum install uwsgi uwsgi-plugin-python2         # Centos 7, v2.0.18

uwsgi testing:

uwsgi --version

2.0.18

foobar.py

def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    return ["Hello World"]

說明

1. By default, uWSGI will look for a callable called application

2. start_response and is the name the app will use internally to refer to the web server (uWSGI) callable that is sent in

Run Code

uwsgi --http-socket :9080 \
  --plugin /usr/lib64/uwsgi/python_plugin.so \
  --wsgi-file foobar.py

Output:

*** Starting uWSGI 2.0.18 (64bit) on [Mon Sep 27 16:13:55 2021] ***
...
Python version: 2.7.5 (default, Nov 16 2020, 22:23:17)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
...

當有 access 時:

[pid: 4108|app: 0|req: 2/2] 192.168.88.150 () {24 vars in 254 bytes} [Mon Sep 27 16:15:57 2021] 
  GET / => generated 11 bytes in 0 msecs (HTTP/1.1 200) 1 headers in 44 bytes (1 switches on core 0)

P.S.

安裝了 "uwsgi-plugin-python2" 才有 --wsgi-file

 


Plugin

 

uwsgi --plugins-list

*** uWSGI loaded generic plugins ***
corerouter

*** uWSGI loaded request plugins ***
100: ping
101: echo
--- end of plugins list ---

rpm -ql uwsgi-plugin-python2         # /usr/lib64/uwsgi/python_plugin.so

uwsgi --http-socket :9090 \
 --plugin /usr/lib64/uwsgi/python_plugin.so \
 --wsgi-file ./respond.py

 


Processes & Threads

 

# This will spawn 4 processes (each with 2 threads)

# A master process will respawn processes when they die

uwsgi_python --http-socket :9090 \
  --master --processes 4 --threads 2 \
  --wsgi-file foobar.py
 

P.S.

If you start uWSGI without threads, the Python GIL will not be enabled

 


Export uWSGI’s internal statistics as JSON

 

uwsgi ... --stats 127.0.0.1:9191

OR

uwsgi ... --stats /tmp/statsock

# 查看 result (json 來)

nc 127.0.0.1 9191

查看 status - uwsgitop

pip install uwsgitop

uwsgitop

 


Loading configuration files

 

uwsgi --ini http://uwsgi.it/configs/myapp.ini # HTTP

uwsgi --xml - # standard input

uwsgi --yaml fd://0 # file descriptor

uwsgi --json 'exec://nc 192.168.11.2:33000' # arbitrary executable

其他有用的 opts

# limit processes address space/vsz (unit: Mbyte)

--limit-as 128

#  daemonize uWSGI

-d|--daemonize

準備結構

mkdir /etc/uwsgi

# 用來改 code 的 user

useradd waf -m

chmod o+x /home/waf

mkdir /home/waf/logs

mkdir /home/waf/waf-panel{code,public_html/static} -p

# 用此 user 去 run code

useradd waf-web -d /home/waf/waf-panel

chown waf:waf-web /home/waf/waf-panel -R

chmod 2751 /home/waf/waf-panel -R

# Static 內容

chmod 2775 /home/waf/waf-panel/public_html -R

# Testing file

echo working > /home/waf/waf-panel/public_html/static/index.htm

ini example

/etc/uwsgi/waf.ini

[uwsgi]
module = wsgi:application

master = true
processes = 5

uid = waf-web
socket = /run/uwsgi/waf.sock
chown-socket = waf-web:nginx
chmod-socket = 660
vacuum = true

die-on-term = true

說明

vacuum

remove the socket when the process stops

die-on-term

Systemd and uWSGI have different ideas about what the SIGTERM signal should do to an application.

"die-on-term" so that uWSGI will kill the process instead of reloading it(respawn)

SIGQUIT 才是 uWSGI 離開的 SIGNAL

 


nginx 設定

 

main configure (one uwsgi one website)

/etc/nginx/sites-enabled/waf.conf

server {
  listen 8080;
  server_name waf.datahunter.org;
 
  access_log /home/waf/logs/access.log;
  error_log  /home/waf/logs/error.log;

  # You can use virtual directory like '/apps/' here, but remember that
  # you should matching 'urls' defined in your web.py application file
  location / {
    include uwsgi_params;
 
    # This should match the 'socket' entry of your uwsgi.xml
    # uwsgi_pass unix:/path/to/myapp.sock;
    uwsgi_pass 127.0.0.1:9090;

    # 由於採用獨立的 uwsgi instanse, 所以不用設定 UWSGI_CHDIR 及 UWSGI_SCRIPT
    # This is the absolute path to the folder containing your application
    # chdir() to the specified directory before managing the request.
    #uwsgi_param UWSGI_CHDIR /var/www/apps;
    # This is the name of your application file, minus the '.py' extension
    # uWSGI can be launched without passing it any application configuration
    #uwsgi_param UWSGI_SCRIPT index;
  }
  location /static {
    root   /home/waf/waf-panel/public_html/;
    index  index.html index.htm;
  }
}

file: uwsgi_params

uwsgi_param QUERY_STRING $query_string;
uwsgi_param REQUEST_METHOD $request_method;
uwsgi_param CONTENT_TYPE $content_type;
uwsgi_param CONTENT_LENGTH $content_length;
uwsgi_param REQUEST_URI $request_uri;
uwsgi_param PATH_INFO $document_uri;
uwsgi_param DOCUMENT_ROOT $document_root;
uwsgi_param SERVER_PROTOCOL $server_protocol;
uwsgi_param REMOTE_ADDR $remote_addr;
uwsgi_param REMOTE_PORT $remote_port;
uwsgi_param SERVER_ADDR $server_addr;
uwsgi_param SERVER_PORT $server_port;
uwsgi_param SERVER_NAME $server_name;

Static files

location /media {
  alias /var/lib/python-support/python2.6/django/contrib/admin/media;
}

if (!-f $request_filename) {
  uwsgi_pass uwsgicluster;
}

 


Automatically starting uWSGI on boot

 

/etc/uwsgi/apps-available/myapp.ini

[uwsgi]
logto = /var/log/uwsgi/test.log

#socket = /tmp/test.socket
socket = 127.0.0.1:9090

gid = www-data
uid = www-data

#vhost = true
#plugins-dir = <path_to_your_plugin_directory>
#plugins = python

# 當 nginx 沒有指定 "UWSGI_CHDIR" 及 "UWSGI_SCRIPT" 時, 那就要在這 ini 設定它
chdir = /var/www/apps/
wsgi-file = index.py

processes = 4
threads = 2

stats = 127.0.0.1:9191

log

... - WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x8b01800 pid: 27296 (default app)
... - *** uWSGI is running in multiple interpreter mode ***

Default configure:

# try to autoload appropriate plugin
autoload = true

master = true

workers = 2

# automatically kill workers on master's death
no-orphans = true

# bind to UNIX socket at /run/uwsgi/<confnamespace>/<confname>/socket
socket = /run/uwsgi/%(deb-confnamespace)/%(deb-confname)/socket

# set mode of created UNIX socket
chmod-socket = 660

# place timestamps into log
log-date = true

# identifier of uWSGI processes
uid = www-data
gid = www-data

Start / Stop Script

/etc/init.d/uwsgi

它會載入 /etc/default/uwsgi

RUN_AT_STARTUP=yes

VERBOSE=yes

# # Should init.d script print configuration file names while marking progress of it's execution
PRINT_CONFNAMES_IN_INITD_SCRIPT_OUTPUT=no

# inherited configuration file
INHERITED_CONFIG=/usr/share/uwsgi/conf/default.ini

每一個 app 的設定在放在此兩個 folder

/etc/uwsgi/apps-available

/etc/uwsgi/apps-enabled

P.S.

uwsgi yourfile.ini

 


Systemd

 

/etc/systemd/system/waf-panel.service

[Unit]
Description=uWSGI instance to serve waf-panel
After=syslog.target

[Service]
ExecStart=/usr/sbin/uwsgi \
          --ini /etc/uwsgi/waf-panel.ini
Type=notify
NotifyAccess=all
StandardError=syslog
Restart=always
RuntimeDirectory=uwsgi
# 由於在 ini 內會設定轉 user/group 所以不用設定
#User=waf-web
#Group=waf-web

[Install]
WantedBy=multi-user.target

systemctl daemon-reload

systemctl start waf-panel

systemctl status waf-panel

RuntimeDirectory

Putting sockets in /run/

On a modern system, /run/ is mounted as a tmpfs and is the right place to put sockets and pidfiles into.
To have systemd automatically create a /run/uwsgi/ subdirectory with the correct user/group ownership,
 as well as cleaning up the directory when the daemon is stopped, add

RuntimeDirectory=uwsgi

 


Cluster App Server 設定 (Nginx)

 

 uwsgi_pass 127.0.0.1:9090;

改成

upstream uwsgicluster {
  server unix:///tmp/uwsgi.sock;
  server 192.168.1.235:3031;
  server 10.0.0.17:3017;
  ...
}

server {
  ...
  location / {
    ...
    uwsgi_pass uwsgicluster;
  }
}

 


uwsgi protocol magic variables

 

Nginx:

uwsgi_param <name> <value>

# http | https

UWSGI_SCHEME

# Load the specified script as a new application

UWSGI_SCRIPT

# Dynamically set the Python Virtualenv support

UWSGI_PYHOME

# chdir() to the specified directory before managing the request.

UWSGI_CHDIR

# Set the specified environment variable for a new dynamic app.

UWSGI_SETENV

uwsgi_param UWSGI_SETENV DJANGO_SETTINGS_MODULE=myapp.settings;

# Check the uWSGI cache for a specified key.
# If the value is found, it will be returned as raw HTTP output instead of the usual processing of the request.

UWSGI_CACHE_GET

uwsgi_param UWSGI_CACHE_GET $request_uri;

 


uwsgi configure files(ini)

 

Placeholders - %(string)

my_funny_domain = uwsgi.it
socket = /tmp/sockets/%(my_funny_domain).sock

Math

[uwsgi]
foo = 17
bar = 30
; total will be 50
total = %(foo + bar + 3)

‘@’ magic

# expand text files embraced in @(FILENAME)

nodename = @(/etc/hostname)

Magic variables

you can access them in Python via uwsgi.magic_table

%n     will be replaced with the name of the config file, without extension

%p     the absolute path of the configuration file

%u     uid of the user running the process
%U     username

%g     gid of the user running the process
%G     group name

i.e.

[uwsgi]
socket = /tmp/%n.sock
module = werkzeug.testapp:test_app
processes = 4
master = 1

–strict

You can easily add non-existent options to your config files
(as placeholders, custom options, or app-related configuration items).

This is a really handy feature, but can lead to headaches on typos.

The strict mode (–strict) will disable this feature, and only valid uWSGI options are tolerated.

Fallback configuration

uwsgi --fallback-config safe.ini --uid 1000 --http-socket :80

Configuration logic

if-not-dir

i.e.

if-not-dir = /run/%n/
 hook-asap = mkdir:/run/%n
 hook-as-root = chown:/run/%n %(uid) %(gid)
end-if =

hook-asap

hook-asap hook:args

run directly after configuration file has been parsed

as-root

run soon before privileges drop

as-user-atexit

run before shutdown of the instance. it is non-fatal.

pidfile = /run/%n/%n.pid
hook-as-user-atexit = unlink:%(pidfile)

for

[uwsgi]
master = true
; iterate over a list of ports
for = 3031 3032 3033 3034 3035
 # %(_) to the current iterated value
 socket = 127.0.0.1:%(_)
endfor =
module = helloworld

if-exists

[uwsgi]
http = :9090
; redirect all requests if a file exists
if-exists = /tmp/maintainance.txt
 route = .* redirect:/offline
endif =

if-env

[uwsgi]
if-env = PATH
 print = Your path is %(_)
 check-static = /var/www
endif =
socket = :3031

Other hook

call - call the specified symbol ignoring return value

callret - call the specified symbol expecting an int return. anything != 0 means failure

cd - convenience handler, same as call:chdir <directory>

print - convenience handler, same as calling the uwsgi_log symbol

write - (from uWSGI 1.9.21), write a string to the specified file using write:<file> <string>

 


uwsgi Protocol (binary protocol)

 

4 bytes of a uwsgi packet describe the type of the data contained by the packet

 


Signals for controlling uWSGI

 

SIGHUP       gracefully reload all the workers and the master process (–reload)

SIGTERM     brutally reload all the workers and the master process     
                   (use –die-on-term to respect the convention of shutting down the instance)

SIGINT        immediately kill the entire uWSGI stack     (–stop)

Example:

uwsgi –reload /tmp/project-master.pid

OR

kill -HUP `cat /tmp/project-master.pid`

* When running with the master process mode,
  the uWSGI server can be gracefully restarted without closing the main sockets.

 


Logging

 

log to file

./uwsgi -s :3031 -w simple_app --daemonize /tmp/mylog.log

OR

./uwsgi -s :3031 -w simple_app --logto /tmp/mylog.log

OR

UDP logging (logs to another machine to offload disk I/O)

./uwsgi -s :3031 -w simple_app --logto 192.168.0.100:1717

OR

log to an unconnected UNIX socket

uwsgi --socket :3031 --logger socket:/tmp/uwsgi.logsock

OR

syslog

uwsgi --socket :3031 --logger syslog:uwsgi1234

OR

remote syslog

uwsgi --socket :3031 --logger rsyslog:12.34.56.78:12345,uwsgi1234

logrotate

/etc/logrotate.d/uwsgi

"/var/log/uwsgi/*.log" "/var/log/uwsgi/*/*.log" {
  copytruncate
  daily
  rotate 5
  compress
  delaycompress
  missingok
  notifempty
}

 


webpy

 

webpy 的 Code:

/home/waf/waf-panel/code/code.py

#!/usr/bin/env python

import web

urls = (
    "/", "index",
)

class index:
    def GET(self):
        return "Test OK"

app = web.application(urls, globals(), autoreload=True)

if ( __name__ == "__main__" ):
    app.run()
else:
    application = app.wsgifunc()

"application =" 係一定要加的 !!

uwsgi Setting

uwsgi: /etc/uwsgi/waf-panel.ini

[uwsgi]
#socket = 127.0.0.1:9080
master = true
processes = 2
threads = 2

uid = waf-web
socket = /run/uwsgi/waf.sock
chown-socket = waf-web:nginx
chmod-socket = 660
vacuum = true

# PID
pidfile = /run/%n/%n.pid
hook-as-user-atexit = unlink:%(pidfile)

die-on-term = true

# rpm -ql uwsgi-plugin-python2  =>  /usr/lib64/uwsgi/python_plugin.so
# "plugin" 會自動加上 "_plugin.so"
plugins-dir = /usr/lib64/uwsgi
plugin = python

# program 的位置
chdir = /home/waf/waf-panel/code
wsgi-file = code.py

stats = 127.0.0.1:9081

測試

uwsgi --ini /etc/uwsgi/waf.ini

Output

detected max file descriptor number: 1024

# master = true, processes = 2, threads = 2

spawned uWSGI master process (pid: 4877)
spawned uWSGI worker 1 (pid: 4878, cores: 2)
spawned uWSGI worker 2 (pid: 4879, cores: 2)

# stats = 127.0.0.1:9081

*** Stats server enabled on 127.0.0.1:9081 fd: 11 ***

ps U waf-web

  PID TTY      STAT   TIME COMMAND
 4884 pts/4    S+     0:00 uwsgi --ini /etc/uwsgi/waf.ini
 4885 pts/4    Sl+    0:00 uwsgi --ini /etc/uwsgi/waf.ini
 4886 pts/4    Sl+    0:00 uwsgi --ini /etc/uwsgi/waf.ini

 


Python in virtualenv

 

/etc/uwsgi/waf.ini

...
plugins-dir = /usr/lib64/uwsgi
plugin = python
virtualenv = /home/waf/waf-panel/public_html/webpy-env

 


VirtualHosting mode(vhost)

 

/etc/uwsgi/myapp.ini

# enable virtualhosting mode (based on SERVER_NAME variable)
vhost = true

 


進階使用

 

UWSGI_TOUCH_RELOAD

location / {
  include uwsgi_params;
  uwsgi_param UWSGI_TOUCH_RELOAD /tmp/touchme.foo;
  uwsgi_pass /tmp/uwsgi.sock;
}

UWSGI_PYHOME

Dynamically set the Python Virtualenv support for a dynamic application

SCRIPT_NAME

the variable used to select a specific application. Hosting multiple apps in the same process

SCRIPT_NAME is the variable used to select a specific application.

 


Router

 

Embedded mode:

# spawn a HTTP server on port 9080 that forwards requests to a pool of 4 uWSGI workers

uwsgi --master --http 127.0.0.1:9080\
  --module mywsgiapp --processes 4

Standalone mode:

# This will spawn a HTTP router that will forward requests to the uwsgi socket

uwsgi --master --http 127.0.0.1:9080
  --http-to /tmp/uwsgi.sock

 


Auto reload python

 

Python auto-reloader enabled

py-autoreload = 2

 


Troubleshoot

 

<2> unable to load app 0 (mountpoint='') (callable not found or import error)

/etc/uwsgi/apps-enabled/webpy.ini

[uwsgi]
...
vhost = true

/etc/nginx/sites-enabled/webpy

server {
  ...
  location / {
    ...
    include uwsgi_params;
    uwsgi_pass 127.0.0.1:9090;
    uwsgi_param UWSGI_CHDIR /home/tim/cms;
    uwsgi_param UWSGI_SCRIPT code;
  }
}

 


Doc

 

http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html

 

 

Creative Commons license icon Creative Commons license icon