介紹

varnish 是一個 CDN 級的 Web Reverse Proxy. 它的設計是在 64-bit multi-core 的環境運作的.

它有兩個特點

1. 用 kernel 的 swap 去管理 memory
2. shared memory log

Home Page:

https://www.varnish-cache.org

 

---

安裝

ubuntu-12.04 install:

Package: Version: 3.0.2-1

apt-get install varnish

Centos-6 install:

http://repo.varnish-cache.org/redhat/varnish-3.0/el6/

設定 - VCL:

• 設定檔: /etc/varnish/default.vcl
• CLI Crontrol PW: /etc/varnish/secret

基本測試:

varnishd -f /usr/local/etc/varnish/default.vcl \
            -s malloc,1G \
            -T 127.0.0.1:2000 \
            -a 0.0.0.0:8080

# -f configuration
# -s storage type Varnish should use for storing its content.
       (Varnish has a overhead of about 1k per object. )
# -T built-in text-based administration interface
# -a incomming HTTP requests. (Default: 80)

/etc/default/varnish:

START=yes
# Maximum number of open files
NFILES=131072
# Maximum locked memory size
MEMLOCK=82000

DAEMON_OPTS="-a :6081 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"

/etc/varnish/default.vcl:

# 設立 兩個 backend

backend default {                     <--- 此 "default" 沒有特別意思, 第一個 backend 就是 default
    .host = "192.168.88.18";
    .port = "80";
}

backend myweb {
        .host = "192.168.88.18";
        .port = "80";
}

sub vcl_recv {
        #for domain and datahunter.org
        if (req.http.host ~ "^datahunter.org$") {
                set req.backend = myweb;
                return (lookup);
        }
}

# Reload VCL

service varnish reload

# 所有 Cache 都會不見了

service varnish restart

# 查看即時設定

varnishadm param.show <parameter>

# 即時設定

varnishadm param.set ....

 

---

network CLI

 

啟動時加入

varnishd ... -S secret-file -T 127.0.0.1:2000 ...

-S secret-file

功能:

• update the file on the fly

使用:

telnet localhost 6082            <-- default: 127.0.0.1:6082

auth <YOUR_PW>

 

---

狀態

 

varnishstat     <--- Munin poll it

# 即時 update 的現在情況

-l    Lists the available fields to use
-f    A comma separated list of the fields to display.
-j    Displays the result as JSON.

Munin:

• Backend traffic
• Hit rates
• Memory usage
• Number of objects
• Object expunging
• Request rates
• Thread status
• Transfer rates
• Varnish uptime

varnishtop

reads the shared memory logs and presents a continuously updated list

varnishtop -i rxurl

will show you what URLs are being asked for by the client.

varnishtop -i txurl

will show you what your backend

varnishhist

utility reads varnishd(1) shared memory logs and presents a continuously

 

---

Varnish logs to a shared memory segment.

first column: same number are part of the same HTTP transaction.
second column: is the tag of the log message.
    Rx indicate Varnish is recieving data and
    Tx indicates sending data.

client (c) or to/from the backend (b).

 

---

X-Forwarded-For

 

sub vcl_recv {
  # Set the X-Forwarded-For header so the backend can see the original
  # IP address. If one is already set by an upstream proxy, we'll just re-use that.
  if (client.ip ~ upstream_proxy && req.http.X-Forwarded-For) {
    set req.http.X-Forwarded-For = req.http.X-Forwarded-For;
  } else {
    set req.http.X-Forwarded-For = regsub(client.ip, ":.*", "");
  }
}

 

---

Compression

 

In Varnish 3.0 compression defaults to "on"

When Varnish then issues a backend request the Accept-Encoding will then only consist of "gzip".

停用:

http_gzip_support = false

uncompress objects before storing it in memory by setting

do_gunzip = true

the default configuration, not cache a object coming from the backend with a Set-Cookie header present. Also, if the client sends a Cookie header, Varnish will bypass the cache and go directly to the backend.(hit-for-pass object)

 

---

Tools

 

varnishreplay

parses varnish logs and attempts to reproduce the traffic.

varnishhist

Varnish request histogram

• "|"    Hits
• "#"    hash

varnishsizes

Varnish object size request histogram

logarithmic representation of bytes

varnishncsa

Display Varnish logs in Apache / NCSA combined log format

192.168.123.1 - - [25/Apr/2013:16:41:24 +0800] "GET http://datahunter.org/ HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"

varnishreplay

The varnishreplay utility parses varnish logs and attempts to reproduce the traffic
(typcally used to warm up caches)

varnishreplay [-D] -a address:port -r file

varnishstat

Varnish Cache statistics

-1        print the statistics once and exit.
-l     Lists the available fields to use with the -f option.

varnishtest

Test program for Varnish

varnishtop

continuously updated list of the most commonly occurring log entries.