Open vSwitch

由 datahunter 在日, 12/08/2012 - 23:43 發表

最後更新: 2017-05-22

介紹

open vswitch 是一個 multilayer virtual switch 來, 支援 vlan, qos, netflow, openflow 等功能

它可以做到 Host A 的 VM1 與 Host B 的 VM2 組成 vlan 的效果

不少 VM 的 infrastructure 用了它, 比如 OpenStack, openQRM, OpenNebula

我們可以用它實現 SDN (Software Defined Networking)

License: Apache 2 license

安裝

以下是 Ubuntu12.04 上的安裝

它一共有 4 個包, 分別是:

openvswitch-switch <-- 主包
- ovs-dpctl
- ovs-vlan-test
- ovsdb-tool
- ovs-pcap
- ovs-vsctl
- ovs-tcpundump
openvswitch-datapath-dkms <-- kernel module
openvswitch-brcompat <-- 令 ovs 與傳統的 brctl 相容的 kernel module
openvswitch-common
- ovs-appctl
- ovs-benchmark
- ovs-pki
- ovs-ofctl
- ovs-parse-leaks
- ovs-appctl
- ovsdb-client
openvswitch-controller <-- 非必要

移除不要的包:

aptitude purge ebtables

安裝:

apt-get install openvswitch-switch openvswitch-datapath-dkms openvswitch-brcompat

Daemon 的狀態:

/etc/init.d/openvswitch-switch status

ovsdb-server is running with pid 1111
ovs-vswitchd is running with pid 1120
ovs-brcompatd is not running

設定

載入 brcom

/etc/default/openvswitch-switch

BRCOMPAT=yes

reboot 後測試相容模式有無生效:

lsmod | grep brcompat

brcompat               13512  0
openvswitch            84038  3 brcompat

OVS + NIC 的設定

auto eth0
allow-br0 eth0
iface eth0 inet manual
        pre-up ifconfig $IFACE up
        post-down ifconfig $IFACE down
        ovs_bridge br0                 # 指出此 port 所屬的 bridge
        ovs_type OVSPort               # 其他設定 OVSBridge, OVSPort, OVSIntPort or OVSBond


auto br0
allow-ovs br0                                         # 設定 br0 是 OVS bridges 來
iface br0 inet static
        address 192.168.123.10
        netmask 255.255.255.0
        network 192.168.123.0
        broadcast 192.168.123.255
        gateway 192.168.123.1
        dns-nameservers 8.8.8.8 8.8.4.4
        ovs_type OVSBridge
        ovs_ports eth0                  # 列出所有屬於此 bridge 的 port

auto lxcbr0
allow-ovs lxcbr0
iface lxcbr0 inet static
    address 192.168.1.1
    netmask 255.255.255.0
    ovs_type OVSBridge

OVS + Bonding 的設定

allow-ovs br0
iface br0 inet static
    address 192.168.1.1
    netmask 255.255.255.0
    ovs_type OVSBridge
    ovs_ports bond0

allow-br0 bond0
iface bond0 inet manual
    ovs_bridge br0
    ovs_type OVSBond
    ovs_bonds eth0 eth1
    ovs_options bond_mode=balance-tcp lacp=active

建立 vswitch

關係:

ovs-vsctl ---> ovs-vswitchd <---> ovsdb-server ---> conf.db(/etc/openvswitch)

ovs-ofctl ---> ovs-controller

指令:

# ovs-vsctl add-br br0

# ovs-vsctl add-port br0 eth0

# ovs-vsctl list-br

br0
lxcbr0

啟用:

service openvswitch-switch restart

output:

 * Killing ovs-brcompatd (1530)
 * Killing ovs-vswitchd (1513)
 * Killing ovsdb-server (1504)
 * Starting ovsdb-server
 * Configuring Open vSwitch system IDs
 * Starting ovs-vswitchd
 * Starting ovs-brcompatd
 * Enabling gre with iptables

P.S.

只有 Database 及 interfaces 同時設定好那 interface 才會啟動 !!

Command - ovs-vsctl

Version

ovs-vsctl -V

ovs-vsctl (Open vSwitch) 1.4.0+build0
Compiled Feb 18 2013 13:13:22

list-br

ovs-vsctl list-br

br0
lxcbr0

add-br

add-br BRIDGE

del-br

del-br BRIDGE

list-ports

ovs-vsctl list-ports BRIDGE

eth0
lamp
nginx
sshgw

add-port

ovs-vsctl add-port BRIDGE PORT

i.e.

ovs-vsctl add-port br0 tap0

del-port [BRIDGE] PORT

get-controller BRIDGE

get-fail-mode BRIDGE

get-manager

DB 內資料:

ovs-vsctl show

f56dd7f1-7eb5-4dc9-8246-af234b364ab8
    Bridge "br0"
        Port xpenology
            Interface xpenology
                error: "could not open network device xpenology (No such device)"
        Port "br0"
            Interface "br0"
                type: internal
        Port sshgw
            Interface sshgw
        Port "eth0"
            Interface "eth0"
        Port nginx
            Interface nginx
        Port "tap0"
            Interface "tap0"
        Port lamp
            Interface lamp
    ovs_version: "2.5.2"

brcompat

/usr/share/openvswitch/scripts/ovs-ctl: 75: /usr/share/openvswitch/scripts/ovs-ctl: -e: not found
* Inserting brcompat module
* could not load brcompat module, disabling bridge compatibility

/usr/share/openvswitch/scripts/ovs-ctl: 75: /usr/share/openvswitch/scripts/ovs-ctl: -e: not found
* Inserting brcompat module
* could not load brcompat module, disabling bridge compatibility
* ovsdb-server is already running
* ovs-vswitchd is already running

insert_brcompat_mod_if_required () {
    if test -e /sys/module/brcompat_mod || -e /sys/module/brcompat; then
        return 0
    fi
    action "Inserting brcompat module" do_modprobe brcompat
}

一定要安好 linux-headers 才 complie 到.

Troubleshoot

問題1: boot 機時要等一段長時間 timeout 才開到機

解決1: 把 /etc/init/failsafe.conf 內的 "Waiting for network configuration..." 之間的 sleep 改為 3 秒

Qos 限速

VM (eth0 <---> veth-ubuntu) --- ||OVS|| ---> Network

--------------- traffic --------------->

VM to 1 Mbps, run:

ovs-vsctl set Interface veth-ubuntu ingress_policing_rate=1000 <-- 單位: kbps, does not queue packets(drop)

ovs-vsctl set Interface veth-ubuntu ingress_policing_burst=100 <-- 跳動空間, 1100

sFlow

原理:

agent --- UDP --->  sFlow collector

  VM
   |
--------------
| eth0 | eth1 |-----mgt
--------------
   |
  WAN

collector:

sFlowTrend

http://www.inmon.com/products/sFlowTrend.php

Java 6.0

COLLECTOR_IP=10.0.0.1
COLLECTOR_PORT=6343
AGENT_IP=eth1
HEADER_BYTES=128
SAMPLING_N=64
POLLING_SECS=10

ovs-vsctl — –id=@sflow create sflow agent=${AGENT_IP} \
target=\"${COLLECTOR_IP}:${COLLECTOR_PORT}\" \
header=${HEADER_BYTES} \
sampling=${SAMPLING_N} \
polling=${POLLING_SECS} \
— set bridge br0 sflow=@sflow

ovs-vsctl list sflow

ovs-vsctl remove bridge br0 sflow $SFLOWUUID

測試

tcpdump -ni eth1 udp port 6343

Win32 agent

http://host-sflow.sourceforge.net

vlan

Host1: VM1,VM3
Host2: VM2,VM4
VLAN 1: VM1,VM3
VLAN 2: VM2,VM4

Host1

ovs-vsctl add-br br0
ovs-vsctl add-port br0 eth0
ovs-vsctl add-port br0 tap0 tag=1
ovs-vsctl add-port br0 tap1 tag=2

Host2