![首頁 Logo Keith ]p !!](/themes/mytheme/logo.png){kind=logo}
最後更新: 2019-04-17
目錄
- 查看系統對 lxc 的支援
- LXC 的 Namespace (kernel 3.2)
- 安裝
- 建立 - lxc-create
- system 與 application containers 的分別
- 設定
- Host Network Configure
- VPS Network Configure
- lxc.arch
-
資源
- CPU,
- RAM,
- Device,
- pts 與 tty,
- mount 與 rootfs,
- lxc.autodev,
- lxc.cgroup - Capabilities
- 啟動
- Console
- apparmor
- 啟動大量 VPS
- Debug
- Whole Configure
- Run time 設定
- Logging
- lxc-stop
- Attach
- lxc-device
- restricting container view of dmesg
- Template Turning
- 參考
- 其他工具
查看系統對 lxc 的支援
lxc-checkconfig
Found kernel config file /boot/config-3.2.0-23-generic-pae
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled
--- Control groups ---
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled
--- Misc ---
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
File capabilities: enabled
...
必須的 LXC Namespace
- Utsname
- Ipc
- Pid
- User
- Network
安裝
# U16
apt-get install lxc1
建立 - lxc-create
用 ubuntu template 建立一個名叫 my-container 的 vps:
* same version and architecture as your machine
* 新建立的 vps default 存放在 /var/lib/lxc
lxc-create -t ubuntu -n my-container
Output:
............................................................... ## # The default user is 'ubuntu' with password 'ubuntu'! # Use the 'sudo' command to run tasks as root in the container. ## 'my-container' template installed 'my-container' created
Templates 與 templace script
可用的 templates 在 /usr/share/lxc/templates 內
- lxc-debian
- lxc-ubuntu-cloud
- lxc-busybox
- lxc-fedora
- lxc-sshd
- lxc-ubuntu <-- 與 EC2/cloud images 相同
- ..............
查看 template 的選項:
lxc-create -t ubuntu -h
/usr/share/lxc/templates/lxc-ubuntu -h|--help [-a|--arch] [-b|--bindhome <user>] [-d|--debug]
[-F | --flush-cache] [-r|--release <release>] [-v|--variant] [ -S | --auth-key <keyfile>]
[--rootfs <rootfs>] [--packages <packages>] [-u|--user <user>] [--password <password>]
[--mirror <url>] [--security-mirror <url>]
release: the ubuntu release (e.g. precise): defaults to host release on ubuntu, otherwise uses latest LTS
variant: debootstrap variant to use (see debootstrap(8))
bindhome: bind <user>'s home into the container
The ubuntu user will not be created, and <user> will have
sudo access.
arch: the container architecture (e.g. amd64): defaults to host arch
auth-key: SSH Public key file to inject into container
packages: list of packages to add comma separated
mirror,security-mirror: mirror for download and /etc/apt/sources.listubuntu template:
arch= amd64|i386
FS Backend - LVM
# 系統上有一個叫 lxc 的 LVM Group 時, 那用它那立一個 10G 的 my-ubuntu LV, 並且用 i386:
lxc-create -n my-ubuntu -t ubuntu -B lvm --vgname myvg --fssize 10G --fstype ext4 -- --arch i386
- -B backing store lvm | none | btrfs (default: none(/var/lib/lxc/<name>/rootfs))
- lvname
- vgname default: lxc
- fstype default: ext4
- fssize default: 1G
設定 lvm 的 default group
/etc/lxc/lxc.conf
lxc.bdev.lvm.vg = myvgFS Backend - loop
lxc-create -n C1 -t ubuntu -B loop
config
lxc.rootfs = loop:/var/lib/lxc/C1/rootdev lxc.rootfs.backend = loop
file rootdev
rootdev: Linux rev 1.0 ext4 filesystem data, UUID=fcbed240-8d2b-42e1-91bb-0777b9c0d621 (extents) (large files) (huge files)
ls -lh rootdev
-rw------- 1 root root 1.1G Apr 25 11:56 rootdev
在 Container 內
df -h
Filesystem Size Used Avail Use% Mounted on /dev/loop0 976M 344M 566M 38% /
# 設定 fstype 及 fssize
# rootdev 11g
# 可以用 "--fstype btrfs"
lxc-create -n C1 -t ubuntu -B loop --fstype ext4 --fssize 10g
Templace Cache
Cache: /var/cache/lxc/OS_NAME/ARCH/VERSION/partial/
安裝舊版 LXC
/usr/share/lxc/templates/lxc-centos # script 有 "--repo" 設定
lxc-create -t centos -n centos6 -- --repo https://vault.centos.org/6.10/os/i386
system 與 application containers 的分別
# system container
lxc-start -n name init
# only creates separate namespaces needed to isolate a single application.
lxc-execute -n name cmd
設定
設定檔
- /etc/init/lxc.conf # U16
# Boot 機時會自動啟動 vps LXC_AUTO="true"
- /etc/default/lxc-net # U16
# lxc 會為 containers 建立 NATed bridge # 並以 dhcp 派 IP USE_LXC_BRIDGE="true"
Storage
- /var/lib/lxc
- /var/cache/lxc
更改:
mkdir /srv/lxclib /srv/lxccache
rm -rf /var/lib/lxc /var/cache/lxc
ln -s /srv/lxclib /var/lib/lxc
ln -s /srv/lxccache /var/cache/lxc/etc/default/lxc:
LXC_AUTO="true"
USE_LXC_BRIDGE="true" <--- lxc 內置的 network 功能 (Private Network)
LXC_BRIDGE="lxcbr0" <--- 當前一項是 true 時, 那 lxcbr0 會自動建立
LXC_ADDR="10.0.3.1"
LXC_NETMASK="255.255.255.0"
LXC_NETWORK="10.0.3.0/24"
LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
LXC_DHCP_MAX="253"
LXC_SHUTDOWN_TIMEOUT=120 <-- 當 Server 關機時, 待多久才 kill 其他的 vps (service lxc restart)
/etc/lxc/lxc.conf
所有 vps 的 default Network 設定值(只適用於 lxc-create)
lxc.network.type=veth lxc.network.link=lxcbr0 <-- 對應 /etc/default/lxc 內的 LXC_BRIDGE lxc.network.flags=up
Host Network Configure
Bridge Network Setting:
/etc/network/interfaces
二選一設定
# Wan Bridge
auto eth0
iface eth0 inet manual
address 0.0.0.0
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
bridge_waitport 0
bridge_fd 0
# Internal Bridge
auto lxcbr0
iface lxcbr0 inet static
address 192.168.200.254
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_waitport 0
bridge_fd 0
bridge_waitport 0
VPS Network Configure
lxc.network.type
lxc.network.type=empty
- empty
- veth (default) <-- 此 veth 將會 bridge 到 lxc.network.link
-
phys <-- an already existing interface specified by lxc.network.link
The interface "moves" from one network namespace to the new one - vlan <-- lxc.network.vlan.id
- macvlan <-- lxc.network.macvlan.mode (Default: private), 另有 vepa, bridge
Full Example:
一般:
lxc.network.flags = up lxc.network.link = lxcbr0 lxc.network.ipv4 = 10.2.3.4/24 <-- vps 有兩組 ip (沒用的..) lxc.network.ipv4 = 192.168.10.125/24
名稱:
lxc.network.veth.pair = veth-lamp <-- host 見到的網卡名 lxc.network.name = eth0 <-- vps 內見到的網卡名, 如 eth0, eth1, eth2
進階:
# 第一組 Network lxc.network.type = veth <--- 用 lxc.network.type 為分隔 lxc.network.flags = up lxc.network.link = br0 lxc.network.hwaddr = 4a:49:43:49:79:bf lxc.network.ipv4 = 10.2.3.5/24 10.2.3.255 lxc.network.veth.pair= <-- 當沒有注名時, 會用此格調 veth?????? # 第二組 Network lxc.network.type = phys lxc.network.flags = up lxc.network.link = dummy0 lxc.network.hwaddr = 4a:49:43:49:79:ff lxc.network.ipv4 = 10.2.3.6/24
mini:
lxc.network.type=veth lxc.network.link=br0 lxc.network.flags=up lxc.network.hwaddr = 00:16:3e:64:f3:11 lxc.network.name = eth0 lxc.network.veth.pair = veth-u12
P.S.
所有設定都要 shutdown vps 之後 start 才可套用 !!
lxc.arch
lxc.arch
- x86
- i686
- x86_64
- amd64
資源
CPU:
lxc.cgroup.cpuset.cpus = 0,3 <-- 設定 vps 可以用 CPU 的 core 0 及 3, default 只有 core 0 可用
/sys/fs/cgroup/cpuset/lxc/<vps>/cpuset.cpus
lxc.cgroup.cpu.shares = 1024 <-- Default, 比例來
/sys/fs/cgroup/cpu/lxc/<vps>/cpu.shares
即時修改
echo 1,2,3 > /sys/fs/cgroup/cpuset/lxc/<vps>/cpuset.cpus
echo 0-7 > /sys/fs/cgroup/cpuset/lxc/<vps>/cpuset.cpus
RAM:
# RAM = 512M lxc.cgroup.memory.limit_in_bytes = 512M # RAM + SWAP = 1G lxc.cgroup.memory.memsw.limit_in_bytes = 1G
* "lxc.cgroup.memory.memsw.limit_in_bytes" is Reached
=> start firing up the OOM killer and killing stuff inside the container
Device:
lxc.cgroup.devices.deny = a lxc.cgroup.devices.allow = c 1:3 rw lxc.cgroup.devices.allow = b 8:0 rw
查看所有 device:
lxc-cgroup -n debian devices.list
c 1:3 rwm c 1:5 rwm c 5:1 rwm c 5:0 rwm c 4:0 rwm c 4:1 rwm c 1:9 rwm c 1:8 rwm c 136:* rwm c 5:2 rwm c 254:0 rwm
pts 與 tty
lxc.pts = 16
lxc.tty = 4
lxc.console <-- path to a file where the console output will be written.('none' will simply disable the console.)
!!! 注意數量 !!!
The number of getty(tty)/ssh(pts) in the container should not be greater than the number of value in configuration file,
otherwise the excess getty sessions will die and respawn indefinitly giving annoying messages
PTY allocation request failed on channel 0 mesg: ttyname failed: Inappropriate ioctl for device
LXC consoles are provided through Unix98 PTYs created on the host and
bind-mounted over the expected devices in the container.
By default, they are bind-mounted over /dev/console and /dev/ttyN.
# In VPS
mount | grep devpts
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=16) devpts on /dev/ptmx type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=16) devpts on /dev/tty1 type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=16) devpts on /dev/tty2 type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=16) devpts on /dev/tty3 type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=16) devpts on /dev/tty4 type devpts (rw,nosuid,noexec,relatime,gid=100005,mode=620,ptmxmode=666,max=16)
mount 與 rootfs
rootfs - VM 的 File 在那裡
lxc.rootfs = /dev/lxc/debian # LVM
OR
lxc.rootfs = /lxc/u12/rootfs # Foldermount - VM 要有的 mount point
# specify a file location in the fstab format
lxc.mount = /lxc/u12/fstab
OR
# 直接加 entry
lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry = sysfs sys sysfs defaults,ro 0 0fstab 內容
proc proc proc nodev,noexec,nosuid 0 0 sysfs sys sysfs defaults,ro 0 0
lxc.autodev
lxc.autodev = 1
after mounting the container's rootfs LXC will mount a fresh tmpfs under /dev and fill in a minimal set of initial devices.
This is generally required when starting a container containing a "systemd" based "init"
(limited to 500K by default, unless defined in lxc.autodev.tmpfs.size)
(Additional devices in the containers /dev directory may be created through the use of the lxc.hook.autodev hook.)
df -h
Filesystem Size Used Avail Use% Mounted on none 492K 0 492K 0% /dev
lxc.cgroup.[subsystem name]
Capabilities
lxc.cap.drop # space separation items
Example
lxc.cap.drop = sys_admin mac_override sys_module
DOC
https://datahunter.org/capabilities
MySetting
# ubuntu 12 要它才 start 到 # lxc.cap.drop = sys_admin # 在 vps 內行 reboot 要它 # lxc.cap.drop = sys_boot # ssh 要它才 start 到 #lxc.cap.drop = sys_chroot # dhcp, iptables 及 tcpdump 要用它 # lxc.cap.drop = net_raw # U16 要有它們才 login 到, 否則一直會 login fail # lxc.cap.drop = audit_control audit_write # 最好有它, 因為 /dev 下有機會少左野 lxc.cap.drop = mknod #### Capabilities #### lxc.cap.drop = sys_time sys_module lxc.cap.drop = sys_tty_config sys_resource lxc.cap.drop = sys_pacct sys_ptrace lxc.cap.drop = sys_rawio lxc.cap.drop = mac_admin mac_override lxc.cap.drop = setfcap setpcap
net_raw
root@ubuntu:~# tcpdump -i eth0 tcpdump: eth0: You don't have permission to capture on that device root@ubuntu:~# iptables -nL iptables v1.4.12: can't initialize iptables table `filter': Permission denied (you must be root) Perhaps iptables or your kernel needs to be upgraded.
Remark
dropping sys_admin and net_admin isn't very practical, you won't make your container much safer,
as root in the container will be able to re-grant itself any dropped capability
啟動
lxc-start -n myvminlxc -f myvm.conf
Useful Options
- -o, --logfile=FILE # Output log to FILE instead of stderr
- -l, --logpriority=LEVEL # Set log priority to LEVEL
- -d, --daemon # Daemonize the container (default)
Console
# 連進 VPS - debian 的 console
lxc-console -n debian
# Specify the tty number to connect to or 0 for the console.
# If not specified the next available tty number will be automatically chosen by the container.
lxc-console -n debian -t 2
Type <Ctrl+a q> to exit the console Debian GNU/Linux 6.0 tty2 login:
Remark
如果 VPS 內只有 console 而沒有 ttyX,
ps a | grep [t]ty
146 console Ss+ 0:00 /sbin/agetty --noclear --keep-baud console 115200 38400 9600 linux那一定要用 "-t 0"
Apparmor
Ubuntu 16.04 自帶 4 個 apparmor profiles, 它們由 lxc-common package 提供.
Profiles 存放在 /etc/apparmor.d/lxc 裡, 它們都是以 lxc- 開頭
i.e.
/etc/apparmor.d/lxc/lxc-default
Default profiles
# 當 vps 沒有設定 "lxc.aa_profile" 時, 那就會用此設定
lxc-container-default
Container nesting profiles
/etc/apparmor.d/lxc/lxc-with-nesting
Checking: lxc-ls --fancy --nesting
mounting profiles
# same as default but allows mounting ext*, xfs and btrfs file systems
lxc-container-default-with-mounting
unconfined profiles
# a special value which will disable apparmor support for the container
unconfined
reload 或 add profile:
cat /etc/apparmor.d/lxc-containers | apparmor_parser -r
OR
apparmor_parser -r /etc/apparmor.d/lxc-containers
OR
/etc/init.d/apparmor reload
Ubuntu mini profile
VPS Config
lxc.aa_profile = lxc-u12-profile
/etc/apparmor.d/lxc/lxc-u12
# Do not load this file. Rather, load /etc/apparmor.d/lxc-containers, which
# will source all profiles under /etc/apparmor.d/lxc
profile lxc-u12-profile flags=(attach_disconnected,mediate_deleted) {
network,
capability,
file,
umount,
# ignore DENIED message on / remount
deny mount options=(ro, remount) -> /,
# allow tmpfs mounts everywhere
mount fstype=tmpfs,
# deny mounts everywhere
deny mount fstype=mqueue,
deny mount fstype=fuse.*,
deny mount fstype=devpts,
#deny mount fstype=binfmt_misc,
deny mount fstype=efivarfs,
mount fstype=binfmt_misc -> /proc/sys/fs/binfmt_misc/,
# allow bind mount of /lib/init/fstab for lxcguest
#mount options=(rw, bind) /lib/init/fstab.lxc/ -> /lib/init/fstab/,
# block some other dangerous paths
deny @{PROC}/sys/fs/** wklx,
deny @{PROC}/sysrq-trigger rwklx,
deny @{PROC}/mem rwklx,
deny @{PROC}/kmem rwklx,
deny @{PROC}/sys** rwklx,
deny @{PROC}/scsi** rwklx,
deny @{PROC}/fs** rwklx,
deny @{PROC}/acpi** rwklx,
deny @{PROC}/asound** rwklx,
deny @{PROC}/bus** rwklx,
deny @{PROC}/driver** rwklx,
deny @{PROC}/irq** rwklx,
deny @{PROC}/sysvipc** rwklx,
deny @{PROC}/tty** rwklx,
#deny @{PROC}/sys/kernel/[^s][^h][^m]* wklx,
#deny @{PROC}/sys/kernel/*/** wklx,
# deny writes in /sys except for /sys/fs/cgroup, also allow
# fusectl, securityfs and debugfs to be mounted there (read-only)
mount fstype=fusectl -> /sys/fs/fuse/connections/,
mount fstype=securityfs -> /sys/kernel/security/,
mount fstype=debugfs -> /sys/kernel/debug/,
deny mount fstype=debugfs -> /var/lib/ureadahead/debugfs/,
mount fstype=proc -> /proc/,
mount fstype=sysfs -> /sys/,
deny /sys/** rwklx,
}aa_allow_incomplete
# If this flag is 0 (default), if the kernel lacks the apparmor mount features
# then the container will not be started
# kernel lacks the apparmor mount: mount restrictions are not available in the upstream kernel
# To start the container under partial apparmor protection, set this flag to 1
lxc.aa_allow_incomplete = 1
Apparmor policies require mount restrictions to fullfill many of their promises
for instance if proc can be mounted anywhere,
then 'deny /proc/sysrq-trigger w' prevents only accidents, not malice.
Ephemeral containers
# discarding any change you make on shutdown
sudo lxc-start-ephemeral -o my-container
啟動大量 VPS
修改 /etc/init/lxc.conf, 在後面加入 sleep 5, 如下
pre-start script
[ -f /etc/default/lxc ] && . /etc/default/lxc
[ "x$LXC_AUTO" = "xtrue" ] || { stop; exit 0; }
# don't load profiles if mount mediation is not supported
SYSF=/sys/kernel/security/apparmor/features/mount/mask
if [ -f $SYSF ]; then
if [ -x /lib/init/apparmor-profile-load ]; then
/lib/init/apparmor-profile-load usr.bin.lxc-start
/lib/init/apparmor-profile-load lxc-containers
fi
fi
ls /etc/lxc/auto/* > /dev/null 2>&1 || exit 0;
for f in /etc/lxc/auto/*; do
sleep 5;
c="$(basename $f .conf)"
lxc-info -n $c 2>/dev/null | grep state | grep -q "RUNNING" || lxc-start -n $c -f $f -d
done
end script
Turning
Stop update-grub from scanning each and every hard drive?
# 在 /etc/grub.d/30_os-prober 加入
GRUB_DISABLE_OS_PROBER=true
Debug
Case: start fail
lxc-start -n debian
lxc-start: failed to attach 'veth5gY80t' to the bridge 'lxcbr0' : No such device lxc-start: failed to create netdev lxc-start: failed to create the network lxc-start: failed to spawn 'debian' lxc-start: No such file or directory - failed to remove cgroup '/sys/fs/cgroup/cpuset//lxc/debian'
# -l, --logpriority=LEVEL
# -o, --logfile=FILE
lxc-start -o start.log -l DEBUG -n container
Whole Configure
# Network lxc.network.type = veth lxc.network.link = br0 lxc.network.flags = up lxc.network.hwaddr = 00:16:3e:64:f3:11 lxc.network.name = eth0 lxc.network.veth.pair = veth-u12 # Host lxc.utsname = u12 lxc.arch = i686 lxc.devttydir = lxc lxc.tty = 4 lxc.pts = 1024 lxc.pivotdir = lxc_putold # Rootfs lxc.mount = /lxc/u12/fstab lxc.rootfs = /lxc/u12/rootfs # CPU lxc.cgroup.cpuset.cpus = 0 lxc.cgroup.cpu.shares = 1024 # RAM lxc.cgroup.memory.limit_in_bytes = 256M lxc.cgroup.memory.memsw.limit_in_bytes = 512M # Permission lxc.cap.drop = sys_module mac_admin mac_override sys_time sys_admin # uncomment the next line to run the container unconfined: #lxc.aa_profile = unconfined #### Device #### lxc.cgroup.devices.deny = a # null, zero, full lxc.cgroup.devices.allow = c 1:3 rwm lxc.cgroup.devices.allow = c 1:5 rwm lxc.cgroup.devices.allow = c 1:7 rwm # consoles, ptmx, pty lxc.cgroup.devices.allow = c 5:1 rwm lxc.cgroup.devices.allow = c 5:2 rwm lxc.cgroup.devices.allow = c 136:* rwm # random, urandom lxc.cgroup.devices.allow = c 1:8 rwm lxc.cgroup.devices.allow = c 1:9 rwm # tty, tty0, tty1 lxc.cgroup.devices.allow = c 5:0 rwm lxc.cgroup.devices.allow = c 4:0 rwm lxc.cgroup.devices.allow = c 4:1 rwm # rtc lxc.cgroup.devices.allow = c 254:0 rm
U14.04
獲得
/usr/share/lxc/templates/lxc-ubuntu
/usr/share/lxc/templates/lxc-ubuntu-cloud
..................
建立
lxc-create -t ubuntu -n pydio -B lvm --vgname=myvg --fssize=20G
Pausing (freeze) # U14
# LXC uses the Linux process freezer available through the cgroup interface:
# 行完 cmd 後, proccess 會不 freeze,
# 不過 vps 仍會 ping 到, ssh 仍會 connect 到
# keyboard sent 過去的 cmd 仍會收到
lxc-freeze -n sshgw
# unfreeze
lxc-unfreeze -n sshgw
freeze 與 SIGTSTP 及 SIGSTOP 的分別
"graceful", and one is "forceful"
"graceful" - SIGTSTP
"forceful" - SIGSTOP
# SIGTSTP
# SIGTSTP, and its purpose is to "nicely" ask the process, if it feels like it, to please suspend execution until it receives a SIGCONT
kill -SIGTSTP <pid> # 20
# It cannot be handled, ignored, or blocked.
# SIGSTOP
kill -SIGSTOP <pid> # 19
# jobs 會見到它
# 行返
kill -SIGCONT <pid> # 18
Run time 設定
* Low-level access
U14
# uses the cgroup filesystem to manage containers.
# Check
# lxc-cgroup -n vpsname cpu.shares
i.e.
lxc-cgroup -n sshgw cpu.shares
1024
# Set
lxc-cgroup -n name cpu.shares value
i.e.
lxc-cgroup -n sshgw cpu.shares 2048
Logging
Container init verbosity
lxc-start -n sshgw /sbin/init loglevel=debug
log to file(沒太大用圖)
Logging can be configured on a per-container basis.
Default container startup is logged only at the ERROR level
# 0 = trace, 1 = debug, 2 = info, 3 = notice, 4 = warn, 5 = error, 6 = critical, 7 = alert, and 8 = fatal.
# log level
lxc.log.level = 0
# The file to which logging info should be written.
lxc.log.file = /var/log/lxc/xxx.log
lxc-stop
Currently "lxc-stop -n vpsname" sends SIGPWR to init inside container, but rc-scripts fails to shutdown things properly (shutdown scripts are not invoked).
For workaround, stop services manually before issueing lxc-stop or run poweroff/halt/reboot from container.
By default,
it will request a clean shutdown of the container by sending lxc.haltsignal (defaults to SIGPWR) to
the container's init process, waiting up to 60 seconds for the container to exit, and then returning.
If the container fails to cleanly exit in 60 seconds, it will be sent the lxc.stopsignal (defaults to SIGKILL)
to force it to shut down.
Default 設定
lxc.haltsignal=SIGPWR lxc.stopsignal=SIGKILL
Force to kill
lxc-stop -k -n vpsname
-k,--kill
Rather than requesting a clean shutdown of the container, explicitly kill all tasks in the container.
This is the legacy lxc-stop behavior.
Troubleshoot
lxc-stop: commands.c: lxc_cmd_stop: 620 failed to stop 'nginx': Operation not permitted
[ 2136.257688] type=1400 audit(1451320321.179:1293): apparmor="DENIED" operation="signal" profile="lxc-u12-profile" pid=5236 comm="lxc-start" requested_mask="receive" denied_mask="receive" signal=kill peer="/usr/bin/lxc-start"
Attach
# start a shell attached to C1's namespaces
lxc-attach -n C1 [-- command]
lxc-attach -n sshgw -- ps -e
Output
PID TTY TIME CMD
1 ? 00:00:00 init
101 ? 00:00:00 upstart-socket-
147 ? 00:00:00 rsyslogd
194 ? 00:00:00 sshd
224 lxc/tty2 00:00:00 mingetty
227 ? 00:00:00 cron
255 ? 00:00:16 python
270 ? 00:00:00 monit
281 lxc/console 00:00:00 mingetty
283 lxc/tty1 00:00:00 mingetty
467 ? 00:00:00 sshd
468 ? 00:00:00 ps
lxc-device
manage devices of running containers
lxc-device [-h] [-n name] [add] [DEVICE] [NAME]
[NAME] Name for the device within the container.
EXAMPLES
# Creates a /dev/video0 device in container p1 based on the matching device on the host.
lxc-device -n p1 add /dev/video0
# Moves eth0 from the host as eth1 in p1.
lxc-device -n p1 add eth0 eth1
restricting container view of dmesg
# restrict unprivileged access to kernel syslog
kernel.dmesg_restrict
/etc/sysctl.conf
# cat /proc/sys/kernel/dmesg_restrict kernel.dmesg_restrict=1
sysctl -p
Guest
dmesg: read kernel buffer failed: Operation not permitted
lxc.kmsg # lxc 1.0
Enable creating "/dev/kmsg" as symlink to /dev/console. (Default: 0)
ls -l /dev/kmsg
lrwxrwxrwx 1 root root 7 Oct 19 19:20 /dev/kmsg -> console
Disable 沒用的 agetty
systemctl | grep getty
console-getty.service loaded active running Console Getty [email protected] loaded active running Container Getty on /dev/pts/0 [email protected] loaded active running Container Getty on /dev/pts/1 [email protected] loaded active running Container Getty on /dev/pts/2 [email protected] loaded active running Container Getty on /dev/pts/3 getty-static.service loaded active exited getty on tty2-tty6 if dbus and logind are not available system-container\x2dgetty.slice loaded active active system-container\x2dgetty.slice system-getty.slice loaded active active system-getty.slice getty.target loaded active active Login Prompts
systemctl mask [email protected]
Created symlink from /etc/systemd/system/[email protected] to /dev/null.
Centos 7 firewalld
... firewalld[344]: ERROR: Failed to load nf_conntrack module: modprobe: ...
Fix
On host
modprobe nf_conntrack
Template Turning
# U16.04
apt-get purge initramfs-tools initramfs-tools-bin initramfs-tools-core
apt-mark hold systemd systemd-sysv udev
參考
https://wiki.ubuntu.com/LxcSecurity
http://manpages.ubuntu.com/manpages/en/man7/capabilities.7.html
http://manpages.ubuntu.com/manpages/precise/en/man5/lxc.conf.5.html
https://help.ubuntu.com/12.04/serverguide/lxc.html
其他工具
- lxctl
- linux-container
