最後更新: 2020-10-10
介紹
Overlay userspace files(filesystem) for cpuinfo, meminfo, stat and uptime.
=> pretty simple, written in C using libfuse(FUSE filesystem)
A cgroupfs compatible tree allowing unprivileged writes.
=> unprivileged user to interact with cgroupfs
---
It's designed to workaround the shortcomings of procfs, sysfs and cgroupfs
by exporting files which match what a system container user would expect.
---
A set of cgroup-aware files, LXCFS will take care
- /proc/cpuinfo
- /proc/diskstats
- /proc/meminfo
- /proc/stat
- /proc/swaps
- /proc/uptime
- /sys/devices/system/cpu/online
---
When a container is started, each filtered hierarchy will be bind-mounted under /sys/fs/cgroup/* in the container
目錄
- Install
- Build From Source
- Upgrading
- lxcfs Service
- Usage
- kmsg
- Version
Install
# U16
apt-get install lxcfs
Build From Source
Prepare
# C7
yum groupinstall "Development Tools"
yum install fuse fuse-lib fuse-devel
# U16
apt-get install libcgmanager-dev libnih-dbus-dev libnih-dev libfuse-dev libdbus-1-dev
apt-get install libnih-dbus-dev libnih-dev libfuse-dev
apt-get install automake # 要 aclocal
apt-get install build-essential libtool # 要 libtoolize
apt-get install pkg-config
# Get source
mkdir /usr/src/lxcfs; cd /usr/src/lxcfs
# https://linuxcontainers.org/lxcfs/downloads/
wget https://linuxcontainers.org/downloads/lxcfs/lxcfs-4.0.12.tar.gz
tar -zxf lxcfs-4.0.12.tar.gz
# make
cd lxcfs-4.0.12
./configure --prefix=/opt/lxcfs
make -j
make install
# lib 設定
ls /opt/lxcfs/lib/lxcfs
liblxcfs.la liblxcfs.so
建立 /etc/ld.so.conf.d/lxc.conf
/opt/lxcfs/lib/lxcfs
ldconfig
# Other
mkdir -p /opt/lxcfs/var/lib/lxcfs # 對應 /opt/lxcfs/share/lxcfs/lxc.mount.hook 設定
ln -s /opt/lxcfs/bin/lxcfs /usr/sbin/lxcfs
# Checking
lxcfs -v
4.0.5
Upgrading
* without restart instance
LXCFS is split into a shared library(liblxcfs) and a simple binary(lxcfs)
lxcfs binary will detect that a new version of the shared library(liblxcfs) is available and
will reload it using dlclose(3) and dlopen(3)
# To force a reload of the shared library
kill -s USR1 $(pidof lxcfs)
Remark
Users of LXCFS on musl(類似 glibc) are advised to restart LXCFS completely and all containers making use of it.
lxcfs Service
/lib/systemd/system/lxcfs.service # Ubuntu 18.04
/etc/systemd/system/lxcfs.service # Centos 7
# lxcfs 4.0
[Unit]
Description=FUSE filesystem for LXC
Before=lxc.service
[Service]
ExecStart=/opt/lxcfs/bin/lxcfs -l -f --enable-cfs --enable-pidfd /opt/lxcfs/var/lib/lxcfs
Restart=on-failure
ExecReload=/bin/kill -s USR1 $MAINPID # for upgrade lxcfs
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable lxcfs --now
systemctl status lxcfs
ps aux | grep lxcfs
... /opt/lxcfs/bin/lxcfs -l -f --enable-cfs --enable-pidfd /opt/lxcfs/var/lib/lxcfs
Usage
Options
- -h, --help
- -f, --foreground
- -l, --enable-loadavg
- --enable-cfs # Enable CPU virtualization via CPU shares
- --enable-pidfd # Use pidfd for process tracking
hook file
ls /opt/lxcfs/share/lxcfs
lxc.mount.hook lxc.reboot.hook
vps configuration
[方法A] Global Enable lxcfs
Contaner 的 config
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
ubuntu.common.conf
lxc.include = /usr/share/lxc/config/common.conf
common.conf
lxc.include = /usr/share/lxc/config/common.conf.d/
建立 /usr/share/lxc/config/common.conf.d/00-lxcfs.conf
lxc.hook.mount = /opt/lxcfs/share/lxcfs/lxc.mount.hook lxc.hook.post-stop = /opt/lxcfs/share/lxcfs/lxc.reboot.hook
[方法B] Per vps config
lxc.hook.mount = /opt/lxcfs/share/lxcfs/lxc.mount.hook lxc.hook.post-stop = /opt/lxcfs/share/lxcfs/lxc.reboot.hook
[方法C] 由 Source Compile 的 lxcfs 設定
... #lxc.mount.auto = cgroup:mixed lxc.autodev = 1 # 非必須 lxc.kmsg = 0 # 非必須 lxc.include = /opt/lxcfs/share/lxc/config/common.conf.d/00-lxcfs.conf
Inside Instance checking
mount | grep lxcfs
lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) lxcfs on /proc/loadavg type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) lxcfs on /sys/devices/system/cpu/online type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
uptime # VPS 有自己的 uptime 了
19:16:50 up 0 min, 1 user, load average: 0.82, 0.77, 0.77
Debug
lxc-start --logpriority DEBUG --logfile proxy.log proxy
kmsg
# Enable creating /dev/kmsg as symlink to /dev/console. This defaults to 1.
# Set this to 1 to enable /dev/kmsg symlinking
lxc.kmsg = 1
Version
V4
* Support for the new unified cgroup hierarchy - cgroup2
* /proc/cpuinfo and cpu output in /proc/stat based on cpu shares
* pidfd supported process tracking
LXCFS needs to keep track of each container's init process in order to correctly virtualize various values.
Newer kernels(kernel 5.4) provide the concept of a pidfd which solves pid reuse problems.
* /proc/loadavg virtualization
container-specific /proc/loadavg view based on cgroups
* Compiler based hardening
a set of well-known compiler and linker options are automatically enabled
whenever the compiler or linker support them.
./configure --prefix=/opt/lxcfs
... checking whether C compiler accepts -Wimplicit-fallthrough=5... yes checking whether C compiler accepts -Wcast-align... yes