Updated: 7 November 2021
What follows are mostly my unformatted notes from getting unprivileged containers to work on an OpenRC host.
emerge app-emulation/lxc sys-apps/shadow sys-auth/pambase net-firewall/nftables
rc-update add nftables default
Make sure that you have subuids and subgids for your current user.
grep $USER /etc/sub*
/etc/subgid:jiff:0:0
/etc/subuid:jiff:0:0
Here we don't have the mappings so we'll create them by putting the below in both /etc/sub{uid,gid}
jiff:100000:65536
We'll allow the user to create up to 16 veth (virtual Ethernet) devices without root access. Note that each container needs two veth devices. See man lxc-usernet for more info.
First, we'll turn off automatic creation of bridges and manage it ourselves.
/etc/default/lxc
USE_LXC_BRIDGE="false"
Next, we'll create a basic bridge used for networking. Any connections on the host that we want to handle will need to be forwarded used nftables.
echo '1' | sudo tee -a /proc/sys/net/ipv4/ip_forward
ip link add name xctcontainerbr0 type bridge
ip link set xctcontainerbr0 up
ip addr add 10.10.10.1/24 dev xctcontainerbr0
nft add table inet lxc
nft 'add chain inet lxc postrouting { type nat hook postrouting priority 100; policy accept; }'
nft add rule inet lxc postrouting ip saddr 10.10.10.0/24 ip daddr != 10.10.10.0/24 counter masquerade
nft 'add chain inet lxc input { type filter hook input priority 0; policy accept; }'
nft 'add rule inet lxc input iifname xctcontainerbr0 tcp dport { 80, 443 } accept;'
nft 'add chain inet lxc forward { type filter hook forward priority 100; policy accept; }'
nft 'add rule inet lxc forward iifname xctcontainerbr0 ip daddr 10.10.10.3 tcp dport {80, 443} ct state new accept'
nft 'add chain inet lxc prerouting { type nat hook prerouting priority 0; policy accept; }'
nft 'add rule inet lxc prerouting iifname xctcontainerbr0 ip saddr != 10.10.10.0/24 tcp dport 80 dnat ip to 10.10.10.3:80'
nft 'add rule inet lxc prerouting iifname xctcontainerbr0 ip saddr != 10.10.10.0/24 tcp dport 443 dnat ip to 10.10.10.3:443'Save the configuration changes.
# rc-service nftables save
If you make a mistake you view your rules and their associated handles with.
nft --handle --numeric list ruleset
nft delete rule ip nat postrouting handle 4
If the nft commands don't work you'll need to add the right modules to your kernel. See https://wiki.gentoo.org/wiki/Nftables and https://developers.redhat.com/blog/2018/10/22/introduction-to-linux-interfaces-for-virtual-networking for more insight.
Tell LXC which bridge to use as the default.
/etc/lxc/lxc-usernet jiff veth xctcontainerbr0 16
Make the network config permanent. I'm using netifrc https://gitweb.gentoo.org/proj/netifrc.git/tree/doc/net.example.Linux.in
And add it to the default runlevel.
ln -s /etc/init.d/net.lo /etc/init.d/net.xctcontainerbr0
rc-service net.xctcontainerbr0 start
rc-update add net.xctcontainerbr0 default
Next we'll create the configuration file for the guest by setting up a virtual Ethernet pair connected to a bridge. We'll use static IPs for ease of configuration.
~/.config/lxc/firefox.conf
lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = xctcontainerbr0
lxc.net.0.name = eth0
lxc.net.0.ipv4.gateway = 10.10.10.1
lxc.net.0.ipv4.address = 10.10.10.3/24
lxc.idmap = u 0 100000 165536
lxc.idmap = g 0 100000 165536Try downloading the first container with this command. (Try btrfs with -B)
lxc-create -t download -n firefox -B btrfs -f ~/.config/lxc/firefox.conf -- -d alpine -r 3.14 -a amd64
(or) lxc-create -t download -n firefox -f ~/.config/lxc/firefox.conf -- -d gentoo -r current -a amd64
(or) lxc-create -t download -n firefox -B btrfs -f ~/.local/config/lxc/firefox.conf -- -d debian -r bullseye -a amd64
-- if errors with this inside the container use source /etc/profile then env-update
-- if errors also append Ubuntu's keyserver --key-server hkp://keyserver.ubuntu.com in the download options
-- make sure the btrfs root is mounted with user_subvol_rm_allowed
-- if you're using systemd, use lxc-unpriv-attach and lxc-unpriv-start instead
LXC will need access to your local container directories so we'll add an ACL to permit this. You should have execute permission set for the share directories that stores the root directory of the containers. Using the base uid from above that will represent root in the container. This doesn't feel portable at all but isn't too crazy. shiftfs is supposed to make this easier. https://lists.ubuntu.com/archives/kernel-team/2019-March/099623.html
getfacl /home/jiff/.local/share/lxc
setfacl -m u:100000:x /home/jiff/.local/share/lxc
lxc-start -n firefox -d
The container will be running as shown by lxc-ls on the host.
lxc-info -n firefox
Name: firefox
State: RUNNING
PID: 6228
Link: veth1000_KD5N
TX bytes: 858 bytes
RX bytes: 1.20 KiB
Total bytes: 2.03 KiBLook at the connection status.
lxc-attach --clear-env -n firefoxi -- /bin/sh -c "ip addr"
You may get locale errors.
Get a shell in the container with lxc-attach.
lxc-attach -n firefox
# Install all the fonts
# Remember to set up /etc/resolv.conf for DNS resolution.
# echo 'nameserver 192.168.1.1' > /etc/resolv.conf'
apk add font-bh-100dpi font-sun-misc font-bh-lucidatypewriter-100dpi font-adobe-utopia-type1 font-schumacher-misc font-daewoo-misc font-adobe-utopia-75dpi font-bitstream-100dpi font-xfree86-type1 font-bitstream-75dpi font-bh-ttf font-arabic-misc font-dec-misc font-misc-ethiopic font-micro-misc font-isas-misc font-bh-lucidatypewriter-75dpi font-winitzki-cyrillic font-jis-misc font-bitstream-type1 font-mutt-misc font-misc-misc font-adobe-100dpi font-bh-type1 font-bh-75dpi font-sony-misc font-ibm-type1 font-bitstream-speedo font-adobe-utopia-100dpi font-adobe-75dpi font-misc-meltho font-cursor-misc
# apk add mesa-dri-intel hicolor-icon-theme alsa-lib ca-certificates ttf-dejavu ttf-freefont xorg-server firefox mesa-gles libva-glx mesa-gl mesa-dri-gallium mesa-demos pciutils-dev ffmpeg
# apk add
(or https://download.mozilla.org/?product=firefox-latest-ssl&os=linux64&lang=en-US)
# Run the command by using invocating Firefox. You can view the APKBUILD file here https://git.alpinelinux.org/aports/tree/community/firefox/APKBUILD?h=3.14-stable
# lxc-attach -n firefox -- su -l jiff -c '/usr/bin/firefox --display=:0'
ref: https://bugzilla.mozilla.org/show_bug.cgi?id=1676883And finally, stop the container with lxc-stop.
lxc-attach -n firefox -- /bin/sh -c "ip show"
ip -4 addr add 10.10.10.3/24 dev eth0
ip -4 route add default via 10.10.10.1 dev eth0
echo 'nameserver $NAMESERVER' > /etc/resolv.confIf you get ABORTED errors, you can check the logs with. lxc-start -n alpha -o ./log.txt -l TRACE -- /bin/bash
Easiest way to do this is with pipewire. Install pipewire on the host and use it as the PULSE_SERVER by exporting the variable.
Finally, we'll create the userspace cgroups for the container to run in. This is critically important if you're running these containers on a none OpenRC host.
sudo emerge -av lxcfs
sudo rc-service lxcfs start
sudo rc-update add lxcfs default
sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o rw,nosuid,nodev,noexec,relatime,none,name=systemd cgroup /sys/fs/cgroup/systemd
sudo chmod g+rwX /sys/fs/cgroup/systemd
sudo chown root:100000 /sys/fs/cgroup/systemd