2021-12-18

Sharing storage between containers

Sharing storage between containers with different UIDs can be difficult. The most common resolution I've seen is to map the unprivileged container user to the user inside the container but I feel this is a hack.

lxc.id_map = u 0 100000 1000
lxc.id_map = g 0 100000 1000
lxc.id_map = u 1000 1000 1
lxc.id_map = g 1000 1000 1
lxc.id_map = u 1001 101001 64536
lxc.id_map = g 1001 101001 64536

The solution I've been using it to create a 1GB exFAT file, mount it over loopback, and mount it in a shared space between the containers. 1GB is enough room for simple file sharing between the host and guest.

I haven't noticed any problems with performance with this set up.

Creating the loopback file and mounting

Open a root or sudo-capable shell and run the following. We'll generate the file, turn off compression, atime records, and copy-on-write to improve performance.

# touch /tmp/exfat-loopback.img
# chattr -ABc /tmp/exfat-loopback.img
# dd if=/dev/zero of=/tmp/exfat-loopback.img bs=1M count=1024
# losetup /dev/loop0 /tmp/exfat-loopback.img
# mkfs.exfat /tmp/exfat-loopback.img
# mkdir -p /home/jiff/shared
# mount -o loop,fmask=0111,dmask=0000 /dev/loop0 /home/jiff/shared

Mounting it inside the container

Edit your LXC config file to reference the mount. The details below presumes that the mount was created by a system service and you'll be simply binding it inside the containers.

lxc.mount.entry = /home/jiff/shared home/jiff/shared none bind,create=dir,optional 0 0

Making this automatic

I use OpenRC so I created the following service file in /etc/init.d/container-shared-loopback

<nowiki>
#!/sbin/openrc-run
# Copyright 2021 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
# 2021-12-15: Jiff Slater

name="Container shared loopback daemon"
description="Creates a shared storage for unprivileged containers."

start_pre() {
  local SHARED_USER="jiff"
  local SHARED_LOOPBACK_IMG="/tmp/$SHARED_USER-$SHARED_LOOPBACK_IMG.img"

  if [ ! -f $SHARED_LOOPBACK_IMG ]; then
    return $?
  fi
}

start() {
  local SHARED_USER="jiff"
  local SHARED_LOOPBACK_IMG="/tmp/$SHARED_USER-container-shared-loopback.img"
  local MOUNTPOINT="/home/$SHARED_USER/shared"
  local SHARED_LOOPBACK=$(losetup -f)

  touch $SHARED_LOOPBACK_IMG

  # Turn off compression, CoW, and atime.
  chattr -ACc $SHARED_LOOPBACK_IMG

  dd if=/dev/zero of=$SHARED_LOOPBACK_IMG bs=1M count=1024
  losetup --direct-io=on $SHARED_LOOPBACK $SHARED_LOOPBACK_IMG
  mkfs.exfat $SHARED_LOOPBACK
  mkdir -p $MOUNTPOINT

  # No executable files in the shared storage.
  mount -o loop,fmask=0111,dmask=0000 $SHARED_LOOPBACK $MOUNTPOINT
}

stop() {
  local SHARED_USER="jiff"
  local SHARED_LOOPBACK_IMG="/tmp/$SHARED_USER-container-shared-loopback.img"
  local MOUNTPOINT="/home/$SHARED_USER/shared"

  umount $MOUNTPOINT
  rm $SHARED_LOOPBACK_IMG
}

</nowiki>

Finally, add it to the default runlevel and start it.

rc-service add container-shared-loopback
rc-update container-shared-loopback start