docker build slaves / docker in lxc

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

Harald Welte hwelte at sysmocom.de
Wed Apr 3 17:15:39 UTC 2019


Hi all,

we just ran out of storage on one of our build slaves.  /var/lib/docker
inside the debian9 lxc had grown to 240 GB while actually it should be
much smaller.  "docker system prune" helped to freee 35GB of build cache,
but still the remaining 217 GB was way too much compared to what it should
use as per "docker system df -v" - and also much more than other build
slaves were using (~ 13GB)

It turned out that the files were in /var/lib/docker/vfs, and as I didn't
see that yet on other systems, I investigated:

* if the physical host hosting the lxc container doesn't have 'overlay.ko'
  loaded, overlay/overlay2 storage drivers are not available to docker

* docker "silently" falls back to using "vfs" which is basically not copy-on-write
  but copy-everything-all-the-time, and which consumes massive amounts of storage

I now made sure build2.osmocom.org loads "overlay.ko" by default and I
changed the storage driver on the debian9 lxc.  This however means you
loos all containers/images, and everything has to be rebuilt from
scratch.

How to test:

	$ docker info | grep Storage

should render

	Storage Driver: overlay2

If it shows vfs, something is wrong.  I checked our other build slaves
and they're set up correctly.

Regards,
	Harald
-- 
- Harald Welte <hwelte at sysmocom.de>             http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



More information about the OpenBSC mailing list