disk fill-up of Osmocombuild1, reasons

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

Neels Hofmeyr nhofmeyr at sysmocom.de
Thu Oct 5 17:25:48 UTC 2017


I looked at why the 420G of the Osmocombuild1 root fs are full.

1)

Slave "build1-debian9-lxc" is running as lxc called 'docker'.
/var/lib/lxc/docker/rootfs/var/lib/docker/vfs/dir contains 221G in 333 docker
snapshots. It seems to me that these are growing indefinitely, that nothing is
removing docker hashes that have been built. Does anyone have a discarding
strategy ready that we can deploy there? We need to add one to avoid the build
slave filling up again. https://osmocom.org/issues/2539

For now I did
  docker rmi $(docker images -q -f dangling=true)
and rm'd a few stopped containers that had started an openbsc test:
  docker rm $(docker ps -aq)
which removed hundreds of hashes and freed ~110G
But half of the 221G are still there, not sure if we need those.

I also note that this lxc is not starting automatically after I rebooted
Osmocombuild1 (to get rid of some stuck processes).
I added
  lxc.start.auto = 1
to /var/lib/lxc/docker/config

Now the server is back up and running, the lxc is started, but the jenkins
slave refuses to connect. It tries port 45 but nothing is listening
there. I think I've hit this before but can't remember how to solve it :/
https://osmocom.org/issues/2540


2)

Slave "Osmocombuild1" is running as user osmocom-build on the host's OS itself.
The workspaces make up one large chunk of the disk fill: 94G.

Looking at OpenBSC at 3, it builds up to 3.8G from the various build matrix
workspaces, where each has a compiled libosmocore at >110MB and osmo-iuh
at >130MB. Then the parrallel builds each make an own workspace, exploding the
total. The good news is that it should cap *somewhere* and not grow
indefinitely.

We probably should adjust the OpenBSC and various gerrit jobs to clean up the
workspace after they are done. We so far never needed the binaries.
https://osmocom.org/issues/2538


~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20171005/b7d8c115/attachment.bin>


More information about the OpenBSC mailing list