jenkins slave setup and artifact re-use

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

Harald Welte laforge at gnumonks.org
Thu Sep 7 14:09:26 UTC 2017


Hi Neels,

On Thu, Sep 07, 2017 at 06:53:28AM +0200, Neels Hofmeyr wrote:
> Why not have a series of Dockerfiles for each git, building onto the Dockerfile
> of its "next" dependency?

To avoid complexity and having too maintain too many Dockerfiles, related images, etc.

I think one of the beauties of the proposal is that we reduce the amount of
things that need explicit maintenance.

> You said you would prefer having not so many images around, but what is the
> reason? I thought the images were incremental, and if one references the
> previous, there is only a small difference taking up space for it?

Sure, space-wise it doesn't matter.  I'm more thinking of having to
maintain the Dockerfiles

> I'm thinking of a cascade like this:
> 
>  debian
>   \
>    libosmocore
>    |
>    libosmo-abis
>    |
>    +---------------- osmo-hlr

this would mean you would have to
* docker build the libosmocore image to check/update to current master
* docker build the libosmo-abis image
* docker run the build for osmo-hlr

If this splits up to even more images, you will end up having something
like 8 'docker build' followed by one 'docker run' in each gerrit job.
I'm not sure how much of the performance gain we will loose that way.

> It's a compromise of least dependencies and avoiding dupes, resulting in 13
> different stages.

complexity, and manually having to re-trigger builds in the right
inverse dependency order in every jenkins job.

> The libosmocore job would update the libosmocore image, the libosmo-abis job
> builds on it to produce the libosmo-abis image, and so forth. The jenkins jobs
> would naturally re-use the other jobs' results and be clean every time.

> The downside is to have to run ~8 different images for a core network (e.g. on
> the osmo-gsm-tester). But is docker managing it such that it doesn't take up 8
> x the space and RAM, just one debian + 8 little increments?

I cannot comment on memory usage of running lots of images in parallel.
As indicated, my proposal was related to gerrit builds, not related to
images that can be used to execute the individual osmo-* components.

> On the osmo-gsm-tester we can then freely combine various versions of the
> different core network components by simply running a different image per
> binary. Would be great to not build separate tester images anymore.

The same applies for the TTCN3 tests, for which I build using the
current docker-playground.  There, each element has a separate image.

Sure, we should aim for overlap where possible and have common ancestors
between the Dockerfiles used for gerrit build testing and those for
actual per-network-element-containers. 

But in the end, I think those are fundamentally different.  In terms of
how often you build them, and in terms of whether you actually want to
keep everything you built vs. building in tmpfs and discarding
everything

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)



More information about the OpenBSC mailing list