Re: RFC: jenkins pipeline

FYI, Blobb visited us in the sysmocom office yesterday, and we had a personal conversation on the jenkins build setup. It was good meeting you in person, André :) For the sake of this ML, let me answer briefly here as well. On Mon, Mar 06, 2017 at 04:15:20PM +0100, André Boddenberg wrote: We're rebuilding everything simply because no-one has found time to make this more efficient yet. There is no need to rebuild all of libosmocore through to to libsmpp34 for every matrix cell. It would also be useful to build all libosmo* only once for each update of the master branch, and in the dependent builds (openbsc, openbsc-gerrit, ...) always re-use the last successfully built binaries of the dependencies. How about each built library extends a docker image produced by a previous dependency's last successful build? e.g. if libosmocore rebuilds, that updates the libosmocore docker image, then libosmo-abis is triggered and adds itself, producing a last-successfully-built libosmo-abis docker image, and so on, down to openbsc re-using a docker image that already holds all its dependencies? Even though our libraries don't necessarily have a linear dependency, it could make sense to artificially define such a linear line of dependencies (e.g. even though libsmpp doesn't need libosmo-abis, we make libsmpp build on top of the libosmo-abis docker to collect it into the dependecies docker image). But, then again, if libosmocore succeeded and libosmo-abis failed, we would omit a libsmpp build for no reason. So maybe it would also make sense to somehow build the non-dependent libraries independently and later combine into a joint docker image?? ...I would leave that up to your choices, just brain storming... No idea / not aware of it / no opinion so far :) I was once invited to design a logo for the reproducible builds project, but haven't found time for that (yet?). That's about all I know about the topic... My middle name is "Nit-pick" :P It's sometimes really hard for me to ignore a small detail that is obviously wrong for the benefit of moving ahead faster... In the Wiki's case, there is no drawback of correcting mistakes -- no noise in code review or mailing lists is generated, so if you have the time, just do it, as much as you like. hehe Just to be clear, I'm merely one person here, just because I like the sound of Job-DSL, it doesn't mean everyone else does. I think it would be good to see an example of an openbsc build using it and then let everyone decide. But if we introduce docker images as artifacts, this could become useful? I personally mostly care about the overall time a build takes, so that I get V+1/V-1 on my gerrit patches faster. Install libdbi-dev and libdbd-sqlite3. r-cran-dbi?? what is that? :) That's a good question, it should be, right? According to the dockerfile, libdbi-dev is actually installed there. So is libdbd-sqlite3. Hard to say why configure tells you that that is missing. It should work. I've attached the Dockerfile config used to generate the docker image for openbsc. There's some stuff in there not needed for openbsc in particular, e.g. the smalltalk things near the end. And it should probably use the osmo-ci git repos instead of echoing the osmo-deps.sh to /usr/local/bin manually. Actually, our openbsc-gerrit build job has osmo-ci in ~/bin and links that to /build_bin in the docker build, also adding /build_bin to the PATH ... that's a bit cumbersome and is my fault from when I wasn't too familiar with docker yet. These things could be streamlined. docker run --rm=true -e HOME=/build -e MAKE=make -e PARALLEL_MAKE="$PARALLEL_MAKE" \ -e IU="$IU" -e SMPP="$SMPP" -e MGCP="$MGCP" -e PATH="$PATH:/build_bin" \ -e OSMOPY_DEBUG_TCP_SOCKETS="1" -w /build -i -u build -v "$PWD:/build" \ -v "$HOME/bin:/build_bin" osmocom:amd64 /build/contrib/jenkins.sh OT: does your nick "Blobb" indicate affinity to Binary Large Ob(b)jects? ... like docker images? ;) ~N -- - Neels Hofmeyr <nhofmeyr(a)sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschäftsführer / Managing Directors: Harald Welte