On Tue, May 02, 2017 at 07:59:35PM +0200, André Boddenberg wrote:
Hi all,
here's a (late) recap on the "dependency-artifacts" topic discussed
with Holger and Neels at OsmoDevCon.
The "mv $artifact"-issue, which occurs when run within a docker
container has been resolved. Now artifacts are created inside a
temporary directory within the ARTIFACT_STORE and not inside the
Docker container, so the "mv" command won't take as long as a
"cp"
anymore.
Moreover, artifacts are now stored per jenkins job and not per git
project. A JOB_NAME-directory inside the ARTIFACT_STORE is created for
each jenkins job to distinguish between them. This means artifacts
aren't shared across jobs which build the same git project. This might
change in the future, but is safe for now as some jenkins jobs might
need dependencies built with different configurations.
Note: the necessary disk space for openBSC artifacts of one
matrix-configuration job is ~400 MB (~50 MB per axis).
Furthermore, each jenkins job takes care about garbage collection on
its own by simply deleting the entire content of its
JOB_NAME-directory inside the ARTIFACT_STORE.
Which is, for the record, the reason to have it per-job: we always know
which artifacts need to be / we are allowed to clean up. With multiple
jobs accessing the same artifact store, GC is not trivial.
The two scripts osmo-{build,artifacts}.sh have been
tested [1][2],
shellcheck'ed and pushed to gerrit [3].
nice!
The following actions are pending to enable the use of
artifacts
inside contrib/jenkins.sh within each git repository, but can not be
applied by me:
- setting ARTIFACT_STORE environment variable for each jenkins slave
- copying osmo-artifact.sh and osmo-build.sh to ~/bin of each slave user
Add to osmo-ci. Could you submit a patch for it?
git clone
https://gerrit.osmocom.org/osmo-ci
- adding a new docker volume for the ARTIFACT_STORE
to the image
We're talking about the docker invocation in the OpenBSC and
OpenBSC-gerrit jobs.
But, let's review first. =)
I notice there are hashes of various dependent builds in the tar names:
libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652_libsmpp34.master.cc0bcd6_openggsn.master.19e19e3.tar.gz
I remember that we said to instead have each on its own -- but not saying
we have to, just curious.
So the semantics are: figure out all of the hashes of all dependencies, as
soon as one of them mismatches rebuild all and make a new single tar with
all included. Right?
Also curious about how a matrix build would do it: would every matrix
rebuild the same dependency? (We said that that's fine IIRC)
~N