RFC: speed up gerrit verifications by using dependency artifacts

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

André Boddenberg dr.blobb at gmail.com
Thu Mar 30 23:39:44 UTC 2017


>> Why not use the archive artifact feature of jenkins?

True, archiving artifacts comes ootb, but the functionality to copy/fetch
artifacts from another buildjob/project as a build step has to be installed
via plugin [1].

Would you consider installing the copyArtifact plugin?

In case installing such plugin is not an option one can simply use:

"wget $URL_OF_LATEST_SUCCESSFUL_ARTIFACT" [2]

to fetch artifacts. Such command can be put in an additional
"execute shell" build step, so it wouldn't touch any build script...


But a matrixConfigurationJob [3] archives artifacts per axis, so jenkins would
store 8 artifacts [4]. Unfortunately the "archive artifacts" post-build-step
cannot be triggered by a specific axis configuration (afaik).

A quick brainstorming how to change this behavior ended in:

- additional "execute shell" step evaluating its axes configuration
  and delete artifact accordingly to avoid redundant archiving.

- change "archive artifact" "post build step" to not fail build if no artifact
  could be archived. Unfortunately this configuration applies for all axes,
  so when archiving of the "chosen" artifact fails no one will complain.


Furthermore the "latest successful artifact" might already be outdated when
fetching it. This happens every time when a openbsc [5] build is triggered by
an "upstream project" until its artifact has been archived. So a mechanism is
needed to verify whether fetched artifact is still valid and compiling all
dependencies elsewise.


At this point I just got the feeling that writing a custom script and using a
directory as an artifact store may will be much more straight forward. I agrre,
that this directory approach might be to prototyp'ish.

But let's discuss it. We can change everything. :)


>> Why do we need to keep more than one? In general we build all software
>> against master of libraries (e.g. even for the 3G support).

Good question! If one artifact is sufficient, let's archive only one.
(sure thing)

But according to contrib/jenkins.sh in openbsc repo, some non-master branches
(of libosmo-(netif && sccp)) are used in all axes where "IU" is enabled:

   20) if [ "x$IU" = "x--enable-iu" ]; then
   21)      netif_branch="sysmocom/sctp"
   22)      sccp_branch="sysmocom/iu"
   23) fi
   --
   26) osmo-build-dep.sh libosmo-netif $netif_branch
   27) osmo-build-dep.sh libosmo-sccp $sccp_branch

To be honest I'm a bit confused or am I just missing something?

blobb

[1] https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
[2] wget https://jenkins.blobb.me/job/matrixJobArtifactHandlingExample/HELLO=Salut,WORLD=monde,label=masterSlave/lastSuccessfulBuild/artifact/archive.txt
[3] https://jenkins.blobb.me/job/matrixJobArtifactHandlingExample
[4] https://jenkins.blobb.me/job/matrixJobArtifactHandlingExample/HELLO=Salut,WORLD=monde,label=masterSlave/
[5] http://jenkins.osmocom.org/jenkins/view/OpenBSC/job/OpenBSC/

2017-03-30 17:20 GMT+02:00 Holger Freyther <holger at freyther.de>:
>
>> On 30. Mar 2017, at 10:17, Harald Welte <laforge at gnumonks.org> wrote:
>>
>> Hi Andre,
>
> Hey!
>
>
>> just a heads-up about my complete absence from jenkins/CI related
>> discussions: That doesn't mean I'm not interested, it just means that
>> I'm happy to defer the decisions on this to those who understand
>> anything about it (which I don't claim to), i.e. currently most likely
>> Neels and Holger.
>
> I was very happy to see it. One nice aspect of running in a namespace
> is that all jobs can install to /somedir without interfering with each
> other. This makes caching intermediate results easy and as shown by
> Andre a nice speed-up!
>
> I try to look at it over the next couple of days. My uneducated questions
> right now would be. Why not use the archive artifact feature of jenkins?
> Why do we need to keep more than one? In general we build all software
> against master of libraries (e.g. even for the 3G support).
>
> But this feature is something we want, no need to rebuild software that
> has not changed. :)
>
> holger



-- 
Gruß
Blobb



More information about the OpenBSC mailing list