RFC: jenkins pipeline

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

André Boddenberg dr.blobb at gmail.com
Mon Mar 6 15:15:20 UTC 2017


>> This sounds to me like Pipelines don't actually provide any of the features I
>> was dreaming about, except maybe easier docker integration?

first yes! second as you said -> maybe, still doubting it :)

When we speak about the "easier docker integration", we mean that
everyone would be happy if not every matrix-configuration axis builds
all deps like libosmocore, libosmo-netif etc pp, right?

To address this issue I'd like to create a local temporary docker
image which extends osmocom:amd64  and holds all deps. This image can
then be used for all openBSC builds and the temporary local docker
image can be removed as a "post-build"-step (which should get
triggered regardless the build result).


A slightly different topic, did you think about pushing your docker
images to hub.docker.com [2]?

In my opinion this will be a step forward to a transparent and local
reproducible build environment. Afair Harald was quite clear about
this requirement?



>> and "Wiki" is an ancient African word for "hopelessly outdated"...
>> We appreciate every checked and/or corrected facts on those pages,
>> and the Jenkins_Node_Setup could probably be merged into the Jenkins one.

Thanks for pointing this out, usual I am hesitating to correct
something to avoid being called nit-picky.



>> >> [2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
>> This sounds like we want to use it!

Alright, so I will work on the following migrations:
     - "as is" -> "JobDSL"
     - "as is" -> "Pipeline" (probably just for Max)


>> so far that doesn't look like an improvement of
>> #!/bin/sh
>> ./contrib/jenkins.sh

Agreed, the biggest advantages of Pipeline are:

-  the "eye-candy" (but who cares? :)

- easy artifacts sharing across different steps, which run on
different nodes (but afaics you don't even use the "Copy Artifacts
Plugin", which makes this argument pointless).

- the duration/sub-steps can be easily seen (as mentioned by Max), but
this can be achieved which some simple python scripts/plugins +
influxDB + grafana as well.



Neels, can you please help me fixing the following build error [3]:

01:59:20 checking dbi/dbd.h usability... no
01:59:20 checking dbi/dbd.h presence... no
01:59:20 checking for dbi/dbd.h... no
01:59:20 configure: error: DBI library is not installed
01:59:22 Build step 'Execute shell' marked build as failure
01:59:22 [WARNINGS] Skipping publisher since build result is FAILURE
01:59:22 Finished: FAILURE


I already added the following package to the image, by:

docker run ........ osmocom:amd64 /bin/bash -c sudo apt-get install -y
r-cran-dbi; /build/contrib/jenkins.sh

but the r-cran-dbi package doesn't solve the issue, which package is needed?

Why is this build dependency not part of the Docker image?


André

[1] https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
[2] https://hub.docker.com/
[3] https://jenkins.blobb.me/job/openBSC_multi-configuration/label=master/36/console

2017-03-06 14:53 GMT+01:00 Neels Hofmeyr <nhofmeyr at sysmocom.de>:
> On Sun, Mar 05, 2017 at 10:03:20PM +0100, Klaus Müller wrote:
>> Hi,
>>
>> >> So, what would be the actual benefits?
>>
>> >>* complete job config in git? (to be confirmed)
>>
>> Just to clarify, the Pipeline plugin on its own doesn't provide a
>> complete config handling in git. This means only the code shown in
>> image [3] lives in git, configurations within the "General" or "Build
>> Triggers" tab will still be handled in the web-ui.***
>> For the sake of completeness the JobDSL plugin [2][ has to be mention.
>> This plugin allows to put the entire job configuration to git and
>> deploys jobs without previous manual setup in the web-ui.
>>
>> >>* faster because less things have to be rebuilt? (to be confirmed)
>>
>> Somehow I doubt that this improvement dependents on the Pipeline
>> plugin, but I know to few details to have an opinion based on facts.
>> :)
>>
>> >>* faster because easily parallelizable/easier integration with docker? (to be
>> >>  confirmed)
>>
>> first point: e.g. Pipeline can trigger "down-stream-projects/stages"
>> after one specific part of a
>> parallel-block/matrix-configurations-project is finished.
>>
>> second point: Pipeline brings a docker wrapper, so one don't have to
>> pass one build script holding the entire build sequence when invoking
>> the container. You can invoke as much scripts as you want within the
>> docker wrapper. (no black-box god-script :)
>
> This sounds to me like Pipelines don't actually provide any of the features I
> was dreaming about, except maybe easier docker integration?
>
>> Is this mailing list the appropriate communication channel?
>
> yes.
>
>> Is there already a osmocom ci wiki page, which I overlooked? I'd be
>> happy to contribute!
>
> well, searching osmocom.org for jenkins gave me
>
> https://osmocom.org/projects/cellular-infrastructure/wiki/Jenkins
> https://osmocom.org/projects/osmocom-servers/wiki/Jenkins_Node_Setup
> https://osmocom.org/projects/cellular-infrastructure/wiki/Nightly_Builds
> [...]
>
> and "Wiki" is an ancient African word for "hopelessly outdated"...
> We appreciate every checked and/or corrected facts on those pages,
> and the Jenkins_Node_Setup could probably be merged into the Jenkins one.
>
>
>> [1] https://jenkins.blobb.me/view/osmocom/
>
> nice, a sleepy pipeline ;)
>
>> [2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
>
> This sounds like we want to use it!
>
>> [2.5] https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.DslFactory.pipelineJob
>> [3] http://tinyurl.com/zbavrtp
>
> direct link without javascript cruft: https://media.comsysto.com/images/2016-11-23-jenkins-pipelines/new-jenkins-pipeline-config--c.png
>
> so far that doesn't look like an improvement of
>
>   #!/bin/sh
>   ./contrib/jenkins.sh
>
> ~N
>
> --
> - Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
> =======================================================================
> * sysmocom - systems for mobile communications GmbH
> * Alt-Moabit 93
> * 10559 Berlin, Germany
> * Sitz / Registered office: Berlin, HRB 134158 B
> * Geschäftsführer / Managing Directors: Harald Welte



-- 
Gruß
Blobb



More information about the OpenBSC mailing list