Hi all,
I've created a prototyp'ish JJB YAML file [1] that holds all current gerrit verification jobs on jenkins.osomocom.org. Holger and Lynxis suggested to explore this way of managing Jenkins jobs in the "jenkins build slaves: Rationale for some builds in docker vs. some not?" thread.
Following differences occurred, while clicking through all gerrit jobs configurations:
- some jobs using gcc c compiler warnings v3, some v4 - some jobs are verifying drafts, some not - only two gerrit jobs are allowing concurrent builds
Are these differences wanted? All job differences are commented in the YAML file. The jobs are deployed on jenkins.blobb.me [2].
What do you think about this approach [1] in general?
Regards, André
[1] https://gerrit.osmocom.org/#/c/3911/1 [2] https://jenkins.blobb.me/view/gerrit-JJB/
On 12. Sep 2017, at 20:25, André Boddenberg dr.blobb@gmail.com wrote:
Hi all,
Hey!
I've created a prototyp'ish JJB YAML file [1] that holds all current gerrit verification jobs on jenkins.osomocom.org. Holger and Lynxis suggested to explore this way of managing Jenkins jobs in the "jenkins build slaves: Rationale for some builds in docker vs. some not?" thread.
cool! Do you have an idea of how we could automatically update our jobs based from this?
- some jobs using gcc c compiler warnings v3, some v4
Not on purpose but depends on when we created the job?
- some jobs are verifying drafts, some not
I don't think we want to verify drafts, we rarely use them anyway
- only two gerrit jobs are allowing concurrent builds
On purpose. E.g. when executing tests that listen on a UDP/TCP/SCTP port we can't run them concurrently unless the ports are randomized or somehow virtualize them. In the case of concurrent builds we know we either don't run system tests or we run them in docker.
holger
Hi,
cool! Do you have an idea of how we could automatically update our jobs based from this?
A job that triggers on gerrit post-merge events (osmo-ci) would be my preferred choice. It's more elegant than polling and every merged change will be applied in seconds/minutes. Moreover, this job could also trigger "update-osmo-ci-on-slaves" job as suggested by Max if a change in the scripts/ folder has been introduced.
Another topic to discuss are the credentials. The JJB allows to declare them in a file or pass them while invoking job creation. Would it be "okay" to store them on node(s)? Another way is to declare credentials in Jenkins global configuration and use the 'mask password plugin' to no show them in the console log. Furthermore, a dedicated user with precise permission to deploy jobs is preferrable.
What is your opinion?
Not on purpose but depends on when we created the job?
Max already commented on gerrit that this is probably time related and that the "latest and greatest" version should be used everywhere.
I don't think we want to verify drafts, we rarely use them anyway
Okay, then 'draft verification' will be removed from the YAML file.
On purpose. E.g. when executing tests that listen on a UDP/TCP/SCTP port we can't run them concurrently unless the ports are randomized or somehow virtualize them. In the case of concurrent builds we know we either don't run system tests or we run them in docker.
Ah true, then the current YAML should be sufficient as it allows each job to define whether concurrent builds are allowed (default: false).
Dear Osmocom community,
On Tue, Sep 12, 2017 at 02:25:04PM +0200, André Boddenberg wrote:
I've created a prototyp'ish JJB YAML file [1] that holds all current gerrit verification jobs on jenkins.osomocom.org.
it's been about six weeks since Andre contributed his work on converting our hand-creafted gerrit job descriptions into something that's much cleaner and easier to maintain.
I've taken the liberty to finally deploy this (after another pass manually through all job definitions to ensure we didn't miss any changes). The longer we wait, the more complex the differences between the yml and the acual jobs will get, and we can start from scratch.
As a result, all gerrit jenkins jobs are now the result of http://git.osmocom.org/osmo-ci/tree/jobs/gerrit-verifications.yml
There is not yet a job to automatically deploy any changes in osmo-ci.git to jenkins, but I think it's definitely an improvement.
All jobs generated this way have a clear warning in their description that they should not be modified manually. Please kindly observe this and instead update the yml and deply with "jenkins-jobs update ./jobs/gerrit-verifications.yml" instead.
I apologize in advance for any fall-out from this change. We'll only see if it works once people start pushing patches to gerrit.
Regards, Harald
On Sun, Oct 29, 2017 at 01:23:35PM +0100, Harald Welte wrote:
I apologize in advance for any fall-out from this change. We'll only see if it works once people start pushing patches to gerrit.
Thanks for taking this step, I didn't have the guts yet. I've fixed quite some fallout, take a look at the commits I blatantly pushed directly to osmo-ci master.
Having these textual config files is a huge relief from clicking around in slow web UIs. Kudos to André!
~N
-
André Boddenberg -
Harald Welte -
Holger Freyther -
Neels Hofmeyr