osmith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ci/+/36994?usp=email )
Change subject: jenkins-jobs-osmo: new file
......................................................................
jenkins-jobs-osmo: new file
Do not recommend to the user anymore to put the Osmocom jenkins server
in their global jenkins-job-builder config, as this may lead to jobs
from other repositories being deployed to the Osmocom jenkins by
accident.
Instead of that, let the user put the config in this path:
~/.config/jenkins_jobs/jenkins_jobs_osmo-ci.ini
Add a wrapper script that users this path, and makes sure that no global
config is present.
Related: SYS#6957
Change-Id: I1d7c50b2fef56e343cfe377b51f5f4304219cd22
---
A jenkins-jobs-osmo.sh
M jobs/README.md
2 files changed, 63 insertions(+), 10 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/94/36994/1
diff --git a/jenkins-jobs-osmo.sh b/jenkins-jobs-osmo.sh
new file mode 100755
index 0000000..245ac90
--- /dev/null
+++ b/jenkins-jobs-osmo.sh
@@ -0,0 +1,32 @@
+#!/bin/sh -e
+# jenkins-job-builder wrapper for deploying to the Osmocom jenkins server
+
+CONFIG="$HOME/.config/jenkins_jobs/jenkins_jobs_osmo-ci.ini"
+GLOBAL_CONFIGS="
+ $HOME/.config/jenkins_jobs/jenkins_jobs.ini
+ /etc/jenkins_jobs/jenkins_jobs.ini
+"
+
+if [ "$(basename "$PWD")" != "osmo-ci" ]; then
+ echo "ERROR: run this script from the osmo-ci dir"
+ exit 1
+fi
+
+for i in $GLOBAL_CONFIGS; do
+ if [ -e "$i" ]; then
+ echo "ERROR: global config found: $i"
+ GLOBAL_CONFIG_FOUND=1
+ fi
+done
+if [ "$GLOBAL_CONFIG_FOUND" = 1 ]; then
+ echo "Please rename/remove global config files to prevent deploying to the wrong jenkins server by accident."
+ exit 1
+fi
+
+if ! [ -e "$CONFIG" ]; then
+ echo "ERROR: config not found: $CONFIG"
+ echo "You need to create it first, see: $PWD/jobs/README.md"
+ exit 1
+fi
+
+jenkins-jobs --conf "$CONFIG" "$@"
diff --git a/jobs/README.md b/jobs/README.md
index 8976bbe..7c81af5 100644
--- a/jobs/README.md
+++ b/jobs/README.md
@@ -11,21 +11,18 @@
# apt-get install jenkins-job-builder
```
-Have a jenkins-job-builder.ini file. One of
+Create the following file:
```
-~/.config/jenkins_jobs/jenkins_jobs.ini
-/etc/jenkins_jobs/jenkins_jobs.ini
+~/.config/jenkins_jobs/jenkins_jobs_osmo-ci.ini
```
-or place one in here and pass it to jenkins-jobs using the --conf file.
-
Make sure the file not world readable to minimally safeguard your jenkins password.
Instead of using your jenkins password, use an *API Token*. To retrieve your token go
to Jenkins via a Webbrowser, click on your Username in the right corner, click on configure,
click on *Show API Toke...*.
-`jenkins_jobs.ini`:
+`jenkins_jobs_osmo-ci.ini`:
```
[jenkins]
@@ -37,13 +34,14 @@
and
```
-$ chmod go-rwx jenkins_jobs.ini
+$ chmod go-rwx jenkins_jobs_osmo-ci.ini
```
## Update a single job on jenkins.osmocom.org
```
-$ jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml gerrit-osmo-msc
+$ cd ..
+$ ./jenkins-jobs-osmo.sh update jobs/gerrit-verifications.yml gerrit-osmo-msc
```
NOTE: when you supply a name not defined in that yml file, you will not get an
@@ -52,13 +50,15 @@
## Update all jobs of one file
```
-$ jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml
+$ cd ..
+$ ./jenkins-jobs-osmo.sh update jobs/gerrit-verifications.yml
```
## Update all jobs in all files
```
-$ jenkins-jobs --conf jenkins_jobs.ini update ./
+$ cd ..
+$ ./jenkins-jobs-osmo.sh update jobs/
```
## Troubleshooting
--
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/36994?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: I1d7c50b2fef56e343cfe377b51f5f4304219cd22
Gerrit-Change-Number: 36994
Gerrit-PatchSet: 1
Gerrit-Owner: osmith <osmith(a)sysmocom.de>
Gerrit-MessageType: newchange
osmith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ci/+/36993?usp=email )
Change subject: jobs/README: convert to markdown
......................................................................
jobs/README: convert to markdown
Convert to markdown, so it gets rendered in gitea.
Change-Id: I0b687b651b44c798d5d224b0a3ea734b1bc41dff
---
M README.md
D jobs/README.adoc
A jobs/README.md
3 files changed, 193 insertions(+), 150 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/93/36993/1
diff --git a/README.md b/README.md
index f0a3dc9..cb66e63 100644
--- a/README.md
+++ b/README.md
@@ -30,7 +30,7 @@
```
## jobs
-Jenkins Job Builder YAML files defining jenkins jobs. Read `jobs/README.adoc`
+Jenkins Job Builder YAML files defining jenkins jobs. Read `jobs/README.md`
for more information about deployment.
## lint
diff --git a/jobs/README.adoc b/jobs/README.adoc
deleted file mode 100644
index d403bff..0000000
--- a/jobs/README.adoc
+++ /dev/null
@@ -1,149 +0,0 @@
-These jenkins.osmocom.org job definitions, managed by
-https://docs.openstack.org/infra/jenkins-job-builder/index.html[Jenkins Job Builder]
-
-*Prepare:*
-
-Install jenkins-job-builder:
-
- apt-get install jenkins-job-builder
-
-Have a jenkins-job-builder.ini file. One of
-
- ~/.config/jenkins_jobs/jenkins_jobs.ini
- /etc/jenkins_jobs/jenkins_jobs.ini
-
-or place one in here and pass it to jenkins-jobs using the --conf file.
-
-Make sure the file not world readable to minimally safeguard your jenkins password.
-Instead of using your jenkins password, use an *API Token*. To retrieve your token go
-to Jenkins via a Webbrowser, click on your Username in the right corner, click on configure,
-click on *Show API Toke...*.
-
-jenkins_jobs.ini:
-
- [jenkins]
- user=my_user_name
- password=my_api_token
- url=https://jenkins.osmocom.org/jenkins
-
-and
-
- chmod go-rwx jenkins_jobs.ini
-
-*Update a single job on jenkins.osmocom.org:*
-
- jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml gerrit-osmo-msc
-
-NOTE: when you supply a name not defined in that yml file, you will not get an
-error message, just nothing will happen.
-
-*Update all jobs of one file:*
-
- jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml
-
-*Update all jobs in all files:*
-
- jenkins-jobs --conf jenkins_jobs.ini update ./
-
-*Troubleshooting:*
-
-- 'jenkins.JenkinsException: create[gerrit-osmo-msc] failed'
-
-jenkins.osmocom.org is not reachable, or URL in the config file is erratic.
-Make sure it is exactly
-
- url=https://jenkins.osmocom.org/jenkins
-
-- newlines:
-
-Use 'key: |' to keep new lines in multiline values, e.g.:
- - shell: |
- echo hello
- echo world
-
-See also:
-* https://yaml-multiline.info/
-* https://stackoverflow.com/a/21699210
-
-- jobs named on cmdline are not updated:
-
-Make sure the job name is correct, or just issue an entire yml file without
-individual job names.
-
-Also be aware that jobs are only actually updated when anything changed.
-
-*Jenkins labels*
-
-Most jenkins jobs should run a docker container and install all required
-dependencies inside that, so we don't need to install them on the jenkins node.
-These jobs don't need to set a label, they can just run on any generic jenkins
-node that has docker available. So if you add a new job, you probably don't
-need a label at all.
-
-Existing jobs typically have a label set by the topic they belong to, e.g.:
-- osmocom-master
-- osmocom-gerrit
-- ttcn3
-
-Other labels indicate specific software/hardware works here, e.g.:
-- coverity
-- hdlc
-- osmo-gsm-tester
-- podman
-
-*ccache*
-
-The jobs from master-builds and gerrit-verifications use ccache. View the
-statistics with SSH on the build nodes with:
-
- $ CCACHE_DIR=~/ccache/gerrit-verifications ccache -s
- $ CCACHE_DIR=~/ccache/master-builds ccache -s
-
-Note that running multiple jobs in parallel influence the ccache statistics,
-and it's impossible to tell which job caused which change in the stats (that's
-why they are not printed at the end of each job, it would be confusing).
-
-*timers*
-
-A lot of the jenkins jobs run daily with a timer:
-
- triggers:
- - timed: "H 20 * * *"
-
-or weekly:
-
- triggers:
- - timed: "H 20 * * H"
-
-Use H for the minute / day of week, to have it derivated as hash of the job
-name. Replace 20 with the hour (UTC) the job should run.
-
-The jobs follow this timetable, to ensure we don't attempt to use binary
-packages before they have been built (leading to failing jobs).
-
- 18:00 - 21:00 OBS related
- 18:XX osmocom-obs-sync (sync Debian:12 etc. with openSUSE OBS)
- 19:XX osmocom-obs-check-new-distros
- 19:XX osmocom-obs-wireshark
- 20:XX osmocom-obs (new binary packages start building on OBS!)
-
- 22:00 - 03:00 Jobs that don't need binary packages
- 22:XX coverity
- 22:XX octsim_osmo-ccid-firmware
- 22:XX osmo-gsm-tester-runner (virtual)
- 23:XX master-builds-dahdi
- 00:XX osmocom-api
- 00:XX registry-update-base-images
- 01:XX osmocom-build-tags-against-master
- 01:XX osmocom-list-commits
- 02:XX registry-triggers
- 02:XX registry-rebuild-upload-fpga-build (weekly)
-
- 03:00 - 18:00 Jobs that need binary packages
- ttcn3-testsuites
- 04:XX osmocom-release-manuals
- 05:XX osmocom-release-tarballs
- 06:XX repo-install-test
- 06:XX coverity-status (runs intentionally much later than the coverity job)
-
-master-builds: to avoid complexity, these run throughout the day (H H * * *).
diff --git a/jobs/README.md b/jobs/README.md
new file mode 100644
index 0000000..8976bbe
--- /dev/null
+++ b/jobs/README.md
@@ -0,0 +1,181 @@
+# Osmocom jenkins jobs
+
+These jenkins.osmocom.org job definitions, managed by
+[Jenkins Job Builder](https://docs.openstack.org/infra/jenkins-job-builder/index.html)
+
+## Prepare
+
+Install jenkins-job-builder:
+
+```
+# apt-get install jenkins-job-builder
+```
+
+Have a jenkins-job-builder.ini file. One of
+
+```
+~/.config/jenkins_jobs/jenkins_jobs.ini
+/etc/jenkins_jobs/jenkins_jobs.ini
+```
+
+or place one in here and pass it to jenkins-jobs using the --conf file.
+
+Make sure the file not world readable to minimally safeguard your jenkins password.
+Instead of using your jenkins password, use an *API Token*. To retrieve your token go
+to Jenkins via a Webbrowser, click on your Username in the right corner, click on configure,
+click on *Show API Toke...*.
+
+`jenkins_jobs.ini`:
+
+```
+[jenkins]
+user=my_user_name
+password=my_api_token
+url=https://jenkins.osmocom.org/jenkins
+```
+
+and
+
+```
+$ chmod go-rwx jenkins_jobs.ini
+```
+
+## Update a single job on jenkins.osmocom.org
+
+```
+$ jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml gerrit-osmo-msc
+```
+
+NOTE: when you supply a name not defined in that yml file, you will not get an
+error message, just nothing will happen.
+
+## Update all jobs of one file
+
+```
+$ jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml
+```
+
+## Update all jobs in all files
+
+```
+$ jenkins-jobs --conf jenkins_jobs.ini update ./
+```
+
+## Troubleshooting
+
+### jenkins.JenkinsException: create[gerrit-osmo-msc] failed
+
+jenkins.osmocom.org is not reachable, or URL in the config file is erratic.
+Make sure it is exactly
+
+```
+url=https://jenkins.osmocom.org/jenkins
+```
+
+### Newlines
+
+Use 'key: |' to keep new lines in multiline values, e.g.:
+
+```
+- shell: |
+ echo hello
+ echo world
+```
+
+See also:
+
+* https://yaml-multiline.info/
+* https://stackoverflow.com/a/21699210
+
+### Jobs named on cmdline are not updated
+
+Make sure the job name is correct, or just issue an entire yml file without
+individual job names.
+
+Also be aware that jobs are only actually updated when anything changed.
+
+## Jenkins labels
+
+Most jenkins jobs should run a docker container and install all required
+dependencies inside that, so we don't need to install them on the jenkins node.
+These jobs don't need to set a label, they can just run on any generic jenkins
+node that has docker available. So if you add a new job, you probably don't
+need a label at all.
+
+Existing jobs typically have a label set by the topic they belong to, e.g.:
+
+- osmocom-master
+- osmocom-gerrit
+- ttcn3
+
+Other labels indicate specific software/hardware works here, e.g.:
+
+- coverity
+- hdlc
+- osmo-gsm-tester
+- podman
+
+## ccache
+
+The jobs from master-builds and gerrit-verifications use ccache. View the
+statistics with SSH on the build nodes with:
+
+```
+$ CCACHE_DIR=~/ccache/gerrit-verifications ccache -s
+$ CCACHE_DIR=~/ccache/master-builds ccache -s
+```
+
+Note that running multiple jobs in parallel influence the ccache statistics,
+and it's impossible to tell which job caused which change in the stats (that's
+why they are not printed at the end of each job, it would be confusing).
+
+## Timers
+
+A lot of the jenkins jobs run daily with a timer:
+
+```
+triggers:
+ - timed: "H 20 * * *"
+```
+
+or weekly:
+
+```
+triggers:
+ - timed: "H 20 * * H"
+```
+
+Use H for the minute / day of week, to have it derivated as hash of the job
+name. Replace 20 with the hour (UTC) the job should run.
+
+The jobs follow this timetable, to ensure we don't attempt to use binary
+packages before they have been built (leading to failing jobs).
+
+```
+18:00 - 21:00 OBS related
+ 18:XX osmocom-obs-sync (sync Debian:12 etc. with openSUSE OBS)
+ 19:XX osmocom-obs-check-new-distros
+ 19:XX osmocom-obs-wireshark
+ 20:XX osmocom-obs (new binary packages start building on OBS!)
+
+22:00 - 03:00 Jobs that don't need binary packages
+ 22:XX coverity
+ 22:XX octsim_osmo-ccid-firmware
+ 22:XX osmo-gsm-tester-runner (virtual)
+ 23:XX master-builds-dahdi
+ 00:XX osmocom-api
+ 00:XX registry-update-base-images
+ 01:XX osmocom-build-tags-against-master
+ 01:XX osmocom-list-commits
+ 02:XX registry-triggers
+ 02:XX registry-rebuild-upload-fpga-build (weekly)
+
+03:00 - 18:00 Jobs that need binary packages
+ ttcn3-testsuites
+ 04:XX osmocom-release-manuals
+ 05:XX osmocom-release-tarballs
+ 06:XX repo-install-test
+ 06:XX coverity-status (runs intentionally much later than the coverity job)
+```
+
+`master-builds`: to avoid complexity, these run throughout the day (H H * * *).
--
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/36993?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: I0b687b651b44c798d5d224b0a3ea734b1bc41dff
Gerrit-Change-Number: 36993
Gerrit-PatchSet: 1
Gerrit-Owner: osmith <osmith(a)sysmocom.de>
Gerrit-MessageType: newchange