laforge has submitted this change. (
https://gerrit.osmocom.org/c/docker-playground/+/30387 )
Change subject: release-tarball-build-dist: check exists on server
......................................................................
release-tarball-build-dist: check exists on server
Previously this job relied on always running on the same jenkins node,
and having the previously built release tarballs in a workspace
directory that does not get removed. It would only skip building a
release tarball if the file already exists locally.
As part of OS#5793 this job needs to be moved away from that jenkins
node. When trying to run the job on a different node, it will build all
release tarballs again which takes unnecessarsily long and actually
fails on libosmocore-0.9.4 due to a missing python2.
This probably happens because the job was refactored at some point and I
didn't realize that the python2 dep is now missing as the previous
tarballs existed.
In general it doesn't make much sense to build previous release tarballs
again and to always keep the legacy environments around for that. Change
the code to check if a tarball exists on the server, and skip building
the tarball in that case.
Related: OS#5793
Related:
https://jenkins.osmocom.org/jenkins/view/All%20no%20Gerrit/job/Osmocom-rele…
Change-Id: I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce
---
M release-tarball-build-dist/osmocom-release-tarballs.sh
1 file changed, 16 insertions(+), 2 deletions(-)
Approvals:
Jenkins Builder: Verified
laforge: Looks good to me, approved
diff --git a/release-tarball-build-dist/osmocom-release-tarballs.sh
b/release-tarball-build-dist/osmocom-release-tarballs.sh
index 04b8350..8f3d3ac 100644
--- a/release-tarball-build-dist/osmocom-release-tarballs.sh
+++ b/release-tarball-build-dist/osmocom-release-tarballs.sh
@@ -191,6 +191,15 @@
fi
}
+get_existing_tarballs() {
+ if ! $SSH_COMMAND releases(a)ftp.osmocom.org -- \
+ find web-files -name '*.tar.bz2' \
+ > "$TEMP"/existing_tarballs; then
+ echo "ERROR: getting existing tarballs from remote failed!"
+ exit 1
+ fi
+}
+
# Clone an Osmocom repository to $TEMP/repos/$repo, clean it, checkout a tag.
# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host)
# $2: tag (optional, default: master)
@@ -318,13 +327,15 @@
upload() {
cd _release_tarballs
- rsync -avz --delete -e "$SSH_COMMAND" . releases@ftp.osmocom.org:web-files/
+ rsync -avz -e "$SSH_COMMAND" . releases@ftp.osmocom.org:web-files/
}
remove_temp_dir
mkdir -p "$TEMP/repos"
echo "Temp dir: $TEMP"
+get_existing_tarballs
+
for repo in $OSMO_RELEASE_REPOS; do
echo "$repo"
tags="$(osmo_git_last_commits_tags "$repo" "all" | cut -d / -f
3)"
@@ -342,7 +353,10 @@
echo " $tarball (ignored)"
continue
elif [ -e "$OUTPUT/$repo/$tarball" ]; then
- echo " $tarball (exists)"
+ echo " $tarball (exists locally)"
+ continue
+ elif grep -q "^web-files/$repo/$tarball$"
"$TEMP"/existing_tarballs; then
+ echo " $tarball (exists on server)"
continue
fi
--
To view, visit
https://gerrit.osmocom.org/c/docker-playground/+/30387
To unsubscribe, or for help writing mail filters, visit
https://gerrit.osmocom.org/settings
Gerrit-Project: docker-playground
Gerrit-Branch: master
Gerrit-Change-Id: I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce
Gerrit-Change-Number: 30387
Gerrit-PatchSet: 1
Gerrit-Owner: osmith <osmith(a)sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-MessageType: merged