Change in docker-playground[master]: release-tarball-build-dist: check exists on server

osmith has uploaded this change for review. ( https://gerrit.osmocom.org/c/docker-playground/+/30387 ) Change subject: release-tarball-build-dist: check exists on server ...................................................................... release-tarball-build-dist: check exists on server Previously this job relied on always running on the same jenkins node, and having the previously built release tarballs in a workspace directory that does not get removed. It would only skip building a release tarball if the file already exists locally. As part of OS#5793 this job needs to be moved away from that jenkins node. When trying to run the job on a different node, it will build all release tarballs again which takes unnecessarsily long and actually fails on libosmocore-0.9.4 due to a missing python2. This probably happens because the job was refactored at some point and I didn't realize that the python2 dep is now missing as the previous tarballs existed. In general it doesn't make much sense to build previous release tarballs again and to always keep the legacy environments around for that. Change the code to check if a tarball exists on the server, and skip building the tarball in that case. Related: OS#5793 Related: https://jenkins.osmocom.org/jenkins/view/All%20no%20Gerrit/job/Osmocom-rele… Change-Id: I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce --- M release-tarball-build-dist/osmocom-release-tarballs.sh 1 file changed, 16 insertions(+), 2 deletions(-) git pull ssh://gerrit.osmocom.org:29418/docker-playground refs/changes/87/30387/1 diff --git a/release-tarball-build-dist/osmocom-release-tarballs.sh b/release-tarball-build-dist/osmocom-release-tarballs.sh index 04b8350..8f3d3ac 100644 --- a/release-tarball-build-dist/osmocom-release-tarballs.sh +++ b/release-tarball-build-dist/osmocom-release-tarballs.sh @@ -191,6 +191,15 @@ fi } +get_existing_tarballs() { + if ! $SSH_COMMAND releases(a)ftp.osmocom.org -- \ + find web-files -name '*.tar.bz2' \ + > "$TEMP"/existing_tarballs; then + echo "ERROR: getting existing tarballs from remote failed!" + exit 1 + fi +} + # Clone an Osmocom repository to $TEMP/repos/$repo, clean it, checkout a tag. # $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) # $2: tag (optional, default: master) @@ -318,13 +327,15 @@ upload() { cd _release_tarballs - rsync -avz --delete -e "$SSH_COMMAND" . releases@ftp.osmocom.org:web-files/ + rsync -avz -e "$SSH_COMMAND" . releases@ftp.osmocom.org:web-files/ } remove_temp_dir mkdir -p "$TEMP/repos" echo "Temp dir: $TEMP" +get_existing_tarballs + for repo in $OSMO_RELEASE_REPOS; do echo "$repo" tags="$(osmo_git_last_commits_tags "$repo" "all" | cut -d / -f 3)" @@ -342,7 +353,10 @@ echo " $tarball (ignored)" continue elif [ -e "$OUTPUT/$repo/$tarball" ]; then - echo " $tarball (exists)" + echo " $tarball (exists locally)" + continue + elif grep -q "^web-files/$repo/$tarball$" "$TEMP"/existing_tarballs; then + echo " $tarball (exists on server)" continue fi -- To view, visit https://gerrit.osmocom.org/c/docker-playground/+/30387 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: docker-playground Gerrit-Branch: master Gerrit-Change-Id: I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce Gerrit-Change-Number: 30387 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: newchange