<p>osmith <strong>merged</strong> this change.</p><p><a href="https://gerrit.osmocom.org/c/osmo-ci/+/14651">View Change</a></p><div style="white-space:pre-wrap">Approvals:
pespin: Looks good to me, but someone else must approve
laforge: Looks good to me, but someone else must approve
osmith: Looks good to me, approved; Verified
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">scripts: add osmocom-release-tarballs.sh<br><br>Generate tarball releases for all tags in all repositories.<br><br>Related: OS#3870<br>Change-Id: I0a904c26e1c45aa46ddc59a01b776f2c7c5f947d<br>---<br>M .gitignore<br>A scripts/osmocom-release-tarballs.sh<br>2 files changed, 216 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/.gitignore b/.gitignore</span><br><span>index e101b84..ff207ee 100644</span><br><span>--- a/.gitignore</span><br><span>+++ b/.gitignore</span><br><span>@@ -8,6 +8,7 @@</span><br><span> __pycache__/</span><br><span> _temp/</span><br><span> _deps/</span><br><span style="color: hsl(120, 100%, 40%);">+_release_tarballs/</span><br><span> </span><br><span> # osmocom-nightly-packages.sh</span><br><span> nightly-3g_*</span><br><span>diff --git a/scripts/osmocom-release-tarballs.sh b/scripts/osmocom-release-tarballs.sh</span><br><span>new file mode 100755</span><br><span>index 0000000..b9d21c3</span><br><span>--- /dev/null</span><br><span>+++ b/scripts/osmocom-release-tarballs.sh</span><br><span>@@ -0,0 +1,215 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/bin/sh -e</span><br><span style="color: hsl(120, 100%, 40%);">+# Iterate over all relevant Osmocom repositories and generate release tarballs for each of the repository tags. The tags</span><br><span style="color: hsl(120, 100%, 40%);">+# are queried from the git server without cloning the repositories first, so we can clone them only if we need to build</span><br><span style="color: hsl(120, 100%, 40%);">+# a missing tarball. All repositories are deleted afterwards to save space.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Environment variables:</span><br><span style="color: hsl(120, 100%, 40%);">+# * KEEP_TEMP: do not delete cloned repositories (use for development)</span><br><span style="color: hsl(120, 100%, 40%);">+# * PARALLEL_MAKE: -jN argument for make (default: -j5).</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+cd "$(dirname "$0")"</span><br><span style="color: hsl(120, 100%, 40%);">+. ./common.sh</span><br><span style="color: hsl(120, 100%, 40%);">+PARALLEL_MAKE="${PARALLEL_MAKE:--j5}"</span><br><span style="color: hsl(120, 100%, 40%);">+OUTPUT="$(cd ..; pwd)/_release_tarballs"</span><br><span style="color: hsl(120, 100%, 40%);">+TEMP="$(cd ..; pwd)/_temp"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Print all tags for which no release tarball should be built.</span><br><span style="color: hsl(120, 100%, 40%);">+# $1: Osmocom repository</span><br><span style="color: hsl(120, 100%, 40%);">+tags_to_ignore() {</span><br><span style="color: hsl(120, 100%, 40%);">+ case "$1" in</span><br><span style="color: hsl(120, 100%, 40%);">+ libosmocore)</span><br><span style="color: hsl(120, 100%, 40%);">+ # configure.ac:144: error: required file 'src/gb/Makefile.in' not found</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.5.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.5.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ libsmpp34)</span><br><span style="color: hsl(120, 100%, 40%);">+ # duplicate of 1.12.0</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.12"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-bsc)</span><br><span style="color: hsl(120, 100%, 40%);">+ # openbsc</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.0.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ # Requires libosmo-legacy-mgcp</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.1.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.1.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.1.2"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.2.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.2.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.2.2"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-bts)</span><br><span style="color: hsl(120, 100%, 40%);">+ # gsm_data_shared.h:464:26: error: field 'power_params' has incomplete type</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.2.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.3.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-hlr)</span><br><span style="color: hsl(120, 100%, 40%);">+ # Not using autotools</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.0.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-mgw)</span><br><span style="color: hsl(120, 100%, 40%);">+ # openbsc</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.0.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-msc)</span><br><span style="color: hsl(120, 100%, 40%);">+ # openbsc</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "1.0.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-pcu)</span><br><span style="color: hsl(120, 100%, 40%);">+ # Duplicates of 0.1.0, 0.2.0</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.2"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-sgsn)</span><br><span style="color: hsl(120, 100%, 40%);">+ # openbsc</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.9.0 0.9.1 0.9.2 0.9.3 0.9.4 0.9.5 0.9.6 0.9.8 0.9.9 0.9.10 0.9.11 0.9.12 0.9.13 0.9.14"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.9.15 0.9.16 0.10.0 0.10.1 0.11.0 0.12.0 0.13.0 0.14.0 0.15.0 1.0.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-sip-connector)</span><br><span style="color: hsl(120, 100%, 40%);">+ # make: *** No rule to make target 'osmoappdesc.py'</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.0.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-trx)</span><br><span style="color: hsl(120, 100%, 40%);">+ # cp: cannot stat './/home/user/code/osmo-dev/src/osmo-ci/_temp/repos/osmo-trx/configure'</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.2.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "0.3.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ esac</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Clone dependency repositories.</span><br><span style="color: hsl(120, 100%, 40%);">+# $1: Osmocom repository</span><br><span style="color: hsl(120, 100%, 40%);">+prepare_depends() {</span><br><span style="color: hsl(120, 100%, 40%);">+ case "$1" in</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-bts)</span><br><span style="color: hsl(120, 100%, 40%);">+ # Includes openbsc/gsm_data_shared.h</span><br><span style="color: hsl(120, 100%, 40%);">+ prepare_repo "openbsc"</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ esac</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Apply workarounds for bugs that break too many releases. This function runs between ./configure and make dist-bzip2.</span><br><span style="color: hsl(120, 100%, 40%);">+# $1: Osmocom repository</span><br><span style="color: hsl(120, 100%, 40%);">+fix_repo() {</span><br><span style="color: hsl(120, 100%, 40%);">+ case "$1" in</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-mgw)</span><br><span style="color: hsl(120, 100%, 40%);">+ # No rule to make target 'osmocom/mgcp_client/mgcp_common.h' (OS#4084)</span><br><span style="color: hsl(120, 100%, 40%);">+ make -C "$TEMP/repos/$1/include/osmocom/mgcp_client" mgcp_common.h || true</span><br><span style="color: hsl(120, 100%, 40%);">+ ;;</span><br><span style="color: hsl(120, 100%, 40%);">+ esac</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Check if one specific tag should be ignored.</span><br><span style="color: hsl(120, 100%, 40%);">+# $1: Osmocom repository</span><br><span style="color: hsl(120, 100%, 40%);">+# $2: tag (e.g. "1.0.0")</span><br><span style="color: hsl(120, 100%, 40%);">+ignore_tag() {</span><br><span style="color: hsl(120, 100%, 40%);">+ local repo="$1"</span><br><span style="color: hsl(120, 100%, 40%);">+ local tag="$2"</span><br><span style="color: hsl(120, 100%, 40%);">+ local tags="$(tags_to_ignore "$repo")"</span><br><span style="color: hsl(120, 100%, 40%);">+ for tag_i in $tags; do</span><br><span style="color: hsl(120, 100%, 40%);">+ if [ "$tag" = "$tag_i" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ return 0</span><br><span style="color: hsl(120, 100%, 40%);">+ fi</span><br><span style="color: hsl(120, 100%, 40%);">+ done</span><br><span style="color: hsl(120, 100%, 40%);">+ return 1</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Delete existing temp dir (unless KEEP_TEMP is set). If all repos were checked out, this restores ~500 MB of space.</span><br><span style="color: hsl(120, 100%, 40%);">+remove_temp_dir() {</span><br><span style="color: hsl(120, 100%, 40%);">+ if [ -n "$KEEP_TEMP" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "NOTE: not removing temp dir, because KEEP_TEMP is set: $TEMP"</span><br><span style="color: hsl(120, 100%, 40%);">+ elif [ -d "$TEMP" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ rm -rf "$TEMP"</span><br><span style="color: hsl(120, 100%, 40%);">+ fi</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Clone an Osmocom repository to $TEMP/repos/$repo, clean it, checkout a tag.</span><br><span style="color: hsl(120, 100%, 40%);">+# $1: Osmocom repository</span><br><span style="color: hsl(120, 100%, 40%);">+# $2: tag (optional, default: master)</span><br><span style="color: hsl(120, 100%, 40%);">+prepare_repo() {</span><br><span style="color: hsl(120, 100%, 40%);">+ local repo="$1"</span><br><span style="color: hsl(120, 100%, 40%);">+ local tag="${2:-master}"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if ! [ -d "$TEMP/repos/$repo" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ git -C "$TEMP/repos" clone "$OSMO_GIT_URL/$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+ fi</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ cd "$TEMP/repos/$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+ git clean -qdxf</span><br><span style="color: hsl(120, 100%, 40%);">+ git reset --hard HEAD # in case the tracked files were modified (e.g. libsmpp34 1.10)</span><br><span style="color: hsl(120, 100%, 40%);">+ git checkout -q "$tag"</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Checkout a given tag and build a release tarball.</span><br><span style="color: hsl(120, 100%, 40%);">+# $1: Osmocom repository</span><br><span style="color: hsl(120, 100%, 40%);">+# $2: tag</span><br><span style="color: hsl(120, 100%, 40%);">+create_tarball() {</span><br><span style="color: hsl(120, 100%, 40%);">+ local repo="$1"</span><br><span style="color: hsl(120, 100%, 40%);">+ local tag="$2"</span><br><span style="color: hsl(120, 100%, 40%);">+ local tarball="$repo-$tag.tar.bz2"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Be verbose during the tarball build and preparation. Everything else is not verbose, so we can generate an</span><br><span style="color: hsl(120, 100%, 40%);">+ # easy to read overview of tarballs that are already built or are ignored.</span><br><span style="color: hsl(120, 100%, 40%);">+ set -x</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ prepare_repo "$repo" "$tag"</span><br><span style="color: hsl(120, 100%, 40%);">+ prepare_depends "$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ cd "$TEMP/repos/$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+ autoreconf -fi</span><br><span style="color: hsl(120, 100%, 40%);">+ ./configure</span><br><span style="color: hsl(120, 100%, 40%);">+ fix_repo "$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+ make dist-bzip2</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Back to non-verbose mode</span><br><span style="color: hsl(120, 100%, 40%);">+ set +x</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if ! [ -e "$tarball" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "NOTE: tarball has a different name (wrong version in configure.ac?), renaming."</span><br><span style="color: hsl(120, 100%, 40%);">+ mv -v *.tar.bz2 "$tarball"</span><br><span style="color: hsl(120, 100%, 40%);">+ fi</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Move a generated release tarball to the output dir.</span><br><span style="color: hsl(120, 100%, 40%);">+move_tarball() {</span><br><span style="color: hsl(120, 100%, 40%);">+ local repo="$1"</span><br><span style="color: hsl(120, 100%, 40%);">+ local tag="$2"</span><br><span style="color: hsl(120, 100%, 40%);">+ local tarball="$repo-$tag.tar.bz2"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ cd "$TEMP/repos/$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+ mkdir -p "$OUTPUT/$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+ mv "$tarball" "$OUTPUT/$repo/$tarball"</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+remove_temp_dir</span><br><span style="color: hsl(120, 100%, 40%);">+mkdir -p "$TEMP/repos"</span><br><span style="color: hsl(120, 100%, 40%);">+echo "Temp dir: $TEMP"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+for repo in $OSMO_RELEASE_REPOS; do</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "$repo"</span><br><span style="color: hsl(120, 100%, 40%);">+ tags="$(osmo_git_last_commits_tags "$repo" "all" | cut -d / -f 3)"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Skip untagged repos</span><br><span style="color: hsl(120, 100%, 40%);">+ if [ -z "$tags" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ echo " (repository has no release tags)"</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ fi</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Build missing tarballs for each tag</span><br><span style="color: hsl(120, 100%, 40%);">+ for tag in $tags; do</span><br><span style="color: hsl(120, 100%, 40%);">+ tarball="$repo-$tag.tar.bz2"</span><br><span style="color: hsl(120, 100%, 40%);">+ if ignore_tag "$repo" "$tag"; then</span><br><span style="color: hsl(120, 100%, 40%);">+ echo " $tarball (ignored)"</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ elif [ -e "$OUTPUT/$repo/$tarball" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ echo " $tarball (exists)"</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ fi</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ echo " $tarball (creating)"</span><br><span style="color: hsl(120, 100%, 40%);">+ create_tarball "$repo" "$tag"</span><br><span style="color: hsl(120, 100%, 40%);">+ move_tarball "$repo" "$tag"</span><br><span style="color: hsl(120, 100%, 40%);">+ done</span><br><span style="color: hsl(120, 100%, 40%);">+done</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+remove_temp_dir</span><br><span style="color: hsl(120, 100%, 40%);">+echo "done!"</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-ci/+/14651">change 14651</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-ci/+/14651"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-ci </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I0a904c26e1c45aa46ddc59a01b776f2c7c5f947d </div>
<div style="display:none"> Gerrit-Change-Number: 14651 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: osmith <osmith@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: laforge <laforge@gnumonks.org> </div>
<div style="display:none"> Gerrit-Reviewer: osmith <osmith@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>