- gerrit-log - lists.osmocom.org

Change in libosmo-netif[master]: rtp: Delay rtph ptr assign after validating length
by pespin 29 Jun '22

29 Jun '22

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/libosmo-netif/+/28479 ) Change subject: rtp: Delay rtph ptr assign after validating length ...................................................................... rtp: Delay rtph ptr assign after validating length Make coverity happy, since this seems to spread as a tainted error further below. Related: Coverity CID#273000 Change-Id: I5d457183043d4c902f473b828815b9c62a01d47d --- M src/rtp.c 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/libosmo-netif refs/changes/79/28479/1 diff --git a/src/rtp.c b/src/rtp.c index 5718c5f..5c06c42 100644 --- a/src/rtp.c +++ b/src/rtp.c @@ -89,13 +89,14 @@ struct rtp_hdr *osmo_rtp_get_hdr(struct msgb *msg) { - struct rtp_hdr *rtph = (struct rtp_hdr *)msg->data; + struct rtp_hdr *rtph; if (msg->len < sizeof(struct rtp_hdr)) { DEBUGPC(DLMUX, "received RTP frame too short (len = %d)\n", msg->len); return NULL; } + rtph = (struct rtp_hdr *)msg->data; if (rtph->version != RTP_VERSION) { DEBUGPC(DLMUX, "received RTP version %d not supported.\n", rtph->version); -- To view, visit https://gerrit.osmocom.org/c/libosmo-netif/+/28479 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmo-netif Gerrit-Branch: master Gerrit-Change-Id: I5d457183043d4c902f473b828815b9c62a01d47d Gerrit-Change-Number: 28479 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

Change in libosmocore[master]: cbsp: avoid potential msgb write overflow in osmo_cbsp_recv_buffered
by pespin 29 Jun '22

29 Jun '22

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/libosmocore/+/28478 ) Change subject: cbsp: avoid potential msgb write overflow in osmo_cbsp_recv_buffered ...................................................................... cbsp: avoid potential msgb write overflow in osmo_cbsp_recv_buffered >>> CID 273001: Insecure data handling (TAINTED_SCALAR) >>> Passing tainted expression "needed" to "recv", which uses it as an offset. 1444 rc = recv(fd, msg->tail, needed, 0); Fixes: Coverity CID#273001 Change-Id: I17c558254f9c7907b56d61c53c2f597e8e4566cf --- M src/gsm/cbsp.c 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/78/28478/1 diff --git a/src/gsm/cbsp.c b/src/gsm/cbsp.c index 2095003..a31517b 100644 --- a/src/gsm/cbsp.c +++ b/src/gsm/cbsp.c @@ -1441,6 +1441,10 @@ needed = len - msgb_l2len(msg); if (needed > 0) { + if (needed > msgb_tailroom(msg)) { + rc = -ENOMEM; + goto discard_msg; + } rc = recv(fd, msg->tail, needed, 0); if (rc == 0) goto discard_msg; -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/28478 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: I17c558254f9c7907b56d61c53c2f597e8e4566cf Gerrit-Change-Number: 28478 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

Change in osmo-ci[master]: jobs/osmocom-release-tarballs: move to dp
by osmith 29 Jun '22

29 Jun '22

osmith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ci/+/28477 ) Change subject: jobs/osmocom-release-tarballs: move to dp ...................................................................... jobs/osmocom-release-tarballs: move to dp Move the long shell command into a jenkins.sh in docker-playground, and move the osmocom-release-tarballs.sh script there as well. This makes it easier to test the script, and only patching one repository is needed from now on when changing to a newer debian version. Depends: docker-playground I4cf66097f508c029f688ec8af1cab2238d42f84f Change-Id: I15f6c08b807f1462f91e95b624cdc1aad0592dd6 --- M jobs/osmocom-release-tarballs.yml D scripts/osmocom-release-tarballs.sh 2 files changed, 4 insertions(+), 315 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/77/28477/1 diff --git a/jobs/osmocom-release-tarballs.yml b/jobs/osmocom-release-tarballs.yml index 930fb42..0bc1fff 100644 --- a/jobs/osmocom-release-tarballs.yml +++ b/jobs/osmocom-release-tarballs.yml @@ -14,44 +14,19 @@ Publishes to: <a href="https://ftp.osmocom.org/releases/">https://ftp.osmocom.org/releases</a> parameters: - string: - name: BRANCH_OSMO_CI - description: | - osmo-ci.git branch where the osmocom-release-tarballs.sh gets pulled from - default: 'master' - - string: name: BRANCH_DOCKER_PLAYGROUND description: | docker-playground.git branch where the Dockerfile gets pulled from default: 'master' builders: - shell: | - export OSMO_BRANCH_DOCKER_PLAYGROUND="$BRANCH_DOCKER_PLAYGROUND" - . scripts/common.sh - docker_images_require \ - "debian-stretch-obs-latest" \ - "debian-stretch-build-dist" - - docker run \ - --rm=true \ - -v "$PWD:/osmo-ci" \ - -w /osmo-ci \ - -e KEEP_TEMP="$KEEP_TEMP" \ - "$USER/debian-stretch-build-dist" /osmo-ci/scripts/osmocom-release-tarballs.sh - - cat > "$WORKSPACE/known_hosts" <<EOF - [ftp.osmocom.org]:48 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDgQ9HntlpWNmh953a2Gc8NysKE4orOatVT1wQkyzhARnfYUerRuwyNr1GqMyBKdSI9amYVBXJIOUFcpV81niA7zQRUs66bpIMkE9/rHxBd81SkorEPOIS84W4vm3SZtuNqa+fADcqe88Hcb0ZdTzjKILuwi19gzrQyME2knHY71EOETe9Yow5RD2hTIpB5ecNxI0LUKDq+Ii8HfBvndPBIr0BWYDugckQ3Bocf+yn/tn2/GZieFEyFpBGF/MnLbAAfUKIdeyFRX7ufaiWWz5yKAfEhtziqdAGZaXNaLG6gkpy3EixOAy6ZXuTAk3b3Y0FUmDjhOHllbPmTOcKMry9 - [ftp.osmocom.org]:48 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPdWn1kEousXuKsZ+qJEZTt/NSeASxCrUfNDW3LWtH+d8Ust7ZuKp/vuyG+5pe5pwpPOgFu7TjN+0lVjYJVXH54= - [ftp.osmocom.org]:48 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK8iivY70EiR5NiGChV39gRLjNpC8lvu1ZdHtdMw2zuX - EOF - - SSH_COMMAND="ssh -o 'UserKnownHostsFile=$WORKSPACE/known_hosts' -p 48" - cd _release_tarballs - rsync -avz --delete -e "$SSH_COMMAND" . releases@ftp.osmocom.org:web-files/ + cd release-tarball-build-dist + ./jenkins.sh scm: - git: branches: - - '$BRANCH_OSMO_CI' - url: git://git.osmocom.org/osmo-ci + - '$BRANCH_DOCKER_PLAYGROUND' + url: git://git.osmocom.org/docker-playground git-config-name: 'Jenkins Builder' git-config-email: 'jenkins(a)osmocom.org' # Don't wipe workspace to keep generated release tarballs diff --git a/scripts/osmocom-release-tarballs.sh b/scripts/osmocom-release-tarballs.sh deleted file mode 100755 index 3ac4545..0000000 --- a/scripts/osmocom-release-tarballs.sh +++ /dev/null @@ -1,286 +0,0 @@ -#!/bin/sh -e -# Iterate over all relevant Osmocom repositories and generate release tarballs for each of the repository tags. The tags -# are queried from the git server without cloning the repositories first, so we can clone them only if we need to build -# a missing tarball. All repositories are deleted afterwards to save space. -# -# Environment variables: -# * KEEP_TEMP: do not delete cloned repositories (use for development) -# * PARALLEL_MAKE: -jN argument for make (default: -j5). - -. "$(dirname "$0")/common.sh" -cd "$(dirname "$0")" -PARALLEL_MAKE="${PARALLEL_MAKE:--j5}" -OUTPUT="$(cd ..; pwd)/_release_tarballs" -TEMP="$(cd ..; pwd)/_temp" - -# Print all tags for which no release tarball should be built. -# $1: Osmocom repository -tags_to_ignore() { - case "$1" in - libosmocore) - # configure.ac:144: error: required file 'src/gb/Makefile.in' not found - echo "0.5.0" - echo "0.5.1" - ;; - libsmpp34) - # duplicate of 1.12.0 - echo "1.12" - ;; - osmo-bsc) - # openbsc - echo "1.0.1" - # Requires libosmo-legacy-mgcp - echo "1.1.0" - echo "1.1.1" - echo "1.1.2" - echo "1.2.0" - echo "1.2.1" - echo "1.2.2" - ;; - osmo-bts) - # gsm_data_shared.h:464:26: error: field 'power_params' has incomplete type - echo "0.2.0" - echo "0.3.0" - ;; - osmo-hlr) - # Not using autotools - echo "0.0.1" - ;; - osmo-mgw) - # openbsc - echo "1.0.1" - ;; - osmo-msc) - # openbsc - echo "1.0.1" - ;; - osmo-pcap) - # No rule to make target 'osmo-pcap-server.cfg', needed by 'distdir' - echo "0.0.3" - ;; - osmo-pcu) - # Duplicates of 0.1.0, 0.2.0 - echo "0.1" - echo "0.2" - ;; - osmo-sgsn) - # openbsc - echo "0.9.0 0.9.1 0.9.2 0.9.3 0.9.4 0.9.5 0.9.6 0.9.8 0.9.9 0.9.10 0.9.11 0.9.12 0.9.13 0.9.14" - echo "0.9.15 0.9.16 0.10.0 0.10.1 0.11.0 0.12.0 0.13.0 0.14.0 0.15.0 1.0.1" - ;; - osmo-sip-connector) - # make: *** No rule to make target 'osmoappdesc.py' - echo "0.0.1" - ;; - osmo-trx) - # cp: cannot stat './/home/user/code/osmo-dev/src/osmo-ci/_temp/repos/osmo-trx/configure' - echo "0.2.0" - echo "0.3.0" - echo "1.3.0" - ;; - esac -} - -# Clone dependency repositories. -# $1: Osmocom repository -prepare_depends() { - case "$1" in - osmo-bts) - # Includes openbsc/gsm_data_shared.h - prepare_repo "openbsc" - ;; - esac -} - -# Apply workarounds for bugs that break too many releases. This function runs between ./configure and make dist-bzip2. -# $1: Osmocom repository -fix_repo() { - case "$1" in - osmo-mgw) - # No rule to make target 'osmocom/mgcp_client/mgcp_common.h' (OS#4084) - make -C "$TEMP/repos/$1/include/osmocom/mgcp_client" mgcp_common.h || true - ;; - esac -} - -# Check if one specific tag should be ignored. -# $1: Osmocom repository -# $2: tag (e.g. "1.0.0") -ignore_tag() { - local repo="$1" - local tag="$2" - local tags="$(tags_to_ignore "$repo")" - for tag_i in $tags; do - if [ "$tag" = "$tag_i" ]; then - return 0 - fi - done - return 1 -} - -# Delete existing temp dir (unless KEEP_TEMP is set). If all repos were checked out, this restores ~500 MB of space. -remove_temp_dir() { - if [ -n "$KEEP_TEMP" ]; then - echo "NOTE: not removing temp dir, because KEEP_TEMP is set: $TEMP" - elif [ -d "$TEMP" ]; then - rm -rf "$TEMP" - fi -} - -# Clone an Osmocom repository to $TEMP/repos/$repo, clean it, checkout a tag. -# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) -# $2: tag (optional, default: master) -prepare_repo() { - local repo="$1" - local tag="${2:-master}" - - if ! [ -d "$TEMP/repos/$repo" ]; then - git -C "$TEMP/repos" clone "$OSMO_GIT_URL/$repo" - fi - - cd "$TEMP/repos/$repo" - git clean -qdxf - git reset --hard HEAD # in case the tracked files were modified (e.g. libsmpp34 1.10) - git checkout -q "$tag" -} - - -# Get the desired tarball name, replace / with - in $1. -# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) -# $2: tag -tarball_name() { - echo "$(echo "$repo" | tr / -)-$tag.tar.bz2" -} - -# Checkout a given tag and build a release tarball. -# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) -# $2: tag -create_tarball() { - local repo="$1" - local tag="$2" - local tarball="$(tarball_name "$repo" "$tag")" - - # Be verbose during the tarball build and preparation. Everything else is not verbose, so we can generate an - # easy to read overview of tarballs that are already built or are ignored. - set -x - - prepare_repo "$repo" "$tag" - prepare_depends "$repo" - - cd "$TEMP/repos/$repo" - autoreconf -fi - ./configure - fix_repo "$repo" - make dist-bzip2 - - # Back to non-verbose mode - set +x - - if ! [ -e "$tarball" ]; then - echo "NOTE: tarball has a different name (wrong version in configure.ac?), renaming." - mv -v *.tar.bz2 "$tarball" - fi -} - -# Create a release tarball with "git archive" for non-autotools projects. -# $1: Osmocom repository -# $2: tag -create_tarball_git() { - local repo="$1" - local tag="$2" - local tarball="$(tarball_name "$repo" "$tag")" - - set -x - - cd "$TEMP/repos/$repo" - git archive \ - -o "$tarball" \ - "$tag" - - set +x -} - -# Move a generated release tarball to the output dir. -# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) -# $2: tag -move_tarball() { - local repo="$1" - local tag="$2" - local tarball="$(tarball_name "$repo" "$tag")" - local repo_dir="$(echo "$repo" | cut -d / -f 1)" - - cd "$TEMP/repos/$repo" - mkdir -p "$OUTPUT/$repo_dir" - mv "$tarball" "$OUTPUT/$repo_dir/$tarball" -} - -# Check if a git tag has a specific file -# $1: Osmocom repository -# $2: tag -# $3: file -tag_has_file() { - local repo="$1" - local tag="$2" - local file="$3" - - git -C "$TEMP/repos/$repo" show "$tag:$file" >/dev/null 2>&1 -} - -# Create and move tarballs for Osmocom repositories. -# $1: Osmocom repository -# $2: tag -create_move_tarball() { - local repo="$1" - local tag="$2" - - case "$repo" in - simtrace2) - if tag_has_file "$repo" "$tag" host/configure.ac; then - create_tarball "$repo/host" "$tag" - move_tarball "$repo/host" "$tag" - else - prepare_repo "$repo" - fi - - create_tarball_git "$repo" "$tag" - move_tarball "$repo" "$tag" - ;; - *) - create_tarball "$repo" "$tag" - move_tarball "$repo" "$tag" - ;; - esac -} - -remove_temp_dir -mkdir -p "$TEMP/repos" -echo "Temp dir: $TEMP" - -for repo in $OSMO_RELEASE_REPOS; do - echo "$repo" - tags="$(osmo_git_last_commits_tags "$repo" "all" | cut -d / -f 3)" - - # Skip untagged repos - if [ -z "$tags" ]; then - echo " (repository has no release tags)" - continue - fi - - # Build missing tarballs for each tag - for tag in $tags; do - tarball="$repo-$tag.tar.bz2" - if ignore_tag "$repo" "$tag"; then - echo " $tarball (ignored)" - continue - elif [ -e "$OUTPUT/$repo/$tarball" ]; then - echo " $tarball (exists)" - continue - fi - - echo " $tarball (creating)" - create_move_tarball "$repo" "$tag" - done -done - -remove_temp_dir -echo "done!" -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/28477 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: I15f6c08b807f1462f91e95b624cdc1aad0592dd6 Gerrit-Change-Number: 28477 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

Change in osmo-bsc[master]: cbsp: Change log level ERROR->INFO on CBSP tx and link down
by pespin 29 Jun '22

29 Jun '22

pespin has submitted this change. ( https://gerrit.osmocom.org/c/osmo-bsc/+/28429 ) Change subject: cbsp: Change log level ERROR->INFO on CBSP tx and link down ...................................................................... cbsp: Change log level ERROR->INFO on CBSP tx and link down Let's decrease the logging since it's fine simply discarding the message if the link is down. This way all code sending messages doesn't need to care about the link state. Change-Id: I64356ec6a7b3a4e11a0e66b17efab2788b1ca5cc --- M src/osmo-bsc/cbsp_link.c 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: laforge: Looks good to me, approved fixeria: Looks good to me, but someone else must approve Jenkins Builder: Verified diff --git a/src/osmo-bsc/cbsp_link.c b/src/osmo-bsc/cbsp_link.c index d8c32ec..849b802 100644 --- a/src/osmo-bsc/cbsp_link.c +++ b/src/osmo-bsc/cbsp_link.c @@ -301,7 +301,7 @@ struct msgb *msg; if (!cbc->client.cli && !cbc->server.srv) { - LOGP(DCBS, LOGL_ERROR, "Discarding Tx CBSP Message Type %s, link is down\n", + LOGP(DCBS, LOGL_INFO, "Discarding Tx CBSP Message Type %s, link is down\n", get_value_string(cbsp_msg_type_names, cbsp->msg_type)); talloc_free(cbsp); return 0; -- To view, visit https://gerrit.osmocom.org/c/osmo-bsc/+/28429 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-bsc Gerrit-Branch: master Gerrit-Change-Id: I64356ec6a7b3a4e11a0e66b17efab2788b1ca5cc Gerrit-Change-Number: 28429 Gerrit-PatchSet: 3 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in libosmo-abis[master]: configure.ac: Set libosmo-e1d dependency version
by pespin 29 Jun '22

29 Jun '22

pespin has submitted this change. ( https://gerrit.osmocom.org/c/libosmo-abis/+/28467 ) Change subject: configure.ac: Set libosmo-e1d dependency version ...................................................................... configure.ac: Set libosmo-e1d dependency version Change-Id: I2c6e50f73090a2134485d68a189036bc0aa00266 --- M configure.ac 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved fixeria: Looks good to me, but someone else must approve diff --git a/configure.ac b/configure.ac index 46ff48c..1c2988d 100644 --- a/configure.ac +++ b/configure.ac @@ -95,7 +95,7 @@ ENABLE_E1D="no" ]) AS_IF([test "x$ENABLE_E1D" = "xyes"], [ - PKG_CHECK_MODULES(LIBOSMOE1D, libosmo-e1d) + PKG_CHECK_MODULES(LIBOSMOE1D, libosmo-e1d >= 0.4.0) ]) AM_CONDITIONAL(ENABLE_E1D, test "x$ENABLE_E1D" = "xyes") AC_SUBST(ENABLE_E1D) -- To view, visit https://gerrit.osmocom.org/c/libosmo-abis/+/28467 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmo-abis Gerrit-Branch: master Gerrit-Change-Id: I2c6e50f73090a2134485d68a189036bc0aa00266 Gerrit-Change-Number: 28467 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in libosmo-abis[master]: configure.ac: Set libosmo-e1d dependency version
by fixeria 29 Jun '22

29 Jun '22

Attention is currently required from: pespin. fixeria has posted comments on this change. ( https://gerrit.osmocom.org/c/libosmo-abis/+/28467 ) Change subject: configure.ac: Set libosmo-e1d dependency version ...................................................................... Patch Set 1: Code-Review+1 -- To view, visit https://gerrit.osmocom.org/c/libosmo-abis/+/28467 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmo-abis Gerrit-Branch: master Gerrit-Change-Id: I2c6e50f73090a2134485d68a189036bc0aa00266 Gerrit-Change-Number: 28467 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Comment-Date: Wed, 29 Jun 2022 14:45:42 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

Change in osmo-ci[master]: coverity/build_Osmocom.sh: run 'make check' in do_build()
by fixeria 29 Jun '22

29 Jun '22

fixeria has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/28451 ) Change subject: coverity/build_Osmocom.sh: run 'make check' in do_build() ...................................................................... coverity/build_Osmocom.sh: run 'make check' in do_build() This enables building and running tests (check_PROGRAMS), so that Coverity will be able to find bugs in their code too. Change-Id: I3f3149a61df96646798c72d0ad31c41b97e6b81f --- M coverity/build_Osmocom.sh 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: laforge: Looks good to me, but someone else must approve osmith: Looks good to me, approved Jenkins Builder: Verified diff --git a/coverity/build_Osmocom.sh b/coverity/build_Osmocom.sh index a7f8cdd..fe9806c 100755 --- a/coverity/build_Osmocom.sh +++ b/coverity/build_Osmocom.sh @@ -18,7 +18,7 @@ --disable-doxygen \ $* - make $PARALLEL_MAKE + make $PARALLEL_MAKE check make install } -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/28451 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: I3f3149a61df96646798c72d0ad31c41b97e6b81f Gerrit-Change-Number: 28451 Gerrit-PatchSet: 2 Gerrit-Owner: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in docker-playground[master]: release-tarball-build-dist: move osmo-ci code here
by laforge 29 Jun '22

29 Jun '22

laforge has submitted this change. ( https://gerrit.osmocom.org/c/docker-playground/+/28462 ) Change subject: release-tarball-build-dist: move osmo-ci code here ...................................................................... release-tarball-build-dist: move osmo-ci code here Add a jenkins.sh that does the same as the previous lengthy shell command in the jenkins-job-builder file, and add osmocom-release-tarballs.sh. This makes it easier to test the script, and only patching one repository is needed from now on when changing to a newer debian version. Depends: osmo-ci I15f6c08b807f1462f91e95b624cdc1aad0592dd6 Change-Id: I4cf66097f508c029f688ec8af1cab2238d42f84f --- M .gitignore A release-tarball-build-dist/jenkins.sh A release-tarball-build-dist/osmocom-release-tarballs.sh 3 files changed, 380 insertions(+), 0 deletions(-) Approvals: Jenkins Builder: Verified pespin: Looks good to me, but someone else must approve laforge: Looks good to me, approved diff --git a/.gitignore b/.gitignore index 9eb94ca..fe1b979 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,6 @@ *.gen.cfg /_cache + +/release-tarball-build-dist/_release_tarballs +/release-tarball-build-dist/_temp diff --git a/release-tarball-build-dist/jenkins.sh b/release-tarball-build-dist/jenkins.sh new file mode 100755 index 0000000..0177da8 --- /dev/null +++ b/release-tarball-build-dist/jenkins.sh @@ -0,0 +1,28 @@ +#!/bin/sh -ex +. ../jenkins-common.sh + +docker_images_require \ + "debian-bullseye-obs-latest" \ + "release-tarball-build-dist" + +docker run \ + --rm=true \ + -v "$PWD:/build" \ + -w /osmo-ci \ + -e KEEP_TEMP="$KEEP_TEMP" \ + "$USER/release-tarball-build-dist" sh -e /build/osmocom-release-tarballs.sh + +if [ -z "$WORKSPACE" ]; then + set +x + echo "NOTE: not running on jenkins, skipping upload" +fi + +cat > "$WORKSPACE/known_hosts" <<EOF +[ftp.osmocom.org]:48 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDgQ9HntlpWNmh953a2Gc8NysKE4orOatVT1wQkyzhARnfYUerRuwyNr1GqMyBKdSI9amYVBXJIOUFcpV81niA7zQRUs66bpIMkE9/rHxBd81SkorEPOIS84W4vm3SZtuNqa+fADcqe88Hcb0ZdTzjKILuwi19gzrQyME2knHY71EOETe9Yow5RD2hTIpB5ecNxI0LUKDq+Ii8HfBvndPBIr0BWYDugckQ3Bocf+yn/tn2/GZieFEyFpBGF/MnLbAAfUKIdeyFRX7ufaiWWz5yKAfEhtziqdAGZaXNaLG6gkpy3EixOAy6ZXuTAk3b3Y0FUmDjhOHllbPmTOcKMry9 +[ftp.osmocom.org]:48 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPdWn1kEousXuKsZ+qJEZTt/NSeASxCrUfNDW3LWtH+d8Ust7ZuKp/vuyG+5pe5pwpPOgFu7TjN+0lVjYJVXH54= +[ftp.osmocom.org]:48 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK8iivY70EiR5NiGChV39gRLjNpC8lvu1ZdHtdMw2zuX +EOF + +SSH_COMMAND="ssh -o 'UserKnownHostsFile=$WORKSPACE/known_hosts' -p 48" +cd _release_tarballs +rsync -avz --delete -e "$SSH_COMMAND" . releases@ftp.osmocom.org:web-files/ diff --git a/release-tarball-build-dist/osmocom-release-tarballs.sh b/release-tarball-build-dist/osmocom-release-tarballs.sh new file mode 100644 index 0000000..02b81a6 --- /dev/null +++ b/release-tarball-build-dist/osmocom-release-tarballs.sh @@ -0,0 +1,349 @@ +#!/bin/sh -e +# Iterate over all relevant Osmocom repositories and generate release tarballs for each of the repository tags. The tags +# are queried from the git server without cloning the repositories first, so we can clone them only if we need to build +# a missing tarball. All repositories are deleted afterwards to save space. +# +# Environment variables: +# * KEEP_TEMP: do not delete cloned repositories (use for development) +# * PARALLEL_MAKE: -jN argument for make (default: -j5). +OSMO_GIT_URL="https://git.osmocom.org" +OSMO_RELEASE_REPOS=" + libasn1c + libosmo-abis + libosmo-netif + libosmo-sccp + libosmocore + libsmpp34 + libusrp + osmo-bsc + osmo-bts + osmo-cbc + osmo-e1d + osmo-gbproxy + osmo-ggsn + osmo-gsm-manuals + osmo-hlr + osmo-hnodeb + osmo-hnbgw + osmo-iuh + osmo-mgw + osmo-msc + osmo-pcap + osmo-pcu + osmo-remsim + osmo-sgsn + osmo-sip-connector + osmo-smlc + osmo-sysmon + osmo-trx + osmo-uecups + osmocom-bb + simtrace2 +" + +# Print last tags and related commits for an Osmocom git repository, e.g.: +# "ec798b89700dcca5c5b28edf1a1cd16ea311f30a refs/tags/1.0.1" +# $1: Osmocom repository +# $2: amount of commit, tag pairs to print (default: 1, set to "all" to print all) +# $3: string to print when there are no tags (default: empty string) +osmo_git_last_commits_tags() { + # git output: + # ec798b89700dcca5c5b28edf1a1cd16ea311f30a refs/tags/1.0.1 + # eab5f594b0a7cf50ad97b039f73beff42cc8312a refs/tags/1.0.1^{} + # ... + # 41e7cf115d4148a9f34fcb863b68b2d5370e335d refs/tags/1.3.1^{} + # 8a9f12dc2f69bf3a4e861cc9a81b71bdc5f13180 refs/tags/3G_2016_09 + # ee618ecbedec82dfd240334bc87d0d1c806477b0 refs/tags/debian/0.9.13-0_jrsantos.1 + # a3fdd24af099b449c9856422eb099fb45a5595df refs/tags/debian/0.9.13-0_jrsantos.1^{} + # ... + ret="$(git ls-remote --tags "$OSMO_GIT_URL/$1")" + ret="$(echo "$ret" | grep 'refs/tags/[0-9.]*$' || true)" + ret="$(echo "$ret" | sort -V -t/ -k3)" + if [ "$2" != "all" ]; then + ret="$(echo "$ret" | tail -n "$2")" + fi + + if [ -n "$ret" ]; then + echo "$ret" + else + echo "$3" + fi +} + + +cd "$(dirname "$0")" +PARALLEL_MAKE="${PARALLEL_MAKE:--j5}" +OUTPUT="/build/_release_tarballs" +TEMP="/build/_temp" + +# Print all tags for which no release tarball should be built. +# $1: Osmocom repository +tags_to_ignore() { + case "$1" in + libosmocore) + # configure.ac:144: error: required file 'src/gb/Makefile.in' not found + echo "0.5.0" + echo "0.5.1" + ;; + libsmpp34) + # duplicate of 1.12.0 + echo "1.12" + ;; + osmo-bsc) + # openbsc + echo "1.0.1" + # Requires libosmo-legacy-mgcp + echo "1.1.0" + echo "1.1.1" + echo "1.1.2" + echo "1.2.0" + echo "1.2.1" + echo "1.2.2" + ;; + osmo-bts) + # gsm_data_shared.h:464:26: error: field 'power_params' has incomplete type + echo "0.2.0" + echo "0.3.0" + ;; + osmo-hlr) + # Not using autotools + echo "0.0.1" + ;; + osmo-mgw) + # openbsc + echo "1.0.1" + ;; + osmo-msc) + # openbsc + echo "1.0.1" + ;; + osmo-pcap) + # No rule to make target 'osmo-pcap-server.cfg', needed by 'distdir' + echo "0.0.3" + ;; + osmo-pcu) + # Duplicates of 0.1.0, 0.2.0 + echo "0.1" + echo "0.2" + ;; + osmo-sgsn) + # openbsc + echo "0.9.0 0.9.1 0.9.2 0.9.3 0.9.4 0.9.5 0.9.6 0.9.8 0.9.9 0.9.10 0.9.11 0.9.12 0.9.13 0.9.14" + echo "0.9.15 0.9.16 0.10.0 0.10.1 0.11.0 0.12.0 0.13.0 0.14.0 0.15.0 1.0.1" + ;; + osmo-sip-connector) + # make: *** No rule to make target 'osmoappdesc.py' + echo "0.0.1" + ;; + osmo-trx) + # cp: cannot stat './/home/user/code/osmo-dev/src/osmo-ci/_temp/repos/osmo-trx/configure' + echo "0.2.0" + echo "0.3.0" + echo "1.3.0" + ;; + esac +} + +# Clone dependency repositories. +# $1: Osmocom repository +prepare_depends() { + case "$1" in + osmo-bts) + # Includes openbsc/gsm_data_shared.h + prepare_repo "openbsc" + ;; + esac +} + +# Apply workarounds for bugs that break too many releases. This function runs between ./configure and make dist-bzip2. +# $1: Osmocom repository +fix_repo() { + case "$1" in + osmo-mgw) + # No rule to make target 'osmocom/mgcp_client/mgcp_common.h' (OS#4084) + make -C "$TEMP/repos/$1/include/osmocom/mgcp_client" mgcp_common.h || true + ;; + esac +} + +# Check if one specific tag should be ignored. +# $1: Osmocom repository +# $2: tag (e.g. "1.0.0") +ignore_tag() { + local repo="$1" + local tag="$2" + local tags="$(tags_to_ignore "$repo")" + for tag_i in $tags; do + if [ "$tag" = "$tag_i" ]; then + return 0 + fi + done + return 1 +} + +# Delete existing temp dir (unless KEEP_TEMP is set). If all repos were checked out, this restores ~500 MB of space. +remove_temp_dir() { + if [ -n "$KEEP_TEMP" ]; then + echo "NOTE: not removing temp dir, because KEEP_TEMP is set: $TEMP" + elif [ -d "$TEMP" ]; then + rm -rf "$TEMP" + fi +} + +# Clone an Osmocom repository to $TEMP/repos/$repo, clean it, checkout a tag. +# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) +# $2: tag (optional, default: master) +prepare_repo() { + local repo="$1" + local tag="${2:-master}" + + if ! [ -d "$TEMP/repos/$repo" ]; then + git -C "$TEMP/repos" clone "$OSMO_GIT_URL/$repo" + fi + + cd "$TEMP/repos/$repo" + git clean -qdxf + git reset --hard HEAD # in case the tracked files were modified (e.g. libsmpp34 1.10) + git checkout -q "$tag" +} + + +# Get the desired tarball name, replace / with - in $1. +# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) +# $2: tag +tarball_name() { + echo "$(echo "$repo" | tr / -)-$tag.tar.bz2" +} + +# Checkout a given tag and build a release tarball. +# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) +# $2: tag +create_tarball() { + local repo="$1" + local tag="$2" + local tarball="$(tarball_name "$repo" "$tag")" + + # Be verbose during the tarball build and preparation. Everything else is not verbose, so we can generate an + # easy to read overview of tarballs that are already built or are ignored. + set -x + + prepare_repo "$repo" "$tag" + prepare_depends "$repo" + + cd "$TEMP/repos/$repo" + autoreconf -fi + ./configure + fix_repo "$repo" + make dist-bzip2 + + # Back to non-verbose mode + set +x + + if ! [ -e "$tarball" ]; then + echo "NOTE: tarball has a different name (wrong version in configure.ac?), renaming." + mv -v *.tar.bz2 "$tarball" + fi +} + +# Create a release tarball with "git archive" for non-autotools projects. +# $1: Osmocom repository +# $2: tag +create_tarball_git() { + local repo="$1" + local tag="$2" + local tarball="$(tarball_name "$repo" "$tag")" + + set -x + + cd "$TEMP/repos/$repo" + git archive \ + -o "$tarball" \ + "$tag" + + set +x +} + +# Move a generated release tarball to the output dir. +# $1: Osmocom repository (may end in subdir, e.g. simtrace2/host) +# $2: tag +move_tarball() { + local repo="$1" + local tag="$2" + local tarball="$(tarball_name "$repo" "$tag")" + local repo_dir="$(echo "$repo" | cut -d / -f 1)" + + cd "$TEMP/repos/$repo" + mkdir -p "$OUTPUT/$repo_dir" + mv "$tarball" "$OUTPUT/$repo_dir/$tarball" +} + +# Check if a git tag has a specific file +# $1: Osmocom repository +# $2: tag +# $3: file +tag_has_file() { + local repo="$1" + local tag="$2" + local file="$3" + + git -C "$TEMP/repos/$repo" show "$tag:$file" >/dev/null 2>&1 +} + +# Create and move tarballs for Osmocom repositories. +# $1: Osmocom repository +# $2: tag +create_move_tarball() { + local repo="$1" + local tag="$2" + + case "$repo" in + simtrace2) + if tag_has_file "$repo" "$tag" host/configure.ac; then + create_tarball "$repo/host" "$tag" + move_tarball "$repo/host" "$tag" + else + prepare_repo "$repo" + fi + + create_tarball_git "$repo" "$tag" + move_tarball "$repo" "$tag" + ;; + *) + create_tarball "$repo" "$tag" + move_tarball "$repo" "$tag" + ;; + esac +} + +remove_temp_dir +mkdir -p "$TEMP/repos" +echo "Temp dir: $TEMP" + +for repo in $OSMO_RELEASE_REPOS; do + echo "$repo" + tags="$(osmo_git_last_commits_tags "$repo" "all" | cut -d / -f 3)" + + # Skip untagged repos + if [ -z "$tags" ]; then + echo " (repository has no release tags)" + continue + fi + + # Build missing tarballs for each tag + for tag in $tags; do + tarball="$repo-$tag.tar.bz2" + if ignore_tag "$repo" "$tag"; then + echo " $tarball (ignored)" + continue + elif [ -e "$OUTPUT/$repo/$tarball" ]; then + echo " $tarball (exists)" + continue + fi + + echo " $tarball (creating)" + create_move_tarball "$repo" "$tag" + done +done + +remove_temp_dir +echo "done!" -- To view, visit https://gerrit.osmocom.org/c/docker-playground/+/28462 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: docker-playground Gerrit-Branch: master Gerrit-Change-Id: I4cf66097f508c029f688ec8af1cab2238d42f84f Gerrit-Change-Number: 28462 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in docker-playground[master]: release-tarball-build-dist: upgrade from debian 9
by laforge 29 Jun '22

29 Jun '22

laforge has submitted this change. ( https://gerrit.osmocom.org/c/docker-playground/+/28461 ) Change subject: release-tarball-build-dist: upgrade from debian 9 ...................................................................... release-tarball-build-dist: upgrade from debian 9 Let it use debian bullseye and rename the directory once, so we don't need to rename it in the future when switching the debian version again. The reason I'm doing this now is that building with debian 9 failed due to obs.osmocom.org related changes. It would be trivial to fix that, but I decided to rather upgrade to the latest debian version. Change-Id: Ib6039724a7200abce5252f95407047296b72cd51 --- R release-tarball-build-dist/Dockerfile R release-tarball-build-dist/Makefile 2 files changed, 1 insertion(+), 3 deletions(-) Approvals: Jenkins Builder: Verified pespin: Looks good to me, but someone else must approve laforge: Looks good to me, approved diff --git a/debian-stretch-build-dist/Dockerfile b/release-tarball-build-dist/Dockerfile similarity index 94% rename from debian-stretch-build-dist/Dockerfile rename to release-tarball-build-dist/Dockerfile index e444ed5..bc0215e 100644 --- a/debian-stretch-build-dist/Dockerfile +++ b/release-tarball-build-dist/Dockerfile @@ -1,5 +1,5 @@ ARG USER -FROM $USER/debian-stretch-obs-latest +FROM $USER/debian-bullseye-obs-latest # Arguments used after FROM must be specified again RUN apt-get update && \ @@ -66,8 +66,6 @@ libzmq3-dev \ make \ mscgen \ - python-nwdiag \ - python-pychart \ python3-nwdiag \ xsltproc \ && \ diff --git a/debian-stretch-build-dist/Makefile b/release-tarball-build-dist/Makefile similarity index 100% rename from debian-stretch-build-dist/Makefile rename to release-tarball-build-dist/Makefile -- To view, visit https://gerrit.osmocom.org/c/docker-playground/+/28461 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: docker-playground Gerrit-Branch: master Gerrit-Change-Id: Ib6039724a7200abce5252f95407047296b72cd51 Gerrit-Change-Number: 28461 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in docker-playground[master]: release-tarball-build-dist: move osmo-ci code here
by laforge 29 Jun '22

29 Jun '22

Attention is currently required from: osmith. laforge has posted comments on this change. ( https://gerrit.osmocom.org/c/docker-playground/+/28462 ) Change subject: release-tarball-build-dist: move osmo-ci code here ...................................................................... Patch Set 1: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/docker-playground/+/28462 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: docker-playground Gerrit-Branch: master Gerrit-Change-Id: I4cf66097f508c029f688ec8af1cab2238d42f84f Gerrit-Change-Number: 28462 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-Attention: osmith <osmith(a)sysmocom.de> Gerrit-Comment-Date: Wed, 29 Jun 2022 13:23:32 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

2025

2024

2023

2022

gerrit-log