September 2023 - gerrit-log - lists.osmocom.org

[S] Change in osmo-bts[master]: bts-trx: Fix CCCH not enabled if BS_AG_BLKS_RES!=1 is provided by BSC
by pespin 08 Sep '23

08 Sep '23

Attention is currently required from: fixeria, laforge. pespin has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-bts/+/34340?usp=email ) Change subject: bts-trx: Fix CCCH not enabled if BS_AG_BLKS_RES!=1 is provided by BSC ...................................................................... Patch Set 1: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/osmo-bts/+/34340?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-bts Gerrit-Branch: master Gerrit-Change-Id: I61e1681fbaa2c993b529d58b581c99166b62bda3 Gerrit-Change-Number: 34340 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Comment-Date: Fri, 08 Sep 2023 09:40:12 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

[S] Change in osmo-ci[master]: OBS: build_srcpkg.Dockerfile: use bookworm
by osmith 08 Sep '23

08 Sep '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/34331?usp=email ) Change subject: OBS: build_srcpkg.Dockerfile: use bookworm ...................................................................... OBS: build_srcpkg.Dockerfile: use bookworm rebar3 is in debian bookworm's repositories, hence it is not needed anymore to use osmocom:latest here. Change-Id: Iddd0295ee928fc9a47d4d206e414b78a2561d13d --- M scripts/obs/data/build_srcpkg.Dockerfile M scripts/obs/lib/config.py 2 files changed, 17 insertions(+), 15 deletions(-) Approvals: fixeria: Looks good to me, approved pespin: Looks good to me, but someone else must approve Jenkins Builder: Verified diff --git a/scripts/obs/data/build_srcpkg.Dockerfile b/scripts/obs/data/build_srcpkg.Dockerfile index f5d72cd..ebb30cd 100644 --- a/scripts/obs/data/build_srcpkg.Dockerfile +++ b/scripts/obs/data/build_srcpkg.Dockerfile @@ -1,29 +1,19 @@ -FROM debian:bullseye +# Change distro in lib/config.py:docker_distro_default +ARG DISTRO_FROM +FROM ${DISTRO_FROM} ARG UID RUN apt-get update && \ apt-get upgrade -y && \ apt-get install -y --no-install-recommends \ ca-certificates \ - gnupg2 \ - && \ - apt-get clean - -COPY Release.key /tmp/Release.key -RUN apt-key add /tmp/Release.key && \ - rm /tmp/Release.key && \ - echo "deb https://downloads.osmocom.org/packages/osmocom:/latest/Debian_11/ ./" \ - > /etc/apt/sources.list.d/osmocom-latest.list - -RUN apt-get update && \ - apt-get upgrade -y && \ - apt-get install -y --no-install-recommends \ debhelper \ dh-python \ dpkg-dev \ fakeroot \ git \ git-review \ + gnupg2 \ meson \ osc \ python3-setuptools \ diff --git a/scripts/obs/lib/config.py b/scripts/obs/lib/config.py index 3942f43..c3b172d 100644 --- a/scripts/obs/lib/config.py +++ b/scripts/obs/lib/config.py @@ -111,7 +111,7 @@ "open5gs": "^v[0-9]*\\.[0-9]*\\.[0-9]*$", } -docker_distro_default = "debian:11" +docker_distro_default = "debian:12" docker_distro_other = [ "almalinux:*", # instead of centos (SYS#5818) "debian:*", -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/34331?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: Iddd0295ee928fc9a47d4d206e414b78a2561d13d Gerrit-Change-Number: 34331 Gerrit-PatchSet: 2 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-ci[master]: OBS: lib/osc: add private proj variable
by osmith 08 Sep '23

08 Sep '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/34332?usp=email ) Change subject: OBS: lib/osc: add private proj variable ...................................................................... OBS: lib/osc: add private proj variable Adjust lib/osc.py to take the OBS project from a global variable that can be set with lib.osc.set_apiurl(), instead of using lib.args.obs_project. Move the sanity check to set_apiurl(). This is in preparation for adding a new sync script, which will operate on multiple OBS instances. Related: OS#6165 Change-Id: Iabb871bcb432f2fbdaec9cbcab3d663ccf969901 --- M scripts/obs/lib/osc.py M scripts/obs/update_obs_project.py 2 files changed, 29 insertions(+), 18 deletions(-) Approvals: fixeria: Looks good to me, approved pespin: Looks good to me, but someone else must approve Jenkins Builder: Verified diff --git a/scripts/obs/lib/osc.py b/scripts/obs/lib/osc.py index c9040ec..1113032 100644 --- a/scripts/obs/lib/osc.py +++ b/scripts/obs/lib/osc.py @@ -10,15 +10,7 @@ import lib.config apiurl = None - - -def check_proj(): - proj = lib.args.obs_project - if ":" in proj: - return - - print(f"ERROR: this doesn't look like a valid OBS project: {proj}") - exit(1) +proj = None def check_oscrc(): @@ -33,8 +25,16 @@ exit(1) -def set_apiurl(url): +def set_apiurl(url, obs_proj=None): global apiurl + global proj + + if obs_proj is not None: + if ":" not in obs_proj: + print(f"ERROR: this doesn't look like a valid OBS project: {obs_proj}") + exit(1) + proj = obs_proj + apiurl = url @@ -58,7 +58,6 @@ def get_remote_pkgs(): - proj = lib.args.obs_project print(f"OBS: getting packages in {proj}") ret = run_osc(["list", proj]) return ret.output.rstrip().split("\n") @@ -66,7 +65,6 @@ def get_package_version(package): feed = lib.args.feed - proj = lib.args.obs_project print(f"{package}: getting OBS version") ret = run_osc(["list", proj, os.path.basename(package)]) @@ -92,7 +90,6 @@ def create_package(package): - proj = lib.args.obs_project print(f"{package}: creating new OBS package") # cut off repository prefix like in "python/osmo-python-tests" @@ -117,7 +114,6 @@ def update_package(package, version): - proj = lib.args.obs_project print(f"{package}: updating OBS package") # cut off repository prefix like in "python/osmo-python-tests" @@ -146,5 +142,4 @@ def delete_package(package, commit_msg): print(f"{package}: removing from OBS ({commit_msg})") - run_osc(["rdelete", "-m", commit_msg, lib.args.obs_project, - os.path.basename(package)]) + run_osc(["rdelete", "-m", commit_msg, proj, os.path.basename(package)]) diff --git a/scripts/obs/update_obs_project.py b/scripts/obs/update_obs_project.py index 599e5cd..eee58a7 100755 --- a/scripts/obs/update_obs_project.py +++ b/scripts/obs/update_obs_project.py @@ -231,9 +231,8 @@ if args.docker: lib.docker.run_in_docker_and_exit("update_obs_project.py", True) - lib.osc.check_proj() lib.osc.check_oscrc() - lib.osc.set_apiurl(args.apiurl) + lib.osc.set_apiurl(args.apiurl, args.obs_project) if not args.ignore_req: lib.check_required_programs() -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/34332?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: Iabb871bcb432f2fbdaec9cbcab3d663ccf969901 Gerrit-Change-Number: 34332 Gerrit-PatchSet: 3 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[M] Change in osmo-ci[master]: jobs/osmocom-obs-sync: new job
by osmith 08 Sep '23

08 Sep '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/34334?usp=email ) Change subject: jobs/osmocom-obs-sync: new job ...................................................................... jobs/osmocom-obs-sync: new job Related: OS#6165 Change-Id: Ibcf02eb5feee2b314ff28861835040c21aea4294 --- A jobs/osmocom-obs-sync.yml 1 file changed, 64 insertions(+), 0 deletions(-) Approvals: Jenkins Builder: Verified fixeria: Looks good to me, approved pespin: Looks good to me, but someone else must approve diff --git a/jobs/osmocom-obs-sync.yml b/jobs/osmocom-obs-sync.yml new file mode 100644 index 0000000..2dd9888 --- /dev/null +++ b/jobs/osmocom-obs-sync.yml @@ -0,0 +1,54 @@ +--- +- job: + name: 'Osmocom_OBS_sync' + project-type: freestyle + concurrent: false + defaults: global + description: | + Sync base distributions (Debian:12 etc.) from openSUSE's OBS to our OBS + (<a href="https://osmocom.org/issues/6165">OS#6165</a>). + properties: + - build-discarder: + days-to-keep: 30 + num-to-keep: 30 + node: obs + parameters: + - string: + name: CI_BRANCH + description: osmo-ci.git branch + default: 'master' + scm: + - git: + branches: + - '$CI_BRANCH' + url: https://gerrit.osmocom.org/osmo-ci + git-config-name: 'Jenkins Builder' + git-config-email: 'jenkins(a)osmocom.org' + builders: + - shell: | + export PYTHONUNBUFFERED=1 + export DISTROS=" + AlmaLinux:8 + Debian:10 + Debian:11 + Debian:12 + Debian:Next + Debian:Testing + Raspbian:10 + Raspbian:11 + Ubuntu:20.04 + Ubuntu:22.04 + Ubuntu:23.04 + " + ./scripts/obs/sync_obs_projects.py \ + --docker \ + $DISTROS + triggers: + - timed: "H H * * *" + wrappers: + - ansicolor: + colormap: xterm + publishers: + - email: + notify-every-unstable-build: true + recipients: 'jenkins-notifications(a)lists.osmocom.org' -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/34334?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: Ibcf02eb5feee2b314ff28861835040c21aea4294 Gerrit-Change-Number: 34334 Gerrit-PatchSet: 3 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[L] Change in osmo-ci[master]: OBS: add sync_obs_projects.py
by osmith 08 Sep '23

08 Sep '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/34333?usp=email ) Change subject: OBS: add sync_obs_projects.py ...................................................................... OBS: add sync_obs_projects.py Add a new script that syncronizes base distributions from openSUSE's OBS, like Debian:12. This is done by getting "meta" and "prjconf" from the source OBS, adjusting it and writing it to our OBS instance. The motivation for this is making our OBS more resilient, by using these synchronized projects we can build packages even if the openSUSE OBS API is not reachable. Related: OS#6165 Change-Id: I00122f7819ba682cb791c370d781377cfb302629 --- M scripts/obs/README M scripts/obs/data/build_srcpkg.Dockerfile M scripts/obs/lib/config.py M scripts/obs/lib/osc.py A scripts/obs/sync_obs_projects.py 5 files changed, 392 insertions(+), 0 deletions(-) Approvals: fixeria: Looks good to me, approved pespin: Looks good to me, but someone else must approve Jenkins Builder: Verified diff --git a/scripts/obs/README b/scripts/obs/README index f761f1e..cf8e201 100644 --- a/scripts/obs/README +++ b/scripts/obs/README @@ -7,6 +7,7 @@ * build_srcpkg.py: build one source package for an Osmocom project * update_obs_project.py: generate source packages and upload them to OBS * build_binpkg.py: build rpm/deb packages for one Osmocom project +* sync_obs_projects.py: sync projects from another instance (OS#6165) Docker ------ diff --git a/scripts/obs/data/build_srcpkg.Dockerfile b/scripts/obs/data/build_srcpkg.Dockerfile index ebb30cd..75482bc 100644 --- a/scripts/obs/data/build_srcpkg.Dockerfile +++ b/scripts/obs/data/build_srcpkg.Dockerfile @@ -7,6 +7,7 @@ apt-get upgrade -y && \ apt-get install -y --no-install-recommends \ ca-certificates \ + colordiff \ debhelper \ dh-python \ dpkg-dev \ @@ -14,6 +15,7 @@ git \ git-review \ gnupg2 \ + libxml2-utils \ meson \ osc \ python3-setuptools \ diff --git a/scripts/obs/lib/config.py b/scripts/obs/lib/config.py index c3b172d..038122b 100644 --- a/scripts/obs/lib/config.py +++ b/scripts/obs/lib/config.py @@ -117,3 +117,17 @@ "debian:*", "ubuntu:*", ] + +# +# Options related to sync from build.opensuse.org (OS#6165) +# + +sync_remove_paths = [ + # This path has a kernel-obs-build package that other OBS instances use to + # build armv7l/hl packages, but we don't need it + "OBS:DefaultKernel", +] + +sync_set_maintainers = [ + "osmocom-jenkins", +] diff --git a/scripts/obs/lib/osc.py b/scripts/obs/lib/osc.py index 1113032..656783e 100644 --- a/scripts/obs/lib/osc.py +++ b/scripts/obs/lib/osc.py @@ -143,3 +143,39 @@ def delete_package(package, commit_msg): print(f"{package}: removing from OBS ({commit_msg})") run_osc(["rdelete", "-m", commit_msg, proj, os.path.basename(package)]) + + +def get_prjconf(output_file): + print(f"{proj}: getting prjconf") + prjconf = lib.osc.run_osc(["meta", "prjconf", proj]).output + with open(output_file, "w") as h: + h.write(prjconf) + + +def update_prjconf(prjconf_file, commit_msg): + print(f"{proj}: updating prjconf") + lib.osc.run_osc(["meta", + "prjconf", + "-F", prjconf_file, + "-m", commit_msg, + proj]) + + +def get_meta(output_file): + print(f"{proj}: getting meta") + meta = lib.osc.run_osc(["meta", "prj", proj]).output + with open(output_file, "w") as h: + h.write(meta) + + +def update_meta(meta_file, commit_msg): + print(f"{proj}: updating meta") + lib.osc.run_osc(["meta", + "prj", + "-F", meta_file, + "-m", commit_msg, + proj]) + +def get_projects(): + print(f"OBS: getting list of projects") + return lib.osc.run_osc(["ls"]).output.rstrip().split("\n") diff --git a/scripts/obs/sync_obs_projects.py b/scripts/obs/sync_obs_projects.py new file mode 100755 index 0000000..85ceba3 --- /dev/null +++ b/scripts/obs/sync_obs_projects.py @@ -0,0 +1,321 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright 2023 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> +import argparse +import glob +import hashlib +import html +import os +import shlex +import shutil +import xml.etree.ElementTree + +import lib +import lib.docker +import lib.osc + +temp_source_prjconf = f"{lib.config.path_temp}/sync_source_prjconf" +temp_source_meta = f"{lib.config.path_temp}/sync_source_meta" +temp_dest_old_meta = f"{lib.config.path_temp}/sync_dest_old_meta" +temp_dest_old_prjconf = f"{lib.config.path_temp}/sync_dest_old_prjconf" +temp_dest_new_meta = f"{lib.config.path_temp}/sync_dest_new_meta" +temp_dest_new_prjconf = f"{lib.config.path_temp}/sync_dest_new_prjconf" + + +def parse_args(): + parser = argparse.ArgumentParser(description="Sync OBS projects (prjconf," + " meta) from another instance (OS#6165)") + parser.add_argument("-d", "--docker", + help="run in docker to avoid installing required pkgs", + action="store_true") + parser.add_argument("-n", "--no-skip-up-to-date", + dest="skip_up_to_date", action="store_false", + help="always assume projects are outdated") + parser.add_argument("-v", "--verbose", action="store_true", + help="always print shell commands and their output," + " instead of only printing them on error") + parser.add_argument("projects", + help="source OBS project, e.g. Debian:12", + nargs="+") + + advanced = parser.add_argument_group("advanced options") + advanced.add_argument("-A", "--apiurl", help="source OBS API URL" + " (default: https://api.opensuse.org)", + default="https://api.opensuse.org") + advanced.add_argument("-p", "--prefix", default="openSUSE.org-mirror", + help="destination OBS prefix" + " (default: openSUSE.org-mirror)") + advanced.add_argument("-t", "--to-apiurl", help="destination OBS API URL" + " (default: https://obs.osmocom.org)", + default="https://obs.osmocom.org") + advanced.add_argument("-w", "--weburl", default="https://build.opensuse.org", + help="source OBS web URL (default:" + " https://build.opensuse.org)") + + args = parser.parse_args() + lib.set_args(args) + + lib.osc.check_oscrc() + + if args.docker: + lib.docker.run_in_docker_and_exit("sync_obs_projects.py", add_oscrc=True) + + +def check_required_programs(): + required_programs = [ + "colordiff", + "xmllint", + ] + + ok = True + for program in required_programs: + if not shutil.which(program): + print(f"ERROR: missing program: {program}") + ok = False + + if not ok: + print("Either install them or use the -d argument to run in docker") + exit(1) + + +def generate_prjconf_header(project): + """ This header gets prepended to the prjconf, before it gets written to + the destination OBS. This script uses it to determine whether the + project needs to be updated next time it runs. """ + with open(temp_source_prjconf, "rb") as h: + source_prjconf = h.read() + with open(temp_source_meta, "rb") as h: + source_meta = h.read() + + ret = "### This project gets synced from:\n" + ret += f"### {lib.args.weburl}/project/show/{project}\n" + ret += "### \n" + ret += "### Do not modify manually. See OS#6165.\n" + ret += "### \n" + ret += f"### Sync information:\n" + ret += f"### - source meta: {hashlib.md5(source_meta).hexdigest()}\n" + ret += f"### - source prjconf: {hashlib.md5(source_prjconf).hexdigest()}\n" + ret += "\n" + + return ret + + +def is_up_to_date(header, projects, project): + project_new = f"{lib.args.prefix}:{project}" + + if project_new not in projects: + print(f"{project_new}: is outdated (not in destination OBS)") + return False + + lib.osc.get_prjconf(temp_dest_old_prjconf) + with open(temp_dest_old_prjconf, "r") as h: + dest_prjconf = h.read() + + if dest_prjconf.startswith(header): + if not lib.args.skip_up_to_date: + print(f"{project_new}: is up-to-date, but -n is set") + return False + print(f"{project_new}: is up-to-date") + return True + + print(f"{project_new}: is outdated") + return False + + +def get_relevant_arches(project): + if project.startswith("AlmaLinux:"): + return ["x86_64"] + if project.startswith("Raspbian:"): + return ["armv7l"] + + return ["aarch64", + "armv7l", + "i586", + "x86_64"] + + +def rewrite_meta(project): + project_new = f"{lib.args.prefix}:{project}" + print(f"{project}: rewriting meta for {project_new}") + tree = xml.etree.ElementTree.parse(temp_source_meta) + root = tree.getroot() + arches = get_relevant_arches(project) + + # Update <project name="..."> + assert root.get("name") == project + root.set("name", project_new) + + for description in root.findall("description"): + href = f"{lib.args.weburl}/project/show/{project}" + description.text = ("This project gets synced from:" + f" <a href='{html.escape(href)}'>{project}</a>\n" + "Do not modify manually. See OS#6165.\n") + + for repository in root.findall(".repository"): + repo_name = repository.get("name") + print(f" adjusting repository: {repo_name}") + for path in repository.findall(".path"): + # Update <path project="..."> + path_project_old = path.get("project") + path_project_new = f"{lib.args.prefix}:{path_project_old}" + path.set("project", path_project_new) + + # Remove unneeded paths + for path_check in lib.config.sync_remove_paths: + if path_project_old == path_check: + print(f" removing path: {path_project_old}") + repository.remove(path) + break + + # Remove arches we don't build for + for arch in repository.findall(".arch"): + if arch.text not in arches: + print(f" removing arch: {arch.text}") + repository.remove(arch) + for download in repository.findall(".download"): + if download.get("arch") not in arches: + repository.remove(download) + + # Debian: meta configs on build.opensuse.org reference PGP keys with an + # experimental feature that is not yet in the stable version of OBS + # (e.g. <pubkey>debian-archive-12</pubkey>): + # https://github.com/openSUSE/open-build-service/pull/14528 + # Also we don't have such a pubkeydir set up on our OBS server. Assume + # ftp.de.debian.org is a trusted mirror, switch to HTTPS and skip the + # PGP verification by removing the pubkey blocks. + if project.startswith("Debian:"): + for download in repository.findall(".download"): + url = download.get("url") + print(f" changing url to https: {url}") + assert url.startswith("http://ftp.de.debian.org/debian"), \ + f"unexpected mirror URL" + download.set("url", url.replace("http://", "https://")) + for pubkey in download.findall("pubkey"): + download.remove(pubkey) + + # Remove original maintainers + for person in root.findall(".person"): + root.remove(person) + + # Add new maintainers + for userid in lib.config.sync_set_maintainers: + print(f" set maintainer: {userid}") + person = xml.etree.ElementTree.Element("person") + person.set("userid", userid) + person.set("role", "maintainer") + # Insert into same position: after title and description + root.insert(2, person) + + tree.write(temp_dest_new_meta) + + +def rewrite_prjconf(project, header): + project_new = f"{lib.args.prefix}:{project}" + print(f"{project}: rewriting prjconf for {project_new}") + + prjconf = "" + with open(temp_source_prjconf, "r") as f: + for line in f: + line = line.rstrip() + + # Remove unneeded dependencies + if line == "VMInstall: kernel-obs-build": + print(f" commenting out: {line}") + line = f"# (commented out by sync) {line}" + + prjconf += f"{line}\n" + + # Extend the AlmaLinux prjconf to also set CentOS variables, as some of + # our prjconfigs and spec files rely on them + if project == "AlmaLinux:8": + print(f" appending CentOS 8 variables") + prjconf += "\n" + prjconf += "# CentOS 8 compat added by sync script\n" + prjconf += "%define centos_version 800\n" + prjconf += "%define centos_ver 8\n" + prjconf += "%define centos 8\n" + prjconf += "Macros:\n" + prjconf += "%centos_version 800\n" + prjconf += "%centos_ver 8\n" + prjconf += "%centos 8\n" + prjconf += ":Macros\n" + + with open(temp_dest_new_prjconf, "w") as f: + f.write(header) + f.write(prjconf) + + +def show_diff(projects, project): + project_new = f"{lib.args.prefix}:{project}" + if project_new not in projects: + return + + # Show prjconf diff (old prjconf was retrieved in is_up_to_date()) + diff = lib.run_cmd(["colordiff", + "-c3", + temp_dest_old_prjconf, + temp_dest_new_prjconf], + check=False) + if diff.returncode: + print(f"{project_new}: prjconf changes:") + print(diff.output, end="") + else: + print(f"{project_new}: prjconf is unchanged") + + # Show meta diff + lib.osc.get_meta(temp_dest_old_meta) + for file in [temp_dest_old_meta, temp_dest_new_meta]: + lib.run_cmd(f"xmllint --format {shlex.quote(file)} > {shlex.quote(file)}.pretty", + shell=True) + diff = lib.run_cmd(["colordiff", + "-c3", + f"{temp_dest_old_meta}.pretty", + f"{temp_dest_new_meta}.pretty"], + check=False) + if diff.returncode: + print(f"{project_new}: meta changes:") + print(diff.output, end="") + else: + print(f"{project_new}: meta is unchanged") + + +def main(): + parse_args() + check_required_programs() + + os.makedirs(lib.config.path_temp, exist_ok=True) + + # Get destination OBS projects + lib.osc.set_apiurl(lib.args.to_apiurl, None) + dest_projects = lib.osc.get_projects() + + for project in lib.args.projects: + # Talk to source OBS + lib.osc.set_apiurl(lib.args.apiurl, project) + + # Get source prjconf, meta + lib.osc.get_prjconf(temp_source_prjconf) + lib.osc.get_meta(temp_source_meta) + + # Talk to dest OBS + project_new = f"{lib.args.prefix}:{project}" + lib.osc.set_apiurl(lib.args.to_apiurl, project_new) + + # Check if dest is up-to-date + header = generate_prjconf_header(project) + if is_up_to_date(header, dest_projects, project): + continue + + # Rewrite configs and show diff + rewrite_prjconf(project, header) + rewrite_meta(project) + show_diff(dest_projects, project) + + # Update dest prjconf & meta + commit_msg = f"sync with {lib.args.weburl}/project/show/{project}" + lib.osc.update_meta(temp_dest_new_meta, commit_msg) + lib.osc.update_prjconf(temp_dest_new_prjconf, commit_msg) + + +if __name__ == "__main__": + main() -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/34333?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: I00122f7819ba682cb791c370d781377cfb302629 Gerrit-Change-Number: 34333 Gerrit-PatchSet: 3 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-ci[master]: OBS: lib/osc: add private proj variable
by osmith 08 Sep '23

08 Sep '23

osmith has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/34332?usp=email ) Change subject: OBS: lib/osc: add private proj variable ...................................................................... Patch Set 2: (1 comment) File scripts/obs/lib/osc.py: https://gerrit.osmocom.org/c/osmo-ci/+/34332/comment/dc70c817_7652485a PS2, Line 35: exit > btw, where this `exit()` is coming from? shouldn't it be `sys. […] ack, sent a follow-up patch to fix it: https://gerrit.osmocom.org/c/osmo-ci/+/34343/1 -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/34332?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: Iabb871bcb432f2fbdaec9cbcab3d663ccf969901 Gerrit-Change-Number: 34332 Gerrit-PatchSet: 2 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-Comment-Date: Fri, 08 Sep 2023 09:06:45 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-MessageType: comment

1 0

[S] Change in osmo-ci[master]: OBS: build_binpkg: support debian:testing
by osmith 08 Sep '23

08 Sep '23

osmith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ci/+/34342?usp=email ) Change subject: OBS: build_binpkg: support debian:testing ...................................................................... OBS: build_binpkg: support debian:testing Change-Id: I71de94ef34c3ccc3610643bd8861b78cccd266a5 --- M scripts/obs/data/build_binpkg.Dockerfile 1 file changed, 11 insertions(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/42/34342/1 diff --git a/scripts/obs/data/build_binpkg.Dockerfile b/scripts/obs/data/build_binpkg.Dockerfile index 49cdc96..4d4b185 100644 --- a/scripts/obs/data/build_binpkg.Dockerfile +++ b/scripts/obs/data/build_binpkg.Dockerfile @@ -53,8 +53,9 @@ # Add master repository, where packages immediately get updated after merging # patches to master. +# sed: first letter uppercase (testing -> Testing) RUN set -x; \ - VERSION="$(echo "$DISTRO" | cut -d : -f 2)"; \ + VERSION="$(echo "$DISTRO" | cut -d : -f 2 | sed 's/./\u&/')"; \ case "$DISTRO" in \ debian:*) \ apt-key add /tmp/Release.key && \ -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/34342?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: I71de94ef34c3ccc3610643bd8861b78cccd266a5 Gerrit-Change-Number: 34342 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[M] Change in osmo-ci[master]: OBS: exit() -> sys.exit()
by osmith 08 Sep '23

08 Sep '23

osmith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ci/+/34343?usp=email ) Change subject: OBS: exit() -> sys.exit() ...................................................................... OBS: exit() -> sys.exit() As pointed out in code review, sys.exit() should be used instead of exit() from site. https://docs.python.org/3/library/constants.html#exit says: > The site module (which is imported automatically during startup, > except if the -S command-line option is given) adds several > constants to the built-in namespace. They are useful for the > interactive interpreter shell and should not be used in programs. Change-Id: Ia9d355e660afd1ad3c60064900d8be66e8e1d385 --- M scripts/obs/build_binpkg.py M scripts/obs/build_srcpkg.py M scripts/obs/lib/__init__.py M scripts/obs/lib/docker.py M scripts/obs/lib/osc.py M scripts/obs/lib/srcpkg.py M scripts/obs/sync_obs_projects.py M scripts/obs/update_obs_project.py 8 files changed, 39 insertions(+), 15 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/43/34343/1 diff --git a/scripts/obs/build_binpkg.py b/scripts/obs/build_binpkg.py index 67a4bec..f3b4bcf 100755 --- a/scripts/obs/build_binpkg.py +++ b/scripts/obs/build_binpkg.py @@ -3,9 +3,10 @@ # Copyright 2022 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> import argparse import fnmatch +import lib import multiprocessing import os -import lib +import sys import lib.binpkg_deb import lib.config import lib.docker @@ -57,7 +58,7 @@ if not os.path.exists(srcdir): print(f"ERROR: {args.package}: no srcpkg found, run build_srcpkg.py" " first!") - exit(1) + sys.exit(1) bindir = f"{lib.config.path_temp}/binpkgs" lib.run_cmd(["rm", "-rf", bindir]) diff --git a/scripts/obs/build_srcpkg.py b/scripts/obs/build_srcpkg.py index 0ec1958..a5feeea 100755 --- a/scripts/obs/build_srcpkg.py +++ b/scripts/obs/build_srcpkg.py @@ -2,6 +2,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later # Copyright 2022 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> import argparse +import sys import lib import lib.config import lib.docker @@ -26,7 +27,7 @@ if not args.meta and not args.package: print("ERROR: specify -m and/or a package. See -h for help.") - exit(1) + sys.exit(1) lib.set_args(args) diff --git a/scripts/obs/lib/__init__.py b/scripts/obs/lib/__init__.py index 842dd41..1f686e5 100644 --- a/scripts/obs/lib/__init__.py +++ b/scripts/obs/lib/__init__.py @@ -98,7 +98,7 @@ if not ok: print("Either install them or use the -d argument to run in docker") - exit(1) + sys.exit(1) def set_proper_package_name(package): @@ -117,7 +117,7 @@ print(f"ERROR: unknown package: {package}") print("See packages_osmocom and packages_other in obs/lib/config.py") - exit(1) + sys.exit(1) def exit_error_cmd(completed, error_msg): diff --git a/scripts/obs/lib/docker.py b/scripts/obs/lib/docker.py index 4f9f981..779099b 100644 --- a/scripts/obs/lib/docker.py +++ b/scripts/obs/lib/docker.py @@ -55,7 +55,7 @@ return os.path.realpath(ret) print("ERROR: couldn't find ~/.oscrc. Put it there or set OSCRC.") - exit(1) + sys.exit(1) def run_in_docker_and_exit(script_path, add_oscrc=False, @@ -75,7 +75,7 @@ if not shutil.which("docker"): print("ERROR: docker is not installed") - exit(1) + sys.exit(1) if not distro: distro = lib.config.docker_distro_default @@ -116,4 +116,4 @@ print(f"docker: running: {script_path} inside docker") ret = subprocess.run(cmd) - exit(ret.returncode) + sys.exit(ret.returncode) diff --git a/scripts/obs/lib/osc.py b/scripts/obs/lib/osc.py index 656783e..20a63b5 100644 --- a/scripts/obs/lib/osc.py +++ b/scripts/obs/lib/osc.py @@ -6,6 +6,7 @@ import os import shlex import shutil +import sys import lib import lib.config @@ -22,7 +23,7 @@ print("ERROR: oscrc does not exist, please create one as explained in the" " README.") - exit(1) + sys.exit(1) def set_apiurl(url, obs_proj=None): @@ -32,7 +33,7 @@ if obs_proj is not None: if ":" not in obs_proj: print(f"ERROR: this doesn't look like a valid OBS project: {obs_proj}") - exit(1) + sys.exit(1) proj = obs_proj apiurl = url diff --git a/scripts/obs/lib/srcpkg.py b/scripts/obs/lib/srcpkg.py index b97bbf8..13404a7 100644 --- a/scripts/obs/lib/srcpkg.py +++ b/scripts/obs/lib/srcpkg.py @@ -4,6 +4,7 @@ import glob import os import pathlib +import sys import lib.config import lib.debian import lib.rpm_spec @@ -42,7 +43,7 @@ print(f"ERROR: {project}.git doesn't have a git-version-gen script and" " couldn't find libosmocore.git's copy of the script here either: " + ret) - exit(1) + sys.exit(1) def get_git_version(project): diff --git a/scripts/obs/sync_obs_projects.py b/scripts/obs/sync_obs_projects.py index 85ceba3..cc4635a 100755 --- a/scripts/obs/sync_obs_projects.py +++ b/scripts/obs/sync_obs_projects.py @@ -8,6 +8,7 @@ import os import shlex import shutil +import sys import xml.etree.ElementTree import lib @@ -75,7 +76,7 @@ if not ok: print("Either install them or use the -d argument to run in docker") - exit(1) + sys.exit(1) def generate_prjconf_header(project): diff --git a/scripts/obs/update_obs_project.py b/scripts/obs/update_obs_project.py index eee58a7..a456218 100755 --- a/scripts/obs/update_obs_project.py +++ b/scripts/obs/update_obs_project.py @@ -3,6 +3,7 @@ # Copyright 2022 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> import argparse import os +import sys import traceback import lib import lib.config @@ -185,7 +186,7 @@ print(f"Deleted: {len(srcpkgs_deleted)}") if not srcpkgs_failed_build and not srcpkgs_failed_upload: - exit(0) + sys.exit(0) print() print("List of failed packages:") @@ -194,7 +195,7 @@ for package in srcpkgs_failed_upload: print(f"* {package} (srcpkg upload)") - exit(1) + sys.exit(1) def validate_args(args): @@ -202,7 +203,7 @@ # git repository before trying to update/delete a package from OBS if args.delete and args.feed != "master": print("ERROR: --delete can only be used with --feed=master") - exit(1) + sys.exit(1) def main(): -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/34343?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: Ia9d355e660afd1ad3c60064900d8be66e8e1d385 Gerrit-Change-Number: 34343 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[S] Change in osmo-bts[master]: rsl: Improve logic reactivating CCCH upon SI3 BS_AG_BLKS_RES change
by osmith 08 Sep '23

08 Sep '23

Attention is currently required from: pespin. osmith has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-bts/+/34341?usp=email ) Change subject: rsl: Improve logic reactivating CCCH upon SI3 BS_AG_BLKS_RES change ...................................................................... Patch Set 1: Code-Review+1 -- To view, visit https://gerrit.osmocom.org/c/osmo-bts/+/34341?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-bts Gerrit-Branch: master Gerrit-Change-Id: Ibc118e11c64f04de55cb7b94d8bf2c84b431774d Gerrit-Change-Number: 34341 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Comment-Date: Fri, 08 Sep 2023 08:38:06 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

[S] Change in osmo-bts[master]: bts-trx: Fix CCCH not enabled if BS_AG_BLKS_RES!=1 is provided by BSC
by osmith 08 Sep '23

08 Sep '23

Attention is currently required from: fixeria, laforge, pespin. osmith has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-bts/+/34340?usp=email ) Change subject: bts-trx: Fix CCCH not enabled if BS_AG_BLKS_RES!=1 is provided by BSC ...................................................................... Patch Set 1: Code-Review+1 -- To view, visit https://gerrit.osmocom.org/c/osmo-bts/+/34340?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-bts Gerrit-Branch: master Gerrit-Change-Id: I61e1681fbaa2c993b529d58b581c99166b62bda3 Gerrit-Change-Number: 34340 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Comment-Date: Fri, 08 Sep 2023 08:34:37 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

2025

2024

2023

2022

gerrit-log September 2023