- gerrit-log - lists.osmocom.org

Change in osmo-ttcn3-hacks[master]: bts: Log unexpected RSL message
by pespin 06 Oct '22

06 Oct '22

Attention is currently required from: laforge. pespin has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/29582 ) Change subject: bts: Log unexpected RSL message ...................................................................... Patch Set 1: (1 comment) File bts/BTS_Tests.ttcn: https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/29582/comment/79c5bb9f_cbdd… PS1, Line 783: Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, "Unexpected RSL message received"); > this looks a bit odd with the log statement below, one might rename the lower message to "Unexpected […] well it's still RSL related since it's coming from the RSL port, so I think writing what yousay is even more confusing. -- To view, visit https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/29582 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-Change-Id: I6cdc53ff038858211b09c0a8eeec19603107ffe4 Gerrit-Change-Number: 29582 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Comment-Date: Thu, 06 Oct 2022 07:59:37 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: laforge <laforge(a)osmocom.org> Gerrit-MessageType: comment

1 0

Change in osmo-mgw[master]: osmux: Use setsockopt(IPV6_V6ONLY) on IPv6 socket
by pespin 06 Oct '22

06 Oct '22

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-mgw/+/29638 ) Change subject: osmux: Use setsockopt(IPV6_V6ONLY) on IPv6 socket ...................................................................... osmux: Use setsockopt(IPV6_V6ONLY) on IPv6 socket We for sure don't want to mix stuff and send IPv4 stuff over IPv6, so let's make sure we block that possibility. This helps in avoiding configuration problems where same port allocated by an IPv6 address cannot be used by an IPv4 one. Change-Id: I0b120d016ff32369a8c077b601ec168928c38f0b --- M src/libosmo-mgcp/mgcp_osmux.c 1 file changed, 8 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-mgw refs/changes/38/29638/1 diff --git a/src/libosmo-mgcp/mgcp_osmux.c b/src/libosmo-mgcp/mgcp_osmux.c index 3be6d78..889ca76 100644 --- a/src/libosmo-mgcp/mgcp_osmux.c +++ b/src/libosmo-mgcp/mgcp_osmux.c @@ -470,6 +470,7 @@ osmo_sock_get_name2(osmux_fd_v4.fd)); } if (cfg->osmux_addr_v6) { + int one = 1; ret = mgcp_create_bind(cfg->osmux_addr_v6, &osmux_fd_v6, cfg->osmux_port, cfg->endp_dscp, cfg->endp_priority); if (ret < 0) { @@ -478,6 +479,13 @@ return ret; } + ret = setsockopt(osmux_fd_v6.fd, SOL_IPV6, IPV6_V6ONLY, &one, sizeof(one)); + if (ret < 0) { + LOGP(DOSMUX, LOGL_ERROR, "setsockopt(IPV6_V6ONLY, 1) failed on socket %s:%u\n", + cfg->osmux_addr_v6, cfg->osmux_port); + return ret; + } + ret = osmo_fd_register(&osmux_fd_v6); if (ret < 0) { LOGP(DOSMUX, LOGL_ERROR, "Cannot register OSMUX IPv6 socket %s\n", -- To view, visit https://gerrit.osmocom.org/c/osmo-mgw/+/29638 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-mgw Gerrit-Branch: master Gerrit-Change-Id: I0b120d016ff32369a8c077b601ec168928c38f0b Gerrit-Change-Number: 29638 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

Change in libosmocore[master]: msgb: assert msgb->lXh to be not NULL
by laforge 06 Oct '22

06 Oct '22

laforge has submitted this change. ( https://gerrit.osmocom.org/c/libosmocore/+/29591 ) Change subject: msgb: assert msgb->lXh to be not NULL ...................................................................... msgb: assert msgb->lXh to be not NULL When any of l1h, l2h, l2h or l4h is set to NULL (which is the default for newly allocated message buffers). Then the msgb_lXhlen() functions will return the address value of msgb->tail. This can lead to unexpected results at a later point. We should have an OSMO_ASSERT to catch the problem early. Change-Id: I1795c559f190713ebbabfbabf3453ab77da46a49 Related: OS#5645 --- M include/osmocom/core/msgb.h 1 file changed, 4 insertions(+), 0 deletions(-) Approvals: laforge: Looks good to me, approved fixeria: Looks good to me, approved pespin: Looks good to me, but someone else must approve Jenkins Builder: Verified diff --git a/include/osmocom/core/msgb.h b/include/osmocom/core/msgb.h index 117fcb0..fbf1742 100644 --- a/include/osmocom/core/msgb.h +++ b/include/osmocom/core/msgb.h @@ -144,6 +144,7 @@ */ static inline unsigned int msgb_l1len(const struct msgb *msgb) { + OSMO_ASSERT(msgb->l1h); return msgb->tail - (uint8_t *)msgb_l1(msgb); } @@ -156,6 +157,7 @@ */ static inline unsigned int msgb_l2len(const struct msgb *msgb) { + OSMO_ASSERT(msgb->l2h); return msgb->tail - (uint8_t *)msgb_l2(msgb); } @@ -168,6 +170,7 @@ */ static inline unsigned int msgb_l3len(const struct msgb *msgb) { + OSMO_ASSERT(msgb->l3h); return msgb->tail - (uint8_t *)msgb_l3(msgb); } @@ -180,6 +183,7 @@ */ static inline unsigned int msgb_l4len(const struct msgb *msgb) { + OSMO_ASSERT(msgb->l4h); return msgb->tail - (uint8_t *)msgb_sms(msgb); } -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/29591 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: I1795c559f190713ebbabfbabf3453ab77da46a49 Gerrit-Change-Number: 29591 Gerrit-PatchSet: 3 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in libosmocore[master]: msgb: do not use msgb_l4 instead of msgb_sms
by laforge 06 Oct '22

06 Oct '22

laforge has submitted this change. ( https://gerrit.osmocom.org/c/libosmocore/+/29620 ) Change subject: msgb: do not use msgb_l4 instead of msgb_sms ...................................................................... msgb: do not use msgb_l4 instead of msgb_sms The macro msgb_sms() is basically an alias for msgb_l4(). Lets use msgb_l4() in msgb_l4len() instead of msgb_sms(). Change-Id: I90d4f5c07fbaadd9e022752a2c64c4855f0b4227 --- M include/osmocom/core/msgb.h 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: laforge: Looks good to me, approved fixeria: Looks good to me, but someone else must approve pespin: Looks good to me, approved Jenkins Builder: Verified diff --git a/include/osmocom/core/msgb.h b/include/osmocom/core/msgb.h index fbf1742..1e2b023 100644 --- a/include/osmocom/core/msgb.h +++ b/include/osmocom/core/msgb.h @@ -184,7 +184,7 @@ static inline unsigned int msgb_l4len(const struct msgb *msgb) { OSMO_ASSERT(msgb->l4h); - return msgb->tail - (uint8_t *)msgb_sms(msgb); + return msgb->tail - (uint8_t *)msgb_l4(msgb); } /*! determine the length of the header -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/29620 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: I90d4f5c07fbaadd9e022752a2c64c4855f0b4227 Gerrit-Change-Number: 29620 Gerrit-PatchSet: 2 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in libosmocore[master]: msgb: do not use msgb_l4 instead of msgb_sms
by laforge 06 Oct '22

06 Oct '22

Attention is currently required from: dexter. laforge has posted comments on this change. ( https://gerrit.osmocom.org/c/libosmocore/+/29620 ) Change subject: msgb: do not use msgb_l4 instead of msgb_sms ...................................................................... Patch Set 1: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/29620 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: I90d4f5c07fbaadd9e022752a2c64c4855f0b4227 Gerrit-Change-Number: 29620 Gerrit-PatchSet: 1 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-Attention: dexter <pmaier(a)sysmocom.de> Gerrit-Comment-Date: Thu, 06 Oct 2022 07:52:06 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

Change in libosmocore[master]: msgb: assert msgb->lXh to be not NULL
by laforge 06 Oct '22

06 Oct '22

Attention is currently required from: dexter. laforge has posted comments on this change. ( https://gerrit.osmocom.org/c/libosmocore/+/29591 ) Change subject: msgb: assert msgb->lXh to be not NULL ...................................................................... Patch Set 2: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/29591 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: I1795c559f190713ebbabfbabf3453ab77da46a49 Gerrit-Change-Number: 29591 Gerrit-PatchSet: 2 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-Attention: dexter <pmaier(a)sysmocom.de> Gerrit-Comment-Date: Thu, 06 Oct 2022 07:51:49 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

Change in osmo-ttcn3-hacks[master]: bts: Log unexpected RSL message
by laforge 06 Oct '22

06 Oct '22

Attention is currently required from: pespin. laforge has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/29582 ) Change subject: bts: Log unexpected RSL message ...................................................................... Patch Set 1: Code-Review+1 (1 comment) File bts/BTS_Tests.ttcn: https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/29582/comment/b80c6077_c9d4… PS1, Line 783: Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, "Unexpected RSL message received"); this looks a bit odd with the log statement below, one might rename the lower message to "Unexpected non-RSL message received on RSL". -- To view, visit https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/29582 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-Change-Id: I6cdc53ff038858211b09c0a8eeec19603107ffe4 Gerrit-Change-Number: 29582 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Comment-Date: Thu, 06 Oct 2022 07:51:29 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

Change in osmo-ci[master]: obs: build_binpkg: optimize osmo-gsm-manuals-dev
by osmith 06 Oct '22

06 Oct '22

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/29603 ) Change subject: obs: build_binpkg: optimize osmo-gsm-manuals-dev ...................................................................... obs: build_binpkg: optimize osmo-gsm-manuals-dev Installing osmo-gsm-manuals-dev plus depends takes a long time. Don't do this for every build, instead do it once when building a second docker container and then use that. Related: OS#2385 Change-Id: I8475bd954352b572197795ad4cd9461e39896d48 --- M scripts/obs/build_binpkg.py A scripts/obs/data/build_binpkg_manuals.Dockerfile M scripts/obs/lib/docker.py M scripts/obs/lib/srcpkg.py 4 files changed, 57 insertions(+), 4 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved pespin: Looks good to me, but someone else must approve diff --git a/scripts/obs/build_binpkg.py b/scripts/obs/build_binpkg.py index b670af9..27817e2 100755 --- a/scripts/obs/build_binpkg.py +++ b/scripts/obs/build_binpkg.py @@ -59,9 +59,17 @@ script_path = "data/build_rpm.sh" if args.docker: + image_type = "build_binpkg" + + # Optimization: use docker container with osmo-gsm-manuals-dev already + # installed if it is in build depends + if distro.startswith("debian:") \ + and lib.srcpkg.requires_osmo_gsm_manuals_dev(args.package): + image_type += "_manuals" + env["BUILDUSER"] = "user" lib.docker.run_in_docker_and_exit(script_path, - image_type="build_binpkg", + image_type=image_type, distro=distro, pass_argv=False, env=env) else: diff --git a/scripts/obs/data/build_binpkg_manuals.Dockerfile b/scripts/obs/data/build_binpkg_manuals.Dockerfile new file mode 100644 index 0000000..fd4c709 --- /dev/null +++ b/scripts/obs/data/build_binpkg_manuals.Dockerfile @@ -0,0 +1,20 @@ +# Optimization: installing osmo-gsm-manuals-dev and its many, many dependencies +# takes quite a long time - sometimes longer than building the package itself +# (related: OS#4132). Instead of doing this every time before starting a build, +# here is a second docker container that already has it installed. This gets +# used by build_binpkg.py in case the package to build depends on +# osmo-gsm-manuals-dev and the build is done for Debian. Note that right now we +# don't build the manuals for rpm-based distributions. +ARG DISTRO_FROM +FROM ${DISTRO_FROM} +ARG DISTRO + +RUN case "$DISTRO" in \ + debian*) \ + apt-get update && \ + apt-get install -y --no-install-recommends \ + osmo-gsm-manuals-dev \ + && \ + apt-get clean \ + ;; \ + esac diff --git a/scripts/obs/lib/docker.py b/scripts/obs/lib/docker.py index 81464bb..136aeeb 100644 --- a/scripts/obs/lib/docker.py +++ b/scripts/obs/lib/docker.py @@ -14,7 +14,11 @@ return ret -def get_distro_from(distro): +def get_distro_from(distro, image_type): + # Manuals: depend on regular image (data/build_binpkg_manuals.Dockerfile) + if image_type.endswith("_manuals"): + return get_image_name(distro, image_type.replace("_manuals", "")) + # CentOS 8 is EOL (SYS#5818) if distro == "centos:8": return "almalinux:8" @@ -24,7 +28,7 @@ def build_image(distro, image_type): image_name = get_image_name(distro, image_type) - distro_from = get_distro_from(distro) + distro_from = get_distro_from(distro, image_type) print(f"docker: building image {image_name}") @@ -75,10 +79,15 @@ if add_oscrc: oscrc = get_oscrc() - # Build the docker image. Unless it is up-to-date, this will take a few + # Unless the docker image is up-to-date, building will take a few # minutes or so, therefore print the output. No need to restore # set_cmds_verbose, as we use subprocess.run() below and exit afterwards. lib.set_cmds_verbose(True) + + # Manuals: build regular image first (data/build_binpkg_manuals.Dockerfile) + if image_type.endswith("_manuals"): + build_image(distro, image_type.replace("_manuals","")) + build_image(distro, image_type) cmd = ["docker", "run", diff --git a/scripts/obs/lib/srcpkg.py b/scripts/obs/lib/srcpkg.py index 1711f70..b37b8b3 100644 --- a/scripts/obs/lib/srcpkg.py +++ b/scripts/obs/lib/srcpkg.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: GPL-2.0-or-later # Copyright 2022 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> +import glob import os import pathlib import lib.config @@ -173,3 +174,18 @@ lib.remove_cache_extra_files() return version_epoch + + +def requires_osmo_gsm_manuals_dev(project): + """ Check if an already built source package has osmo-gsm-manuals-dev in + Build-Depends of the .dsc file """ + path_dsc = glob.glob(f"{lib.get_output_path(project)}/*.dsc") + assert len(path_dsc) == 1, f"failed to get dsc path for {project}" + + with open(path_dsc[0], "r") as handle: + for line in handle.readlines(): + if line.startswith("Build-Depends:") \ + and "osmo-gsm-manuals-dev" in line: + return True + + return False -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/29603 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: I8475bd954352b572197795ad4cd9461e39896d48 Gerrit-Change-Number: 29603 Gerrit-PatchSet: 2 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in osmo-ci[master]: obs: add build_binpkg.py
by osmith 06 Oct '22

06 Oct '22

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/29601 ) Change subject: obs: add build_binpkg.py ...................................................................... obs: add build_binpkg.py Add a script to build deb/rpm packages, as it would be done on obs.osmocom.org. This will be used by jenkins to verify deb and rpm builds for each submitted gerrit patch. I have attempted to use 'osc build' instead of directly calling apt-get build-dep and dpkg-buildpackage (and rpm equivalents). Using 'osc build' would have the advantage that the build works as close to the OBS build as possible. However it would try to install dependencies with sudo, so we would need to have sudo available in the docker container that builds the untrusted code from gerrit. Let's not do that. Related: OS#2385 Change-Id: I4c6b5d61af35df98cbc70d9ddc8ad36d38a9ce18 --- A scripts/obs/build_binpkg.py A scripts/obs/data/build_binpkg.Dockerfile A scripts/obs/data/build_deb.sh A scripts/obs/data/build_rpm.sh M scripts/obs/lib/__init__.py A scripts/obs/lib/binpkg_deb.py A scripts/obs/lib/binpkg_rpm.py M scripts/obs/lib/config.py M scripts/obs/lib/docker.py 9 files changed, 275 insertions(+), 3 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved pespin: Looks good to me, but someone else must approve diff --git a/scripts/obs/build_binpkg.py b/scripts/obs/build_binpkg.py new file mode 100755 index 0000000..b670af9 --- /dev/null +++ b/scripts/obs/build_binpkg.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright 2022 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> +import argparse +import multiprocessing +import os +import lib +import lib.binpkg_deb +import lib.config +import lib.docker +import lib.git +import lib.metapkg +import lib.srcpkg + + +def main(): + distro_default = lib.config.docker_distro_default + distro_choices = [distro_default] + lib.config.docker_distro_other + jobs_default = multiprocessing.cpu_count() + 1 + + parser = argparse.ArgumentParser( + description="Build a deb or rpm package as it would be done on" + " obs.osmocom.org. Use after building a source package" + " with build_srcpkg.py." + f" Output dir: {lib.config.path_temp}/binpkgs") + parser.add_argument("-d", "--docker", choices=distro_choices, + const=distro_default, nargs="?", + help="build the package in docker for a specific" + f" distro (default: {distro_default})") + parser.add_argument("-j", "--jobs", type=int, default=jobs_default, + help=f"parallel running jobs (default: {jobs_default})") + parser.add_argument("-v", "--verbose", action="store_true", + help="always print shell commands and their output," + " instead of only printing them on error") + parser.add_argument("package", + help="package name, e.g. libosmocore") + args = parser.parse_args() + + lib.set_cmds_verbose(args.verbose) + + srcdir = f"{lib.config.path_temp}/srcpkgs/{args.package}" + if not os.path.exists(srcdir): + print(f"ERROR: {args.package}: no srcpkg found, run build_srcpkg.py" + " first!") + exit(1) + + bindir = f"{lib.config.path_temp}/binpkgs" + lib.run_cmd(["rm", "-rf", bindir]) + os.makedirs(bindir) + + distro = args.docker if args.docker else distro_default + + env = {"JOBS": str(args.jobs), + "PACKAGE": args.package, + "BUILDUSER": os.environ["USER"]} + + script_path = "data/build_deb.sh" + if not distro.startswith("debian:"): + script_path = "data/build_rpm.sh" + + if args.docker: + env["BUILDUSER"] = "user" + lib.docker.run_in_docker_and_exit(script_path, + image_type="build_binpkg", + distro=distro, + pass_argv=False, env=env) + else: + lib.run_cmd(["sudo", "-E", script_path], env=env, + cwd=lib.config.path_top) + +if __name__ == "__main__": + main() diff --git a/scripts/obs/data/build_binpkg.Dockerfile b/scripts/obs/data/build_binpkg.Dockerfile new file mode 100644 index 0000000..feacbc9 --- /dev/null +++ b/scripts/obs/data/build_binpkg.Dockerfile @@ -0,0 +1,72 @@ +ARG DISTRO_FROM +FROM ${DISTRO_FROM} +ARG DISTRO +ARG UID + +COPY Release.key /tmp/Release.key + +RUN useradd --uid=${UID} -m user + +# Only install build-essential here, and what's needed to add the Osmocom +# repository. Everything else must be defined as dependency in the package +# build recipe. For rpm-based distributions, there is no build-essential or +# similar package. Instead add relevant packages from prjconf, e.g.: +# https://build.opensuse.org/projects/CentOS:CentOS-8/prjconf +RUN case "$DISTRO" in \ + debian*) \ + apt-get update && \ + apt-get install -y --no-install-recommends \ + build-essential \ + ca-certificates \ + fakeroot \ + git \ + gnupg2 \ + && \ + apt-get clean \ + ;; \ + centos*) \ + dnf -y install \ + autoconf \ + automake \ + binutils \ + dnf-utils \ + gcc \ + gcc-c++ \ + glibc-devel \ + libtool \ + make \ + redhat-rpm-config \ + rpm-build \ + rpmdevtools \ + wget && \ + yum config-manager --set-enabled powertools && \ + su user -c rpmdev-setuptree \ + ;; \ + esac + +# Add master repository, where packages immediately get updated after merging +# patches to master. +RUN case "$DISTRO" in \ + debian:11) \ + apt-key add /tmp/Release.key && \ + rm /tmp/Release.key && \ + echo "deb https://downloads.osmocom.org/packages/osmocom:/master/Debian_11/ ./" \ + > /etc/apt/sources.list.d/osmocom-master.list \ + ;; \ + centos:8) \ + { echo "[network_osmocom_master]"; \ + echo "name=Nightly packages of the Osmocom project (CentOS_8)"; \ + echo "type=rpm-md"; \ + echo "baseurl=https://downloads.osmocom.org/packages/osmocom:/master/CentOS_8/"; \ + echo "gpgcheck=1"; \ + echo "gpgkey=https://downloads.osmocom.org/packages/osmocom:/master/CentOS_8/repo…"; \ + echo "enabled=1"; \ + } > /etc/yum.repos.d/network:osmocom:master.repo \ + ;; \ + *) \ + echo "can't install repo for $DISTRO" && \ + exit 1 \ + ;; \ + esac + +WORKDIR /obs/ diff --git a/scripts/obs/data/build_deb.sh b/scripts/obs/data/build_deb.sh new file mode 100755 index 0000000..0030278 --- /dev/null +++ b/scripts/obs/data/build_deb.sh @@ -0,0 +1,14 @@ +#!/bin/sh -ex + +apt_get="apt-get" +if [ -n "$INSIDE_DOCKER" ]; then + export DEBIAN_FRONTEND=noninteractive + apt_get="apt-get -y" +fi + +su "$BUILDUSER" -c "tar -C _temp/binpkgs -xvf _temp/srcpkgs/$PACKAGE/*.tar.*" +cd _temp/binpkgs/* + +$apt_get update +$apt_get build-dep . +su "$BUILDUSER" -c "dpkg-buildpackage -us -uc -j$JOBS" diff --git a/scripts/obs/data/build_rpm.sh b/scripts/obs/data/build_rpm.sh new file mode 100755 index 0000000..39f49ee --- /dev/null +++ b/scripts/obs/data/build_rpm.sh @@ -0,0 +1,30 @@ +#!/bin/sh -ex + +if ! [ -d /home/$BUILDUSER/rpmbuild/SOURCES ]; then + set +x + echo "ERROR: rpmdev-setuptree did not run" + echo "If this is an rpm based system and you want to build the package" + echo "here, run rpmdev-setuptree. Otherwise consider building the" + echo "package in docker (-d)." + exit 1 +fi + +yum_builddep="yum-builddep" +if [ -n "$INSIDE_DOCKER" ]; then + yum_builddep="yum-builddep -y" +fi + +spec="$(basename "$(find _temp/srcpkgs/"$PACKAGE" -name '*.spec')")" + +su "$BUILDUSER" -c "cp _temp/srcpkgs/$PACKAGE/$spec ~/rpmbuild/SPECS" +su "$BUILDUSER" -c "cp _temp/srcpkgs/$PACKAGE/*.tar.* ~/rpmbuild/SOURCES" +su "$BUILDUSER" -c "cp _temp/srcpkgs/$PACKAGE/rpmlintrc ~/rpmbuild/SOURCES" + +$yum_builddep "/home/$BUILDUSER/rpmbuild/SPECS/$spec" + +su "$BUILDUSER" -c "rpmbuild -bb ~/rpmbuild/SPECS/$spec" + +# Make built rpms available outside of docker +if [ -n "$INSIDE_DOCKER" ]; then + su "$BUILDUSER" -c "mv ~/rpmbuild/RPMS/*/*.rpm _temp/binpkgs/" +fi diff --git a/scripts/obs/lib/__init__.py b/scripts/obs/lib/__init__.py index 8f52dba..5979623 100644 --- a/scripts/obs/lib/__init__.py +++ b/scripts/obs/lib/__init__.py @@ -14,6 +14,7 @@ def add_shared_arguments(parser): + """ Arguments shared between build_srcpkg.py and update_obs_project.py. """ parser.add_argument("-f", "--feed", help="package feed (default: nightly). The feed" " determines the git revision to be built:" diff --git a/scripts/obs/lib/binpkg_deb.py b/scripts/obs/lib/binpkg_deb.py new file mode 100644 index 0000000..b26623c --- /dev/null +++ b/scripts/obs/lib/binpkg_deb.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright 2022 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> +import os +import glob +import lib.config + + +def extract_source(srcdir, bindir): + tarball = glob.glob(f"{srcdir}/*.tar.*")[0] + + print(f"extracting {os.path.basename(tarball)}") + lib.run_cmd(["tar", "-xf", tarball], cwd=bindir) + + return glob.glob(f"{bindir}/*/")[0] + + +def build(srcdir, jobs): + bindir = f"{lib.config.path_temp}/binpkg" + extractdir = extract_source(srcdir, bindir) + + lib.set_cmds_verbose(True) + + # install deps + lib.run_cmd(["apt-get", "-y", "build-dep", "."], cwd=extractdir) + + print("running dpkg-buildpackage") + lib.run_cmd(["dpkg-buildpackage", "-us", "-uc", f"-j{jobs}"], + cwd=extractdir) diff --git a/scripts/obs/lib/binpkg_rpm.py b/scripts/obs/lib/binpkg_rpm.py new file mode 100644 index 0000000..b26623c --- /dev/null +++ b/scripts/obs/lib/binpkg_rpm.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright 2022 sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> +import os +import glob +import lib.config + + +def extract_source(srcdir, bindir): + tarball = glob.glob(f"{srcdir}/*.tar.*")[0] + + print(f"extracting {os.path.basename(tarball)}") + lib.run_cmd(["tar", "-xf", tarball], cwd=bindir) + + return glob.glob(f"{bindir}/*/")[0] + + +def build(srcdir, jobs): + bindir = f"{lib.config.path_temp}/binpkg" + extractdir = extract_source(srcdir, bindir) + + lib.set_cmds_verbose(True) + + # install deps + lib.run_cmd(["apt-get", "-y", "build-dep", "."], cwd=extractdir) + + print("running dpkg-buildpackage") + lib.run_cmd(["dpkg-buildpackage", "-us", "-uc", f"-j{jobs}"], + cwd=extractdir) diff --git a/scripts/obs/lib/config.py b/scripts/obs/lib/config.py index 52b84fb..393251b 100644 --- a/scripts/obs/lib/config.py +++ b/scripts/obs/lib/config.py @@ -112,3 +112,6 @@ } docker_distro_default = "debian:11" +docker_distro_other = [ + "centos:8", +] diff --git a/scripts/obs/lib/docker.py b/scripts/obs/lib/docker.py index 2944cb3..81464bb 100644 --- a/scripts/obs/lib/docker.py +++ b/scripts/obs/lib/docker.py @@ -14,11 +14,23 @@ return ret +def get_distro_from(distro): + # CentOS 8 is EOL (SYS#5818) + if distro == "centos:8": + return "almalinux:8" + + return distro + + def build_image(distro, image_type): - image_name = get_image_name(distro, image_type), + image_name = get_image_name(distro, image_type) + distro_from = get_distro_from(distro) + print(f"docker: building image {image_name}") + lib.run_cmd(["docker", "build", "--build-arg", f"DISTRO={distro}", + "--build-arg", f"DISTRO_FROM={distro_from}", "--build-arg", f"UID={os.getuid()}", "-t", image_name, "-f", f"{lib.config.path_top}/data/{image_type}.Dockerfile", @@ -38,12 +50,15 @@ def run_in_docker_and_exit(script_path, add_oscrc=False, - image_type="build_srcpkg", distro=None): + image_type="build_srcpkg", distro=None, + pass_argv=True, env={}): """ :param script_path: what to run inside docker, relative to scripts/obs/ :param add_oscrc: put user's oscrc in docker (contains obs credentials!) :param image_type: which Dockerfile to use (data/{image_type}.Dockerfile) :param distro: which Linux distribution to use, e.g. "debian:11" + :param pass_argv: pass arguments from sys.argv to the script + :param env: dict of environment variables """ if "INSIDE_DOCKER" in os.environ: return @@ -72,10 +87,17 @@ "-e", "PYTHONUNBUFFERED=1", "-v", f"{lib.config.path_top}:/obs"] + + for env_key, env_val in env.items(): + cmd += ["-e", f"{env_key}={env_val}"] + if oscrc: cmd += ["-v", f"{oscrc}:/home/user/.oscrc"] - cmd += [image_name, f"/obs/{script_path}"] + sys.argv[1:] + cmd += [image_name, f"/obs/{script_path}"] + + if pass_argv: + cmd += sys.argv[1:] print(f"docker: running: {script_path} inside docker") ret = subprocess.run(cmd) -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/29601 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: I4c6b5d61af35df98cbc70d9ddc8ad36d38a9ce18 Gerrit-Change-Number: 29601 Gerrit-PatchSet: 2 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

Change in osmo-ci[master]: obs: build_binpkg: debian: don't install manpages
by osmith 06 Oct '22

06 Oct '22

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/29602 ) Change subject: obs: build_binpkg: debian: don't install manpages ...................................................................... obs: build_binpkg: debian: don't install manpages Configure dpkg to not extract man pages. Otherwise it will spend some time regenerating the man page index whenever installing build dependencies before starting to build a package. Related: OS#2385 Change-Id: I1c9e3883b976e023c96dfd59eb147770f7ad99a7 --- M scripts/obs/data/build_binpkg.Dockerfile 1 file changed, 5 insertions(+), 0 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved pespin: Looks good to me, but someone else must approve diff --git a/scripts/obs/data/build_binpkg.Dockerfile b/scripts/obs/data/build_binpkg.Dockerfile index feacbc9..de53478 100644 --- a/scripts/obs/data/build_binpkg.Dockerfile +++ b/scripts/obs/data/build_binpkg.Dockerfile @@ -12,8 +12,13 @@ # build recipe. For rpm-based distributions, there is no build-essential or # similar package. Instead add relevant packages from prjconf, e.g.: # https://build.opensuse.org/projects/CentOS:CentOS-8/prjconf +# For debian, make sure we don't have man pages as otherwise it takes some time +# to regenerate the manuals database when installing build dependencies. RUN case "$DISTRO" in \ debian*) \ + echo "path-exclude=/usr/share/man/*" \ + > /etc/dpkg/dpkg.cfg.d/exclude-man-pages && \ + rm -rf /usr/share/man/ && \ apt-get update && \ apt-get install -y --no-install-recommends \ build-essential \ -- To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/29602 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ci Gerrit-Branch: master Gerrit-Change-Id: I1c9e3883b976e023c96dfd59eb147770f7ad99a7 Gerrit-Change-Number: 29602 Gerrit-PatchSet: 2 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged

1 0

2026

2025

2024

2023

2022

gerrit-log