[L] Change in docker-playground[master]: debian-bookworm-build(-arm): new docker containers

laforge has submitted this change. ( https://gerrit.osmocom.org/c/docker-playground/+/33740 ) Change subject: debian-bookworm-build(-arm): new docker containers ...................................................................... debian-bookworm-build(-arm): new docker containers Create debian bookworm (12) based containers for building all Osmocom projects in master/gerrit/ttcn3 jenkins jobs. This is a combination of debian-bullseye-build (ttcn3) and debian-bullseye-jenkins(-arm) (master/gerrit), which were used for this purpose but had several differences. Combining them solves the following problems: * The naming was confusing, as both the -jenkins and -build containers were used in jenkins and were used to build the Osmocom stack. * This lead to adding the dependencies to the wrong containers / not adding them to both containers. * Now we are sure that if a program builds from source in the master and gerrit verifications, it will also build in the ttcn3 jobs. Other notable changes from the debian 11 containers: * Python2 is finally gone (OS#5950, for the few projects that still need it like openbsc, I'll use debian 11 for the jenkins jobs) * Removed osc and git-buildpackage, now that OBS package building is done in different docker containers (see osmo-ci/scripts/obs) * Combined multiple RUN commands into one as they the image build down, used set -x to display what commands run exactly * Use UID variable instead of hardcoding 1000 * Optimized order of commands, so e.g. the LLVM for Arm that we will rarely change gets installed/cached before the big list of apt pkgs * Replaced old git.osmocom.org urls * Removed apt-get upgrade; the base image will get upgraded from time to time, no need to upgrade within the image * Add --depth=1 option to git clones and remove temporary git clones * Removed generating locales code, it was only done by one of the two images and I couldn't find in the git log why we would need this (if we do need it we can add it back later) The following patches add more debian bookworm containers and have similar improvements. Related: OS#6057 Change-Id: I49aaf62b5b97775f923453611df3b91354a640a0 --- A debian-bookworm-build-arm/Dockerfile A debian-bookworm-build-arm/Makefile A debian-bookworm-build/Dockerfile A debian-bookworm-build/Makefile 4 files changed, 355 insertions(+), 0 deletions(-) Approvals: laforge: Looks good to me, approved pespin: Looks good to me, but someone else must approve Jenkins Builder: Verified diff --git a/debian-bookworm-build-arm/Dockerfile b/debian-bookworm-build-arm/Dockerfile new file mode 100644 index 0000000..9a4b2b4 --- /dev/null +++ b/debian-bookworm-build-arm/Dockerfile @@ -0,0 +1,89 @@ +# Image used to run contrib/jenkins.sh scripts of a few Osmocom projects on +# arm. Keep this image minimal so the rpis don't need forever to build it. +# See master-builds.yml, gerrit-verifications.yml in osmo-ci.git. +ARG REGISTRY=docker.io +ARG UPSTREAM_DISTRO=debian:bookworm +FROM ${REGISTRY}/${UPSTREAM_DISTRO} + +# Arguments used after FROM must be specified again +ARG OSMOCOM_REPO_MIRROR="https://downloads.osmocom.org" +ARG OSMOCOM_REPO_PATH="packages/osmocom:" +ARG OSMOCOM_REPO="${OSMOCOM_REPO_MIRROR}/${OSMOCOM_REPO_PATH}/nightly/Debian_12/" +ARG UID + +# Copy from common dir +COPY .common/Release.key /etc/apt/trusted.gpg.d/obs.osmocom.org.asc + +# Configure build user, disable installing man pages +# * man pages: without them we avoid waiting for "generating manpages" +RUN set -x && \ + useradd --uid=${UID} build && \ + mkdir /build && \ + chown -R build:build /build /usr/local && \ + \ + echo "path-exclude=/usr/share/man/*" \ + > /etc/dpkg/dpkg.cfg.d/exclude-man-pages && \ + rm -rf /usr/share/man/ + +# Install packages from Debian repositories (alphabetic order) +RUN set -x && \ + apt-get update && \ + apt-get install -y --no-install-recommends \ + autoconf \ + automake \ + bzip2 \ + ca-certificates \ + ccache \ + g++ \ + gcc \ + git \ + libboost-dev \ + libboost-filesystem-dev \ + libboost-program-options-dev \ + libboost-thread-dev \ + libfftw3-dev \ + libgnutls28-dev \ + libmnl-dev \ + libortp-dev \ + libpcsclite-dev \ + libsctp-dev \ + libtalloc-dev \ + libtool \ + liburing-dev \ + libusb-1.0-0-dev \ + make \ + pkg-config \ + python3-minimal \ + python3-setuptools \ + sdcc \ + stow \ + && \ + apt-get clean + +# Install osmo-python-tests +ADD https://gerrit.osmocom.org/plugins/gitiles/python/osmo-python-tests/+/maste… /tmp/osmo-python-tests-commit +RUN set -x && \ + git clone --depth=1 https://gerrit.osmocom.org/python/osmo-python-tests osmo-python-tests && \ + cd osmo-python-tests && \ + python3 setup.py clean build install && \ + rm -rf osmo-python-tests + +# Install osmo-ci.git/scripts to /usr/local/bin +ADD https://gerrit.osmocom.org/plugins/gitiles/osmo-ci/+/master?format=TEXT /tmp/osmo-ci-commit +RUN set -x && \ + git clone --depth=1 https://gerrit.osmocom.org/osmo-ci osmo-ci && \ + su build -c "cd osmo-ci/scripts && cp -v *.sh *.py /usr/local/bin" && \ + rm -rf osmo-ci + +# Install packages from Osmocom OBS nightly repository +# * osmo-trx: liblimesuite-dev, libuhd-dev +ADD $OSMOCOM_REPO/Release /tmp/Release +RUN set -x && \ + echo "deb [signed-by=/etc/apt/trusted.gpg.d/obs.osmocom.org.asc] $OSMOCOM_REPO ./" \ + > /etc/apt/sources.list.d/osmocom-nightly.list && \ + apt-get update && \ + apt-get install -y --no-install-recommends \ + liblimesuite-dev \ + libuhd-dev \ + && \ + apt-get clean diff --git a/debian-bookworm-build-arm/Makefile b/debian-bookworm-build-arm/Makefile new file mode 100644 index 0000000..cdac90c --- /dev/null +++ b/debian-bookworm-build-arm/Makefile @@ -0,0 +1,3 @@ +UPSTREAM_DISTRO=debian:bookworm +DISTRO=debian-bookworm +include ../make/Makefile diff --git a/debian-bookworm-build/Dockerfile b/debian-bookworm-build/Dockerfile new file mode 100644 index 0000000..f717241 --- /dev/null +++ b/debian-bookworm-build/Dockerfile @@ -0,0 +1,215 @@ +# Image for building all Osmocom projects in master/gerrit/ttcn3 jenkins jobs +ARG REGISTRY=docker.io +ARG UPSTREAM_DISTRO=debian:bookworm +FROM ${REGISTRY}/${UPSTREAM_DISTRO} + +# Arguments used after FROM must be specified again +ARG OSMOCOM_REPO_MIRROR="https://downloads.osmocom.org" +ARG OSMOCOM_REPO_PATH="packages/osmocom:" +ARG OSMOCOM_REPO="$OSMOCOM_REPO_MIRROR/${OSMOCOM_REPO_PATH}/nightly/Debian_12/" +ARG UID +ARG LLVM_VERSION="14.0.0" + +# Copy from common dir +COPY .common/respawn.sh /usr/local/bin/respawn.sh +COPY .common/Release.key /etc/apt/trusted.gpg.d/obs.osmocom.org.asc + +# Configure build user, disable installing man pages +# * /usr/local: osmo-python-tests's contrib/jenkins.sh writes there +# * man pages: without them we avoid waiting for "generating manpages" +RUN set -x && \ + useradd --uid=${UID} build && \ + mkdir /build && \ + chown -R build:build /build /usr/local && \ + \ + echo "path-exclude=/usr/share/man/*" \ + > /etc/dpkg/dpkg.cfg.d/exclude-man-pages && \ + rm -rf /usr/share/man/ + +# Install LLVM-embedded-toolchain-for-Arm +RUN if [ "$(arch)" != "x86_64" ]; then \ + echo "ERROR: use debian-bookworm-build-arm instead"; \ + exit 1; \ + fi && \ + set -x && \ + apt-get update && \ + apt-get install -y --no-install-recommends \ + ca-certificates \ + libtinfo5 \ + wget \ + && \ + apt-get clean && \ + wget https://github.com/ARM-software/LLVM-embedded-toolchain-for-Arm/releases/do… && \ + tar -xf LLVMEmbeddedToolchainForArm-${LLVM_VERSION}-linux.tar.gz && \ + rm LLVMEmbeddedToolchainForArm-${LLVM_VERSION}-linux.tar.gz && \ + mv LLVMEmbeddedToolchainForArm-${LLVM_VERSION} /opt/llvm-arm && \ + /opt/llvm-arm/bin/clang --version && \ + /opt/llvm-arm/bin/clang --print-targets + +# Install packages from Debian repositories (alphabetic order) +RUN set -x && \ + apt-get update && \ + apt-get install -y --no-install-recommends \ + asciidoc \ + asciidoc-dblatex \ + autoconf \ + autoconf-archive \ + autogen \ + automake \ + bc \ + bison \ + build-essential \ + bzip2 \ + ca-certificates \ + ccache \ + cmake \ + coccinelle \ + cppcheck \ + dahdi-source \ + dblatex \ + dbus \ + debhelper \ + devscripts \ + dh-autoreconf \ + docbook5-xml \ + doxygen \ + equivs \ + flex \ + g++ \ + gawk \ + gcc \ + gcc-arm-none-eabi \ + ghostscript \ + git \ + gnupg \ + graphviz \ + htop \ + inkscape \ + iproute2 \ + latexmk \ + lcov \ + libaio-dev \ + libasound2-dev \ + libboost-all-dev \ + libc-ares-dev \ + libcdk5-dev \ + libcsv-dev \ + libdbd-sqlite3 \ + libdbi-dev \ + libelf-dev \ + libffi-dev \ + libfftw3-dev \ + libgmp-dev \ + libgnutls28-dev \ + libgps-dev \ + libgsm1-dev \ + libjansson-dev \ + liblua5.3-dev \ + libmnl-dev \ + libncurses5-dev \ + libnewlib-arm-none-eabi \ + libnl-3-dev \ + libnl-route-3-dev \ + liboping-dev \ + libortp-dev \ + libpcap-dev \ + libpcsclite-dev \ + libreadline-dev \ + libsctp-dev \ + libsigsegv-dev \ + libsnmp-dev \ + libsofia-sip-ua-glib-dev \ + libsqlite3-dev \ + libssl-dev \ + libtalloc-dev \ + libtinfo5 \ + libtool \ + liburing-dev \ + libusb-1.0-0-dev \ + libusb-dev \ + libxml2-utils \ + libzmq3-dev \ + locales \ + lua-socket \ + make \ + mscgen \ + ofono \ + openssh-client \ + patchelf \ + picolibc-arm-none-eabi \ + pkg-config \ + python3 \ + python3-gi \ + python3-mako \ + python3-nwdiag \ + python3-pip \ + python3-pyflakes \ + python3-setuptools \ + python3-usb \ + python3-yaml \ + rsync \ + sdcc \ + source-highlight \ + sqlite3 \ + stow \ + sudo \ + swig \ + systemd \ + tcpdump \ + telnet \ + tex-gyre \ + texinfo \ + unzip \ + xsltproc \ + && \ + apt-get clean + +# Install pip dependencies (alphabetic order) +# break-system-packages: inside docker it's fine to install pkgs system-wide +ADD https://gitea.osmocom.org/sim-card/pysim/raw/branch/master/requirements.txt /tmp/pysim_requirements.txt +RUN set -x && \ + cat /tmp/pysim_requirements.txt && \ + pip3 install --break-system-packages \ + 'git+https://github.com/eriwen/lcov-to-cobertura-xml.git' \ + 'git+https://github.com/osmocom/sphinx-argparse@master#egg=sphinx-argparse' \ + 'git+https://github.com/podshumok/python-smpplib.git' \ + 'pydbus' \ + 'pylint' \ + 'pysispm' \ + 'sphinx' \ + 'sphinxcontrib-napoleon' \ + -r /tmp/pysim_requirements.txt + +# Install osmo-python-tests +ADD https://gerrit.osmocom.org/plugins/gitiles/python/osmo-python-tests/+/maste… /tmp/osmo-python-tests-commit +RUN set -x && \ + git clone --depth=1 https://gerrit.osmocom.org/python/osmo-python-tests osmo-python-tests && \ + cd osmo-python-tests && \ + python3 setup.py clean build install && \ + rm -rf osmo-python-tests + +# Install osmo-ci.git/scripts to /usr/local/bin +ADD https://gerrit.osmocom.org/plugins/gitiles/osmo-ci/+/master?format=TEXT /tmp/osmo-ci-commit +RUN set -x && \ + git clone --depth=1 https://gerrit.osmocom.org/osmo-ci osmo-ci && \ + su build -c "cd osmo-ci/scripts && cp -v *.sh *.py /usr/local/bin" && \ + rm -rf osmo-ci + +# Install osmo-gsm-manuals to /opt/osmo-gsm-manuals +ADD https://gerrit.osmocom.org/plugins/gitiles/osmo-gsm-manuals/+/master?format… /tmp/osmo-gsm-manuals-commit +RUN git -C /opt clone --depth=1 https://gerrit.osmocom.org/osmo-gsm-manuals + +# Install packages from Osmocom OBS nightly repository +# * osmo-remsim: libulfius +# * osmo-trx: liblimesuite-dev, libuhd-dev +ADD $OSMOCOM_REPO/Release /tmp/Release +RUN set -x && \ + echo "deb [signed-by=/etc/apt/trusted.gpg.d/obs.osmocom.org.asc] $OSMOCOM_REPO ./" \ + > /etc/apt/sources.list.d/osmocom-nightly.list && \ + apt-get update && \ + apt-get install -y --no-install-recommends \ + liblimesuite-dev \ + libuhd-dev \ + libulfius-dev \ + && \ + apt-get clean diff --git a/debian-bookworm-build/Makefile b/debian-bookworm-build/Makefile new file mode 100644 index 0000000..cdac90c --- /dev/null +++ b/debian-bookworm-build/Makefile @@ -0,0 +1,3 @@ +UPSTREAM_DISTRO=debian:bookworm +DISTRO=debian-bookworm +include ../make/Makefile -- To view, visit https://gerrit.osmocom.org/c/docker-playground/+/33740 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: docker-playground Gerrit-Branch: master Gerrit-Change-Id: I49aaf62b5b97775f923453611df3b91354a640a0 Gerrit-Change-Number: 33740 Gerrit-PatchSet: 2 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: merged