<p>osmith <strong>merged</strong> this change.</p><p><a href="https://gerrit.osmocom.org/10932">View Change</a></p><div style="white-space:pre-wrap">Approvals:
Neels Hofmeyr: Looks good to me, approved
osmith: Verified
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">osmo-depcheck: script to verify PKG_CHECK_MODULES<br><br>This script verifies that Osomcom programs really build with the<br>dependency versions they claim to support in configure.ac. In order to<br>do that, it clones the dependency repositories if they don't exist<br>already, and checks out the minimum version tag. This happens<br>recursively for their dependencies as well. See 'osmo-depcheck.py -h'<br>for the full usage instructions.<br><br>There's also a new jenkins job in jobs/osmocom-depcheck.yml.<br><br>Change-Id: I8f495dbe030775f66ac125e60ded95c5d7660b65<br>Relates: OS#2642<br>---<br>A jobs/osmocom-depcheck.yml<br>A scripts/osmo-depcheck/buildstack.py<br>A scripts/osmo-depcheck/config.py<br>A scripts/osmo-depcheck/dependencies.py<br>A scripts/osmo-depcheck/osmo-depcheck.py<br>A scripts/osmo-depcheck/parse.py<br>6 files changed, 590 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/jobs/osmocom-depcheck.yml b/jobs/osmocom-depcheck.yml</span><br><span>new file mode 100644</span><br><span>index 0000000..f13d4b7</span><br><span>--- /dev/null</span><br><span>+++ b/jobs/osmocom-depcheck.yml</span><br><span>@@ -0,0 +1,72 @@</span><br><span style="color: hsl(120, 100%, 40%);">+---</span><br><span style="color: hsl(120, 100%, 40%);">+- project:</span><br><span style="color: hsl(120, 100%, 40%);">+ name: Osmocom-depcheck</span><br><span style="color: hsl(120, 100%, 40%);">+ jobs:</span><br><span style="color: hsl(120, 100%, 40%);">+ - Osmocom-depcheck</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+- job-template:</span><br><span style="color: hsl(120, 100%, 40%);">+ name: 'Osmocom-depcheck'</span><br><span style="color: hsl(120, 100%, 40%);">+ project-type: freestyle</span><br><span style="color: hsl(120, 100%, 40%);">+ defaults: global</span><br><span style="color: hsl(120, 100%, 40%);">+ description: |</span><br><span style="color: hsl(120, 100%, 40%);">+ Verifies that Osmocom programs really build with the dependency</span><br><span style="color: hsl(120, 100%, 40%);">+ versions they claim to support in configure.ac.</span><br><span style="color: hsl(120, 100%, 40%);">+ (Generated by job-builder)</span><br><span style="color: hsl(120, 100%, 40%);">+ node: osmocom-master-debian9</span><br><span style="color: hsl(120, 100%, 40%);">+ parameters:</span><br><span style="color: hsl(120, 100%, 40%);">+ - string:</span><br><span style="color: hsl(120, 100%, 40%);">+ name: PROJECTS</span><br><span style="color: hsl(120, 100%, 40%);">+ description: |</span><br><span style="color: hsl(120, 100%, 40%);">+ Which Osmocom projects and revisions to build, leave</span><br><span style="color: hsl(120, 100%, 40%);">+ empty to default to all projects (!),</span><br><span style="color: hsl(120, 100%, 40%);">+ default revision is "master".</span><br><span style="color: hsl(120, 100%, 40%);">+ Examples: "osmo-hlr", "osmo-hlr:0.2.1 osmo-bts:0.8.1"</span><br><span style="color: hsl(120, 100%, 40%);">+ default: 'osmo-hlr:0.2.1'</span><br><span style="color: hsl(120, 100%, 40%);">+ - string:</span><br><span style="color: hsl(120, 100%, 40%);">+ name: GIT_URL_PREFIX</span><br><span style="color: hsl(120, 100%, 40%);">+ description: |</span><br><span style="color: hsl(120, 100%, 40%);">+ Where to clone the sources from</span><br><span style="color: hsl(120, 100%, 40%);">+ default: 'git://git.osmocom.org/'</span><br><span style="color: hsl(120, 100%, 40%);">+ - bool:</span><br><span style="color: hsl(120, 100%, 40%);">+ name: BUILD</span><br><span style="color: hsl(120, 100%, 40%);">+ description: |</span><br><span style="color: hsl(120, 100%, 40%);">+ Attempt to build the project with the minimum dependency</span><br><span style="color: hsl(120, 100%, 40%);">+ versions found in the configure.ac files. If this is unchecked,</span><br><span style="color: hsl(120, 100%, 40%);">+ this job will only clone the git repositories and parse the</span><br><span style="color: hsl(120, 100%, 40%);">+ configure.ac files.</span><br><span style="color: hsl(120, 100%, 40%);">+ default: true</span><br><span style="color: hsl(120, 100%, 40%);">+ - bool:</span><br><span style="color: hsl(120, 100%, 40%);">+ name: PRINT_OLD_DEPENDS</span><br><span style="color: hsl(120, 100%, 40%);">+ description: |</span><br><span style="color: hsl(120, 100%, 40%);">+ Report dependencies on old releases (printed after the other</span><br><span style="color: hsl(120, 100%, 40%);">+ parsing output, before the build starts)</span><br><span style="color: hsl(120, 100%, 40%);">+ default: false</span><br><span style="color: hsl(120, 100%, 40%);">+ - string:</span><br><span style="color: hsl(120, 100%, 40%);">+ name: BRANCH</span><br><span style="color: hsl(120, 100%, 40%);">+ description: |</span><br><span style="color: hsl(120, 100%, 40%);">+ Branch where the osmo-depcheck.py script gets pulled from.</span><br><span style="color: hsl(120, 100%, 40%);">+ Only modify this if you are hacking on osmo-depcheck.py.</span><br><span style="color: hsl(120, 100%, 40%);">+ default: '*/master'</span><br><span style="color: hsl(120, 100%, 40%);">+ builders:</span><br><span style="color: hsl(120, 100%, 40%);">+ - shell: |</span><br><span style="color: hsl(120, 100%, 40%);">+ # Build the arguments</span><br><span style="color: hsl(120, 100%, 40%);">+ args="$PROJECTS"</span><br><span style="color: hsl(120, 100%, 40%);">+ args="$args -j 5"</span><br><span style="color: hsl(120, 100%, 40%);">+ args="$args -g $PWD/DEPCHECK_GITDIR"</span><br><span style="color: hsl(120, 100%, 40%);">+ args="$args -u $GIT_URL_PREFIX"</span><br><span style="color: hsl(120, 100%, 40%);">+ [ "$BUILD" = "true" ] && args="$args -b"</span><br><span style="color: hsl(120, 100%, 40%);">+ [ "$PRINT_OLD_DEPENDS" = "true" ] && args="$args -o"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Run osmo-depcheck</span><br><span style="color: hsl(120, 100%, 40%);">+ mkdir DEPCHECK_GITDIR</span><br><span style="color: hsl(120, 100%, 40%);">+ export PYTHONUNBUFFERED=1</span><br><span style="color: hsl(120, 100%, 40%);">+ scripts/osmo-depcheck/osmo-depcheck.py $args</span><br><span style="color: hsl(120, 100%, 40%);">+ scm:</span><br><span style="color: hsl(120, 100%, 40%);">+ - git:</span><br><span style="color: hsl(120, 100%, 40%);">+ branches:</span><br><span style="color: hsl(120, 100%, 40%);">+ - '$BRANCH'</span><br><span style="color: hsl(120, 100%, 40%);">+ url: git://git.osmocom.org/osmo-ci</span><br><span style="color: hsl(120, 100%, 40%);">+ git-config-name: 'Jenkins Builder'</span><br><span style="color: hsl(120, 100%, 40%);">+ git-config-email: 'jenkins@osmocom.org'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# vim: expandtab tabstop=2 shiftwidth=2</span><br><span>diff --git a/scripts/osmo-depcheck/buildstack.py b/scripts/osmo-depcheck/buildstack.py</span><br><span>new file mode 100644</span><br><span>index 0000000..87210ab</span><br><span>--- /dev/null</span><br><span>+++ b/scripts/osmo-depcheck/buildstack.py</span><br><span>@@ -0,0 +1,144 @@</span><br><span style="color: hsl(120, 100%, 40%);">+# SPDX-License-Identifier: GPL-2.0-or-later</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright 2018 sysmocom - s.f.m.c. GmbH <info@sysmocom.de></span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import atexit</span><br><span style="color: hsl(120, 100%, 40%);">+import collections</span><br><span style="color: hsl(120, 100%, 40%);">+import sys</span><br><span style="color: hsl(120, 100%, 40%);">+import os</span><br><span style="color: hsl(120, 100%, 40%);">+import shutil</span><br><span style="color: hsl(120, 100%, 40%);">+import subprocess</span><br><span style="color: hsl(120, 100%, 40%);">+import tempfile</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def next_buildable(depends, done):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Find the next program that can be built, because it has all</span><br><span style="color: hsl(120, 100%, 40%);">+ dependencies satisfied. Initially this would be libosmocore, as it has</span><br><span style="color: hsl(120, 100%, 40%);">+ no dependencies, then the only library that depends on libosmocore and</span><br><span style="color: hsl(120, 100%, 40%);">+ so on.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param depends: return value of dependencies.generate()</span><br><span style="color: hsl(120, 100%, 40%);">+ :param done: ordered dict of programs that would already have been</span><br><span style="color: hsl(120, 100%, 40%);">+ built at this point.</span><br><span style="color: hsl(120, 100%, 40%);">+ Example: {"lib-a": "0.11.0", "lib-b": "0.5.0"}</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Iterate over dependencies</span><br><span style="color: hsl(120, 100%, 40%);">+ for program, data in depends.items():</span><br><span style="color: hsl(120, 100%, 40%);">+ # Skip what's already done</span><br><span style="color: hsl(120, 100%, 40%);">+ if program in done:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Check for missing dependencies</span><br><span style="color: hsl(120, 100%, 40%);">+ depends_done = True</span><br><span style="color: hsl(120, 100%, 40%);">+ for depend in data["depends"]:</span><br><span style="color: hsl(120, 100%, 40%);">+ if depend not in done:</span><br><span style="color: hsl(120, 100%, 40%);">+ depends_done = False</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # All dependencies satisfied: we have a winner!</span><br><span style="color: hsl(120, 100%, 40%);">+ if depends_done:</span><br><span style="color: hsl(120, 100%, 40%);">+ return program, data["version"]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Impossible to build the dependency tree</span><br><span style="color: hsl(120, 100%, 40%);">+ print_dict(done)</span><br><span style="color: hsl(120, 100%, 40%);">+ print("ERROR: can't figure out how to build the rest!")</span><br><span style="color: hsl(120, 100%, 40%);">+ sys.exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def generate(depends):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Generate an ordered dictionary with the right build order.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param depends: return value of dependencies.generate()</span><br><span style="color: hsl(120, 100%, 40%);">+ :returns: an ordered dict like the following:</span><br><span style="color: hsl(120, 100%, 40%);">+ {"libosmocore": "0.11.0",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-abis": "0.5.0",</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-bts": "master"} """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Iterate over dependencies</span><br><span style="color: hsl(120, 100%, 40%);">+ ret = collections.OrderedDict()</span><br><span style="color: hsl(120, 100%, 40%);">+ count = len(depends.keys())</span><br><span style="color: hsl(120, 100%, 40%);">+ while len(ret) != count:</span><br><span style="color: hsl(120, 100%, 40%);">+ # Continue with the one without unsatisfied dependencies</span><br><span style="color: hsl(120, 100%, 40%);">+ program, version = next_buildable(depends, ret)</span><br><span style="color: hsl(120, 100%, 40%);">+ ret[program] = version</span><br><span style="color: hsl(120, 100%, 40%);">+ return ret</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def print_dict(stack):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Print the whole build stack.</span><br><span style="color: hsl(120, 100%, 40%);">+ :param stack: return value from generate() above """</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Build order:")</span><br><span style="color: hsl(120, 100%, 40%);">+ for program, version in stack.items():</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" * " + program + ":" + version)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def temp_install_folder():</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Generate a temporary installation folder</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ It will be used as configure prefix, so when running 'make install',</span><br><span style="color: hsl(120, 100%, 40%);">+ the files will get copied in there instead of "/usr/local/". The folder</span><br><span style="color: hsl(120, 100%, 40%);">+ will get removed when the script has finished.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :returns: the path to the temporary folder """</span><br><span style="color: hsl(120, 100%, 40%);">+ ret = tempfile.mkdtemp(prefix="depcheck_")</span><br><span style="color: hsl(120, 100%, 40%);">+ atexit.register(shutil.rmtree, ret)</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Temporary install folder: " + ret)</span><br><span style="color: hsl(120, 100%, 40%);">+ return ret</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def set_environment(jobs, tempdir):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Configure the environment variables before running configure, make etc.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param jobs: parallel build jobs (for make)</span><br><span style="color: hsl(120, 100%, 40%);">+ :param tempdir: temporary installation dir (see temp_install_folder())</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Add tempdir to PKG_CONFIG_PATH and LD_LIBRARY_PATH</span><br><span style="color: hsl(120, 100%, 40%);">+ extend = {"PKG_CONFIG_PATH": tempdir + "/lib/pkgconfig",</span><br><span style="color: hsl(120, 100%, 40%);">+ "LD_LIBRARY_PATH": tempdir + "/lib"}</span><br><span style="color: hsl(120, 100%, 40%);">+ for env_var, folder in extend.items():</span><br><span style="color: hsl(120, 100%, 40%);">+ old = os.environ[env_var] if env_var in os.environ else ""</span><br><span style="color: hsl(120, 100%, 40%);">+ os.environ[env_var] = old + ":" + folder</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Set JOBS for make</span><br><span style="color: hsl(120, 100%, 40%);">+ os.environ["JOBS"] = str(jobs)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def build(gitdir, jobs, stack):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Build one program with all its dependencies.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param gitdir: folder to which the sources will be cloned</span><br><span style="color: hsl(120, 100%, 40%);">+ :param jobs: parallel build jobs (for make)</span><br><span style="color: hsl(120, 100%, 40%);">+ :param stack: the build stack as returned by generate() above</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ The dependencies.clone() function has already downloaded missing</span><br><span style="color: hsl(120, 100%, 40%);">+ sources and checked out the right version tags. So in this function we</span><br><span style="color: hsl(120, 100%, 40%);">+ can directly enter the source folder and run the build commands.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ Notes about the usage of 'make clean' and 'make distclean':</span><br><span style="color: hsl(120, 100%, 40%);">+ * Without 'make clean' we might have files in the build directory with</span><br><span style="color: hsl(120, 100%, 40%);">+ a different prefix hardcoded (e.g. from a previous run of</span><br><span style="color: hsl(120, 100%, 40%);">+ osmo-depcheck):</span><br><span style="color: hsl(120, 100%, 40%);">+ <https://lists.gnu.org/archive/html/libtool/2006-12/msg00011.html></span><br><span style="color: hsl(120, 100%, 40%);">+ * 'make distclean' gets used to remove everything that mentioned the</span><br><span style="color: hsl(120, 100%, 40%);">+ prefix set by osmo-depcheck. That way the user won't have it set</span><br><span style="color: hsl(120, 100%, 40%);">+ anymore in case they decide to compile the code again manually from</span><br><span style="color: hsl(120, 100%, 40%);">+ the source folder. """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Prepare the install folder and environment</span><br><span style="color: hsl(120, 100%, 40%);">+ tempdir = temp_install_folder()</span><br><span style="color: hsl(120, 100%, 40%);">+ unitdir = tempdir + "/lib/systemd/system/"</span><br><span style="color: hsl(120, 100%, 40%);">+ set_environment(jobs, tempdir)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Iterate over stack</span><br><span style="color: hsl(120, 100%, 40%);">+ for program, version in stack.items():</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Building " + program + ":" + version)</span><br><span style="color: hsl(120, 100%, 40%);">+ os.chdir(gitdir + "/" + program)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Run the build commands</span><br><span style="color: hsl(120, 100%, 40%);">+ commands = [["autoreconf", "-fi"],</span><br><span style="color: hsl(120, 100%, 40%);">+ ["./configure", "--prefix", tempdir,</span><br><span style="color: hsl(120, 100%, 40%);">+ "--with-systemdsystemunitdir=" + unitdir],</span><br><span style="color: hsl(120, 100%, 40%);">+ ["make", "clean"],</span><br><span style="color: hsl(120, 100%, 40%);">+ ["make"],</span><br><span style="color: hsl(120, 100%, 40%);">+ ["make", "install"],</span><br><span style="color: hsl(120, 100%, 40%);">+ ["make", "distclean"]]</span><br><span style="color: hsl(120, 100%, 40%);">+ for command in commands:</span><br><span style="color: hsl(120, 100%, 40%);">+ print("+ " + " ".join(command))</span><br><span style="color: hsl(120, 100%, 40%);">+ subprocess.run(command, check=True)</span><br><span>diff --git a/scripts/osmo-depcheck/config.py b/scripts/osmo-depcheck/config.py</span><br><span>new file mode 100644</span><br><span>index 0000000..3e993bf</span><br><span>--- /dev/null</span><br><span>+++ b/scripts/osmo-depcheck/config.py</span><br><span>@@ -0,0 +1,43 @@</span><br><span style="color: hsl(120, 100%, 40%);">+# SPDX-License-Identifier: GPL-2.0-or-later</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright 2018 sysmocom - s.f.m.c. GmbH <info@sysmocom.de></span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Where to clone sources from (with trailing slash)</span><br><span style="color: hsl(120, 100%, 40%);">+git_url_prefix = "git://git.osmocom.org/"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Default projects to build when none are specified on the command line</span><br><span style="color: hsl(120, 100%, 40%);">+projects = ("osmo-bts",</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-pcu",</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-hlr",</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-mgw",</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-msc",</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-sgsn",</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-ggsn")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Libraries coming from Osmocom repositories (glob patterns)</span><br><span style="color: hsl(120, 100%, 40%);">+# All other libraries (e.g. libsystemd) are ignored by this script, even if</span><br><span style="color: hsl(120, 100%, 40%);">+# they are mentioned with PKG_CHECK_MODULES in configure.ac.</span><br><span style="color: hsl(120, 100%, 40%);">+relevant_library_patterns = ("libasn1c",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libgtp",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo*")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Library locations in the git repositories</span><br><span style="color: hsl(120, 100%, 40%);">+# Libraries that have the same name as the git repository don't need to be</span><br><span style="color: hsl(120, 100%, 40%);">+# listed here. Left: repository name, right: libraries</span><br><span style="color: hsl(120, 100%, 40%);">+repos = {"libosmocore": ("libosmocodec",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmocoding",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmoctrl",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmogb",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmogsm",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmosim",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmovty"),</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-abis": ("libosmoabis",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmotrau"),</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-sccp": ("libosmo-mtp",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-sigtran",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-xua"),</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-ggsn": ("libgtp"),</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-hlr": ("libosmo-gsup-client"),</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-iuh": ("libosmo-ranap"),</span><br><span style="color: hsl(120, 100%, 40%);">+ "osmo-mgw": ("libosmo-mgcp-client",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-legacy-mgcp")}</span><br><span>diff --git a/scripts/osmo-depcheck/dependencies.py b/scripts/osmo-depcheck/dependencies.py</span><br><span>new file mode 100644</span><br><span>index 0000000..78cf4a0</span><br><span>--- /dev/null</span><br><span>+++ b/scripts/osmo-depcheck/dependencies.py</span><br><span>@@ -0,0 +1,114 @@</span><br><span style="color: hsl(120, 100%, 40%);">+# SPDX-License-Identifier: GPL-2.0-or-later</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright 2018 sysmocom - s.f.m.c. GmbH <info@sysmocom.de></span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import collections</span><br><span style="color: hsl(120, 100%, 40%);">+import os</span><br><span style="color: hsl(120, 100%, 40%);">+import subprocess</span><br><span style="color: hsl(120, 100%, 40%);">+import sys</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Same folder</span><br><span style="color: hsl(120, 100%, 40%);">+import parse</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def git_clone(gitdir, prefix, repository, version):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Clone a missing git repository and checkout a specific version tag.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param gitdir: folder to which the sources will be cloned</span><br><span style="color: hsl(120, 100%, 40%);">+ :param prefix: git url prefix (e.g. "git://git.osmocom.org/")</span><br><span style="color: hsl(120, 100%, 40%);">+ :param repository: Osmocom git repository name (e.g. "libosmo-abis")</span><br><span style="color: hsl(120, 100%, 40%);">+ :param version: "master" or a version tag like "0.11.0" """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Clone when needed</span><br><span style="color: hsl(120, 100%, 40%);">+ if not os.path.exists(gitdir + "/" + repository):</span><br><span style="color: hsl(120, 100%, 40%);">+ url = prefix + repository</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Cloning git repo: " + url)</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ subprocess.run(["git", "-C", gitdir, "clone", "-q", url],</span><br><span style="color: hsl(120, 100%, 40%);">+ check=True)</span><br><span style="color: hsl(120, 100%, 40%);">+ except subprocess.CalledProcessError:</span><br><span style="color: hsl(120, 100%, 40%);">+ print("NOTE: if '" + repository + "' is part of a git repository"</span><br><span style="color: hsl(120, 100%, 40%);">+ " with a different name, please add it to the mapping in"</span><br><span style="color: hsl(120, 100%, 40%);">+ " 'config.py' and try again.")</span><br><span style="color: hsl(120, 100%, 40%);">+ sys.exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Checkout the version tag</span><br><span style="color: hsl(120, 100%, 40%);">+ subprocess.run(["git", "-C", gitdir + "/" + repository, "checkout",</span><br><span style="color: hsl(120, 100%, 40%);">+ version, "-q"], check=True)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def generate(gitdir, prefix, initial, rev):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Generate the dependency graph of an Osmocom program by cloning the git</span><br><span style="color: hsl(120, 100%, 40%);">+ repository, parsing the "configure.ac" file, and recursing.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param gitdir: folder to which the sources will be cloned</span><br><span style="color: hsl(120, 100%, 40%);">+ :param prefix: git url prefix (e.g. "git://git.osmocom.org/")</span><br><span style="color: hsl(120, 100%, 40%);">+ :param initial: the first program to look at (e.g. "osmo-bts")</span><br><span style="color: hsl(120, 100%, 40%);">+ :param rev: the git revision to check out ("master", "0.1.0", ...)</span><br><span style="color: hsl(120, 100%, 40%);">+ :returns: a dictionary like the following:</span><br><span style="color: hsl(120, 100%, 40%);">+ {"osmo-bts": {"version": "master",</span><br><span style="color: hsl(120, 100%, 40%);">+ "depends": {"libosmocore": "0.11.0",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-abis": "0.5.0"}},</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmocore": {"version": "0.11.0",</span><br><span style="color: hsl(120, 100%, 40%);">+ "depends": {}},</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-abis": {"version": "0.5.0",</span><br><span style="color: hsl(120, 100%, 40%);">+ "depends": {"libosmocore": "0.11.0"}} """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Iterate over stack</span><br><span style="color: hsl(120, 100%, 40%);">+ stack = collections.OrderedDict({initial: rev})</span><br><span style="color: hsl(120, 100%, 40%);">+ ret = collections.OrderedDict()</span><br><span style="color: hsl(120, 100%, 40%);">+ while len(stack):</span><br><span style="color: hsl(120, 100%, 40%);">+ # Pop program from stack</span><br><span style="color: hsl(120, 100%, 40%);">+ program, version = next(iter(stack.items()))</span><br><span style="color: hsl(120, 100%, 40%);">+ del stack[program]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Skip when already parsed</span><br><span style="color: hsl(120, 100%, 40%);">+ if program in ret:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Add the programs dependencies to the stack</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Looking at " + program + ":" + version)</span><br><span style="color: hsl(120, 100%, 40%);">+ git_clone(gitdir, prefix, program, version)</span><br><span style="color: hsl(120, 100%, 40%);">+ depends = parse.configure_ac(gitdir, program)</span><br><span style="color: hsl(120, 100%, 40%);">+ stack.update(depends)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Add the program to the ret</span><br><span style="color: hsl(120, 100%, 40%);">+ ret[program] = {"version": version, "depends": depends}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ return ret</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def print_dict(depends):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Print the whole dependency graph.</span><br><span style="color: hsl(120, 100%, 40%);">+ :param depends: return value from generate() above """</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Dependency graph:")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for program, data in depends.items():</span><br><span style="color: hsl(120, 100%, 40%);">+ version = data["version"]</span><br><span style="color: hsl(120, 100%, 40%);">+ depends = data["depends"]</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" * " + program + ":" + version + " depends: " + str(depends))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def git_latest_tag(gitdir, repository):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Get the last release string by asking git for the latest tag.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param gitdir: folder to which the sources will be cloned</span><br><span style="color: hsl(120, 100%, 40%);">+ :param repository: Osmocom git repository name (e.g. "libosmo-abis")</span><br><span style="color: hsl(120, 100%, 40%);">+ :returns: the latest git tag (e.g. "1.0.2") """</span><br><span style="color: hsl(120, 100%, 40%);">+ dir = gitdir + "/" + repository</span><br><span style="color: hsl(120, 100%, 40%);">+ complete = subprocess.run(["git", "-C", dir, "describe", "--abbrev=0",</span><br><span style="color: hsl(120, 100%, 40%);">+ "master"], check=True, stdout=subprocess.PIPE)</span><br><span style="color: hsl(120, 100%, 40%);">+ return complete.stdout.decode().rstrip()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def print_old(gitdir, depends):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Print dependencies tied to an old release tag</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param gitdir: folder to which the sources will be cloned</span><br><span style="color: hsl(120, 100%, 40%);">+ :param depends: return value from generate() above """</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Dependencies on old releases:")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for program, data in depends.items():</span><br><span style="color: hsl(120, 100%, 40%);">+ for depend, version in data["depends"].items():</span><br><span style="color: hsl(120, 100%, 40%);">+ latest = git_latest_tag(gitdir, depend)</span><br><span style="color: hsl(120, 100%, 40%);">+ if latest == version:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" * " + program + ":" + data["version"] + " -> " +</span><br><span style="color: hsl(120, 100%, 40%);">+ depend + ":" + version + " (latest: " + latest + ")")</span><br><span>diff --git a/scripts/osmo-depcheck/osmo-depcheck.py b/scripts/osmo-depcheck/osmo-depcheck.py</span><br><span>new file mode 100755</span><br><span>index 0000000..92c0ce6</span><br><span>--- /dev/null</span><br><span>+++ b/scripts/osmo-depcheck/osmo-depcheck.py</span><br><span>@@ -0,0 +1,101 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span style="color: hsl(120, 100%, 40%);">+# SPDX-License-Identifier: GPL-2.0-or-later</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright 2018 sysmocom - s.f.m.c. GmbH <info@sysmocom.de></span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import argparse</span><br><span style="color: hsl(120, 100%, 40%);">+import os</span><br><span style="color: hsl(120, 100%, 40%);">+import sys</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Same folder</span><br><span style="color: hsl(120, 100%, 40%);">+import config</span><br><span style="color: hsl(120, 100%, 40%);">+import dependencies</span><br><span style="color: hsl(120, 100%, 40%);">+import buildstack</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def parse_arguments():</span><br><span style="color: hsl(120, 100%, 40%);">+ # Create argparser</span><br><span style="color: hsl(120, 100%, 40%);">+ description = ("This script verifies that Osmocom programs really build"</span><br><span style="color: hsl(120, 100%, 40%);">+ " with the dependency versions they claim to support in"</span><br><span style="color: hsl(120, 100%, 40%);">+ " configure.ac. In order to do that, it clones the"</span><br><span style="color: hsl(120, 100%, 40%);">+ " dependency repositories if they don't exist in gitdir"</span><br><span style="color: hsl(120, 100%, 40%);">+ " already, and checks out the minimum version tag. This"</span><br><span style="color: hsl(120, 100%, 40%);">+ " happens recursively for their dependencies as well.")</span><br><span style="color: hsl(120, 100%, 40%);">+ parser = argparse.ArgumentParser(description=description)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Git sources folder</span><br><span style="color: hsl(120, 100%, 40%);">+ gitdir_default = os.path.expanduser("~") + "/code"</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument("-g", "--gitdir", default=gitdir_default,</span><br><span style="color: hsl(120, 100%, 40%);">+ help="folder to which the sources will be cloned"</span><br><span style="color: hsl(120, 100%, 40%);">+ " (default: " + gitdir_default + ")")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Build switch</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument("-b", "--build", action="store_true",</span><br><span style="color: hsl(120, 100%, 40%);">+ help="don't only parse the dependencies, but also try"</span><br><span style="color: hsl(120, 100%, 40%);">+ " to build the program")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Build switch</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument("-o", "--old", action="store_true",</span><br><span style="color: hsl(120, 100%, 40%);">+ help="report dependencies on old releases")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Job count</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument("-j", "--jobs", type=int,</span><br><span style="color: hsl(120, 100%, 40%);">+ help="parallel build jobs (for make)")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Git URL prefix</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument("-u", "--git-url-prefix", dest="prefix",</span><br><span style="color: hsl(120, 100%, 40%);">+ default=config.git_url_prefix,</span><br><span style="color: hsl(120, 100%, 40%);">+ help="where to clone the sources from (default: " +</span><br><span style="color: hsl(120, 100%, 40%);">+ config.git_url_prefix + ")")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Projects</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument("projects_revs", nargs="*", default=config.projects,</span><br><span style="color: hsl(120, 100%, 40%);">+ help="which Osmocom projects to look at"</span><br><span style="color: hsl(120, 100%, 40%);">+ " (e.g. 'osmo-hlr:0.2.1', 'osmo-bts', defaults to"</span><br><span style="color: hsl(120, 100%, 40%);">+ " all projects defined in config.py, default"</span><br><span style="color: hsl(120, 100%, 40%);">+ " revision is 'master')",</span><br><span style="color: hsl(120, 100%, 40%);">+ metavar="project[:revision]")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Gitdir must exist</span><br><span style="color: hsl(120, 100%, 40%);">+ ret = parser.parse_args()</span><br><span style="color: hsl(120, 100%, 40%);">+ if not os.path.exists(ret.gitdir):</span><br><span style="color: hsl(120, 100%, 40%);">+ print("ERROR: gitdir does not exist: " + ret.gitdir)</span><br><span style="color: hsl(120, 100%, 40%);">+ sys.exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ return ret</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def main():</span><br><span style="color: hsl(120, 100%, 40%);">+ # Iterate over projects</span><br><span style="color: hsl(120, 100%, 40%);">+ args = parse_arguments()</span><br><span style="color: hsl(120, 100%, 40%);">+ for project_rev in args.projects_revs:</span><br><span style="color: hsl(120, 100%, 40%);">+ # Split the git revision from the project name</span><br><span style="color: hsl(120, 100%, 40%);">+ project = project_rev</span><br><span style="color: hsl(120, 100%, 40%);">+ rev = "master"</span><br><span style="color: hsl(120, 100%, 40%);">+ if ":" in project_rev:</span><br><span style="color: hsl(120, 100%, 40%);">+ project, rev = project_rev.split(":", 1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Clone and parse the repositories</span><br><span style="color: hsl(120, 100%, 40%);">+ depends = dependencies.generate(args.gitdir, args.prefix, project, rev)</span><br><span style="color: hsl(120, 100%, 40%);">+ print("---")</span><br><span style="color: hsl(120, 100%, 40%);">+ dependencies.print_dict(depends)</span><br><span style="color: hsl(120, 100%, 40%);">+ stack = buildstack.generate(depends)</span><br><span style="color: hsl(120, 100%, 40%);">+ print("---")</span><br><span style="color: hsl(120, 100%, 40%);">+ buildstack.print_dict(stack)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Old versions</span><br><span style="color: hsl(120, 100%, 40%);">+ if args.old:</span><br><span style="color: hsl(120, 100%, 40%);">+ print("---")</span><br><span style="color: hsl(120, 100%, 40%);">+ dependencies.print_old(args.gitdir, depends)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Build</span><br><span style="color: hsl(120, 100%, 40%);">+ if args.build:</span><br><span style="color: hsl(120, 100%, 40%);">+ print("---")</span><br><span style="color: hsl(120, 100%, 40%);">+ buildstack.build(args.gitdir, args.jobs, stack)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Success</span><br><span style="color: hsl(120, 100%, 40%);">+ print("---")</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Success for " + project + ":" + rev + "!")</span><br><span style="color: hsl(120, 100%, 40%);">+ print("---")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+if __name__ == '__main__':</span><br><span style="color: hsl(120, 100%, 40%);">+ main()</span><br><span>diff --git a/scripts/osmo-depcheck/parse.py b/scripts/osmo-depcheck/parse.py</span><br><span>new file mode 100644</span><br><span>index 0000000..c6297d6</span><br><span>--- /dev/null</span><br><span>+++ b/scripts/osmo-depcheck/parse.py</span><br><span>@@ -0,0 +1,116 @@</span><br><span style="color: hsl(120, 100%, 40%);">+# SPDX-License-Identifier: GPL-2.0-or-later</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright 2018 sysmocom - s.f.m.c. GmbH <info@sysmocom.de></span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import sys</span><br><span style="color: hsl(120, 100%, 40%);">+import fnmatch</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Same folder</span><br><span style="color: hsl(120, 100%, 40%);">+import config</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def error(line_i, message):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Print a configure.ac error message with the line number.</span><br><span style="color: hsl(120, 100%, 40%);">+ :param line_i: the zero based line counter """</span><br><span style="color: hsl(120, 100%, 40%);">+ print("ERROR: configure.ac line " + str(line_i+1) + ": " + message)</span><br><span style="color: hsl(120, 100%, 40%);">+ sys.exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def repository(library, version):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Find the git repository that contains a certain library. Based on the</span><br><span style="color: hsl(120, 100%, 40%);">+ information in config.py.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param library: the name as referenced in the PKG_CHECK_MODULES</span><br><span style="color: hsl(120, 100%, 40%);">+ statement. For example: "libosmoabis"</span><br><span style="color: hsl(120, 100%, 40%);">+ :param version: for example "0.5.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ :returns: the repository name, e.g. "libosmo-abis" """</span><br><span style="color: hsl(120, 100%, 40%);">+ for repo, libraries in config.repos.items():</span><br><span style="color: hsl(120, 100%, 40%);">+ if library in libraries:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" * " + library + ":" + version + " (part of " + repo + ")")</span><br><span style="color: hsl(120, 100%, 40%);">+ return repo</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" * " + library + ":" + version)</span><br><span style="color: hsl(120, 100%, 40%);">+ return library</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def library_is_relevant(library):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ :returns: True when we would build the library in question from source,</span><br><span style="color: hsl(120, 100%, 40%);">+ False otherwise. """</span><br><span style="color: hsl(120, 100%, 40%);">+ for pattern in config.relevant_library_patterns:</span><br><span style="color: hsl(120, 100%, 40%);">+ if fnmatch.fnmatch(library, pattern):</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def parse_condition(line):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Find the PKG_CHECK_MODULES conditions in any line from a configure.ac.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ Example lines:</span><br><span style="color: hsl(120, 100%, 40%);">+ PKG_CHECK_MODULES(LIBOSMOCORE, libosmocore >= 0.10.0)</span><br><span style="color: hsl(120, 100%, 40%);">+ PKG_CHECK_MODULES(LIBSYSTEMD, libsystemd)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :returns: * None when there's no condition in that line</span><br><span style="color: hsl(120, 100%, 40%);">+ * a string like "libosmocore >= 0.1.0" """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Only look at PKG_CHECK_MODULES lines</span><br><span style="color: hsl(120, 100%, 40%);">+ if "PKG_CHECK_MODULES" not in line:</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Extract the condition</span><br><span style="color: hsl(120, 100%, 40%);">+ ret = line.split(",")[1].split(")")[0].strip()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Only look at Osmocom libraries</span><br><span style="color: hsl(120, 100%, 40%);">+ library = ret.split(" ")[0]</span><br><span style="color: hsl(120, 100%, 40%);">+ if library_is_relevant(library):</span><br><span style="color: hsl(120, 100%, 40%);">+ return ret</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def library_version(line_i, condition):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Get the library and version strings from a condition.</span><br><span style="color: hsl(120, 100%, 40%);">+ :param line_i: the zero based line counter</span><br><span style="color: hsl(120, 100%, 40%);">+ :param condition: a condition like "libosmocore >= 0.1.0" """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Split by space and remove empty list elements</span><br><span style="color: hsl(120, 100%, 40%);">+ split = list(filter(None, condition.split(" ")))</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(split) != 3:</span><br><span style="color: hsl(120, 100%, 40%);">+ error(line_i, "invalid condition format, expected something"</span><br><span style="color: hsl(120, 100%, 40%);">+ " like 'libosmocore >= 0.10.0' but got: '" +</span><br><span style="color: hsl(120, 100%, 40%);">+ condition + "'")</span><br><span style="color: hsl(120, 100%, 40%);">+ library, operator, version = split</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Right operator</span><br><span style="color: hsl(120, 100%, 40%);">+ if operator == ">=":</span><br><span style="color: hsl(120, 100%, 40%);">+ return (library, version)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Wrong operator</span><br><span style="color: hsl(120, 100%, 40%);">+ error(line_i, "invalid operator, expected '>=' but got: '" +</span><br><span style="color: hsl(120, 100%, 40%);">+ operator + "'")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def configure_ac(gitdir, repo):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ Parse the PKG_CHECK_MODULES statements of a configure.ac file.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ :param gitdir: parent folder of all locally cloned git repositories</span><br><span style="color: hsl(120, 100%, 40%);">+ :param repo: the repository to look at (e.g. "osmo-bts")</span><br><span style="color: hsl(120, 100%, 40%);">+ :returns: a dictionary like the following:</span><br><span style="color: hsl(120, 100%, 40%);">+ {"libosmocore": "0.11.0",</span><br><span style="color: hsl(120, 100%, 40%);">+ "libosmo-abis": "0.5.0"} """</span><br><span style="color: hsl(120, 100%, 40%);">+ # Read configure.ac</span><br><span style="color: hsl(120, 100%, 40%);">+ path = gitdir + "/" + repo + "/configure.ac"</span><br><span style="color: hsl(120, 100%, 40%);">+ with open(path) as handle:</span><br><span style="color: hsl(120, 100%, 40%);">+ lines = handle.readlines()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Parse the file into ret</span><br><span style="color: hsl(120, 100%, 40%);">+ ret = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ for i in range(0, len(lines)):</span><br><span style="color: hsl(120, 100%, 40%);">+ # Parse the line</span><br><span style="color: hsl(120, 100%, 40%);">+ condition = parse_condition(lines[i])</span><br><span style="color: hsl(120, 100%, 40%);">+ if not condition:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ (library, version) = library_version(i, condition)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Add to ret (with duplicate check)</span><br><span style="color: hsl(120, 100%, 40%);">+ repo_dependency = repository(library, version)</span><br><span style="color: hsl(120, 100%, 40%);">+ if repo_dependency in ret and version != ret[repo_dependency]:</span><br><span style="color: hsl(120, 100%, 40%);">+ error(i, "found multiple PKG_CHECK_MODULES statements for " +</span><br><span style="color: hsl(120, 100%, 40%);">+ repo_dependency + ".git, and they have different"</span><br><span style="color: hsl(120, 100%, 40%);">+ " versions!")</span><br><span style="color: hsl(120, 100%, 40%);">+ ret[repo_dependency] = version</span><br><span style="color: hsl(120, 100%, 40%);">+ return ret</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/10932">change 10932</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/10932"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-ci </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: merged </div>
<div style="display:none"> Gerrit-Change-Id: I8f495dbe030775f66ac125e60ded95c5d7660b65 </div>
<div style="display:none"> Gerrit-Change-Number: 10932 </div>
<div style="display:none"> Gerrit-PatchSet: 5 </div>
<div style="display:none"> Gerrit-Owner: osmith <osmith@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: Harald Welte <laforge@gnumonks.org> </div>
<div style="display:none"> Gerrit-Reviewer: Neels Hofmeyr <nhofmeyr@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: osmith <osmith@sysmocom.de> </div>
<div style="display:none"> Gerrit-CC: Pau Espin Pedrol <pespin@sysmocom.de> </div>