<p>pespin has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/18469">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">Support identifying different tgz files based on run_label attribute<br><br>tgz files in trials can be categorized in subdirectories, allowing to<br>select different bianry files at runtime based on the target run node<br>which is going to run them. This way for instance one can have a binary<br>linked against libs for eg. CentOS under run_label "centos/" or an ARM<br>target under "arm", and then use "run_label: arm" on the resource using<br>it.<br><br>Change-Id: Iaf2e97da3aff693395f44f0e93b184d4846cf6da<br>---<br>M .gitignore<br>M doc/manuals/chapters/trial.adoc<br>M selftest/Makefile<br>A selftest/trial_test/run_label/checksums.md5<br>A selftest/trial_test/run_label/foobar/sample.tar.gz<br>A selftest/trial_test/run_label/sample.tar.gz<br>M selftest/trial_test/trial_test.ok<br>M selftest/trial_test/trial_test.ok.ign<br>M selftest/trial_test/trial_test.py<br>M src/osmo_gsm_tester/core/trial.py<br>M src/osmo_gsm_tester/obj/run_node.py<br>11 files changed, 75 insertions(+), 21 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/69/18469/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/.gitignore b/.gitignore</span><br><span>index ad9b423..ff09035 100644</span><br><span>--- a/.gitignore</span><br><span>+++ b/.gitignore</span><br><span>@@ -4,13 +4,16 @@</span><br><span> .version</span><br><span> _version.py</span><br><span> tags</span><br><span style="color: hsl(0, 100%, 40%);">-set_pythonpath</span><br><span style="color: hsl(0, 100%, 40%);">-test_work</span><br><span> state</span><br><span> *.pyc</span><br><span> sysmocom/resources.conf</span><br><span> sysmocom/ttcn3/resources.conf</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+# selftest:</span><br><span style="color: hsl(120, 100%, 40%);">+set_pythonpath</span><br><span style="color: hsl(120, 100%, 40%);">+test_work</span><br><span style="color: hsl(120, 100%, 40%);">+selftest/trial_test/run_label/inst/</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # manuals</span><br><span> doc/manuals/*.html</span><br><span> doc/manuals/*.svg</span><br><span>diff --git a/doc/manuals/chapters/trial.adoc b/doc/manuals/chapters/trial.adoc</span><br><span>index 4b52608..3ee6ae6 100644</span><br><span>--- a/doc/manuals/chapters/trial.adoc</span><br><span>+++ b/doc/manuals/chapters/trial.adoc</span><br><span>@@ -48,13 +48,22 @@</span><br><span> </span><br><span> {app-name} tests create objects to manage the allocated resources during test</span><br><span> lifetime. These objects, in turn, usually run and manage processes started from</span><br><span style="color: hsl(0, 100%, 40%);">-the trail's sysroot binaries. {app-name} provide APIs for those object classes</span><br><span style="color: hsl(120, 100%, 40%);">+the trial's sysroot binaries. {app-name} provide APIs for those object classes</span><br><span> to discover, unpack and run those binaries. An object class simply needs to</span><br><span> request the name of the sysroot it wants to use (for instance 'osmo-bsc'), and</span><br><span> {app-name} will take care of preparing everything and providing the sysroot path</span><br><span> to it. It's a duty of the resource class to copy over the sysroot to the</span><br><span> destination if the intention is to run the binary remotely on another host.</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+Moreover, {app-name} supports working with several different versions of a given</span><br><span style="color: hsl(120, 100%, 40%);">+sysroot by means of storing them in different subdirectories, which are later</span><br><span style="color: hsl(120, 100%, 40%);">+referenced by an object's class 'run_label' attribute named after that</span><br><span style="color: hsl(120, 100%, 40%);">+subdirectory. This way, for instance, a sysroot can be provided containing</span><br><span style="color: hsl(120, 100%, 40%);">+binaries linked against libraries present on a CentOS distribution, and other</span><br><span style="color: hsl(120, 100%, 40%);">+sysroots with the same name can also be provided which are linked against</span><br><span style="color: hsl(120, 100%, 40%);">+different versions of CentOS, or a different distro like Debian, or even a</span><br><span style="color: hsl(120, 100%, 40%);">+different arch like ARM.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> When seeking a sysroot of a given name '<inst-name>' in the 'inst/' directory,</span><br><span> {app-name} will look for 'tgz' files starting with the pattern '<inst-name>.'</span><br><span> (up to the first dot). That means, suffixes are available for {app-name} user to</span><br><span>@@ -63,3 +72,6 @@</span><br><span> selected by {app-name} for 'osmo-bsc': 'osmo-bsc.tgz', 'osmo-bsc.build-23.tgz',</span><br><span> 'osmo-bsc.5f3e0dd2.tgz', 'osmo-bsc.armv7.build-2.tgz'. If either none or more</span><br><span> than one valid file is found matching the pattern, an exception will be thrown.</span><br><span style="color: hsl(120, 100%, 40%);">+If a 'run_label=foobar' is provided, {app-name} will look up for the 'tgz' in</span><br><span style="color: hsl(120, 100%, 40%);">+exactly the same way, but this time in 'inst/foobar/' directory instead of</span><br><span style="color: hsl(120, 100%, 40%);">+'inst/'.</span><br><span>diff --git a/selftest/Makefile b/selftest/Makefile</span><br><span>index fb8618c..53d882c 100644</span><br><span>--- a/selftest/Makefile</span><br><span>+++ b/selftest/Makefile</span><br><span>@@ -12,6 +12,7 @@</span><br><span> clean:</span><br><span> @find . -name "*__pycache__" -type d -print0 | xargs -0 rm -rvf</span><br><span> @find . -name "*test_work" -type d -print0 | xargs -0 rm -rvf</span><br><span style="color: hsl(120, 100%, 40%);">+ @rm -rfv ./trial_test/run_label/inst</span><br><span> @rm -fv ./set_pythonpath</span><br><span> </span><br><span> # vim: noexpandtab tabstop=8 shiftwidth=8</span><br><span>diff --git a/selftest/trial_test/run_label/checksums.md5 b/selftest/trial_test/run_label/checksums.md5</span><br><span>new file mode 100644</span><br><span>index 0000000..a162c56</span><br><span>--- /dev/null</span><br><span>+++ b/selftest/trial_test/run_label/checksums.md5</span><br><span>@@ -0,0 +1,2 @@</span><br><span style="color: hsl(120, 100%, 40%);">+b13c9c94d41c45caaa08ad9ad3ee0f11 sample.tar.gz</span><br><span style="color: hsl(120, 100%, 40%);">+430ec62329747b5f15fe6e2cb141b909 foobar/sample.tar.gz</span><br><span>diff --git a/selftest/trial_test/run_label/foobar/sample.tar.gz b/selftest/trial_test/run_label/foobar/sample.tar.gz</span><br><span>new file mode 100644</span><br><span>index 0000000..2c1b029</span><br><span>--- /dev/null</span><br><span>+++ b/selftest/trial_test/run_label/foobar/sample.tar.gz</span><br><span>Binary files differ</span><br><span>diff --git a/selftest/trial_test/run_label/sample.tar.gz b/selftest/trial_test/run_label/sample.tar.gz</span><br><span>new file mode 100644</span><br><span>index 0000000..3932964</span><br><span>--- /dev/null</span><br><span>+++ b/selftest/trial_test/run_label/sample.tar.gz</span><br><span>Binary files differ</span><br><span>diff --git a/selftest/trial_test/trial_test.ok b/selftest/trial_test/trial_test.ok</span><br><span>index 8c6a567..75bd1a7 100644</span><br><span>--- a/selftest/trial_test/trial_test.ok</span><br><span>+++ b/selftest/trial_test/trial_test.ok</span><br><span>@@ -14,3 +14,12 @@</span><br><span> ok, got RuntimeError: Checksum mismatch for '[PATH]/trial_test/invalid_checksum/file2' vs. '[PATH]/trial_test/invalid_checksum/checksums.md5' line 2</span><br><span> - detect missing file</span><br><span> ok, got RuntimeError: File listed in checksums file but missing in trials dir: '[PATH]/trial_test/missing_file/file2' vs. '[PATH]/trial_test/missing_file/checksums.md5' line 2</span><br><span style="color: hsl(120, 100%, 40%);">+- Verify trials based on run_label</span><br><span style="color: hsl(120, 100%, 40%);">+tst run_label: DBG: has bin_tar {bin_name='sample', matches=['sample.tar.gz'], run_label='foobar'}</span><br><span style="color: hsl(120, 100%, 40%);">+inst: [PATH]/trial_test/run_label/inst/foobar/sample</span><br><span style="color: hsl(120, 100%, 40%);">+content file2: subhello</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+tst run_label: DBG: has bin_tar {bin_name='sample', matches=['sample.tar.gz'], run_label=''}</span><br><span style="color: hsl(120, 100%, 40%);">+inst: [PATH]/trial_test/run_label/inst/sample</span><br><span style="color: hsl(120, 100%, 40%);">+content file1: hello</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span>diff --git a/selftest/trial_test/trial_test.ok.ign b/selftest/trial_test/trial_test.ok.ign</span><br><span>index 1a969de..46c3943 100644</span><br><span>--- a/selftest/trial_test/trial_test.ok.ign</span><br><span>+++ b/selftest/trial_test/trial_test.ok.ign</span><br><span>@@ -1,3 +1,3 @@</span><br><span> /tmp/[^/]* [TMP]</span><br><span> ....-..-.._..-..-.. [TIMESTAMP]</span><br><span style="color: hsl(0, 100%, 40%);">-'[^']*/trial_test '[PATH]/trial_test</span><br><span style="color: hsl(120, 100%, 40%);">+/[^ ]*/trial_test/ [PATH]/trial_test/</span><br><span>diff --git a/selftest/trial_test/trial_test.py b/selftest/trial_test/trial_test.py</span><br><span>index a99428a..cf91a85 100755</span><br><span>--- a/selftest/trial_test/trial_test.py</span><br><span>+++ b/selftest/trial_test/trial_test.py</span><br><span>@@ -1,8 +1,10 @@</span><br><span> #!/usr/bin/env python3</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-import time</span><br><span> import _prep</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import time</span><br><span> import os</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> from osmo_gsm_tester.core import util</span><br><span> from osmo_gsm_tester.core.trial import Trial</span><br><span> </span><br><span>@@ -46,4 +48,17 @@</span><br><span> except RuntimeError as e:</span><br><span> print('ok, got RuntimeError: %s' % str(e))</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+print('- Verify trials based on run_label')</span><br><span style="color: hsl(120, 100%, 40%);">+d = util.Dir('trial_test')</span><br><span style="color: hsl(120, 100%, 40%);">+t = Trial(d.child('run_label'))</span><br><span style="color: hsl(120, 100%, 40%);">+t.verify()</span><br><span style="color: hsl(120, 100%, 40%);">+inst = util.Dir(t.get_inst('sample', 'foobar'))</span><br><span style="color: hsl(120, 100%, 40%);">+print('inst: ' + str(inst))</span><br><span style="color: hsl(120, 100%, 40%);">+with open(inst.child('file2'), 'r') as f:</span><br><span style="color: hsl(120, 100%, 40%);">+ print('content file2: %s' % f.read())</span><br><span style="color: hsl(120, 100%, 40%);">+inst = util.Dir( t.get_inst('sample'))</span><br><span style="color: hsl(120, 100%, 40%);">+print('inst: ' + str(inst))</span><br><span style="color: hsl(120, 100%, 40%);">+with open(inst.child('file1'), 'r') as f:</span><br><span style="color: hsl(120, 100%, 40%);">+ print('content file1: %s' % f.read())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # vim: expandtab tabstop=4 shiftwidth=4</span><br><span>diff --git a/src/osmo_gsm_tester/core/trial.py b/src/osmo_gsm_tester/core/trial.py</span><br><span>index 83b0f11..eaf18c3 100644</span><br><span>--- a/src/osmo_gsm_tester/core/trial.py</span><br><span>+++ b/src/osmo_gsm_tester/core/trial.py</span><br><span>@@ -57,7 +57,7 @@</span><br><span> super().__init__(log.C_TST, os.path.basename(self.path))</span><br><span> self.dir = util.Dir(self.path)</span><br><span> self.inst_dir = util.Dir(self.dir.child('inst'))</span><br><span style="color: hsl(0, 100%, 40%);">- self.bin_tars = []</span><br><span style="color: hsl(120, 100%, 40%);">+ self.bin_tars = {}</span><br><span> self.suites = []</span><br><span> self.status = Trial.UNKNOWN</span><br><span> self._run_dir = None</span><br><span>@@ -125,10 +125,10 @@</span><br><span> line_nr += 1</span><br><span> if not line:</span><br><span> continue</span><br><span style="color: hsl(0, 100%, 40%);">- md5, filename = line.split(' ')</span><br><span style="color: hsl(0, 100%, 40%);">- file_path = self.dir.child(filename)</span><br><span style="color: hsl(120, 100%, 40%);">+ md5, relpath = line.split(' ')</span><br><span style="color: hsl(120, 100%, 40%);">+ file_path = self.dir.child(relpath)</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if not self.dir.isfile(filename):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not self.dir.isfile(relpath):</span><br><span> raise RuntimeError('File listed in checksums file but missing in trials dir:'</span><br><span> ' %r vs. %r line %d' % (file_path, checksums, line_nr))</span><br><span> </span><br><span>@@ -136,28 +136,34 @@</span><br><span> raise RuntimeError('Checksum mismatch for %r vs. %r line %d'</span><br><span> % (file_path, checksums, line_nr))</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if filename.endswith('.tgz'):</span><br><span style="color: hsl(0, 100%, 40%);">- self.bin_tars.append(filename)</span><br><span style="color: hsl(120, 100%, 40%);">+ if relpath.endswith('.tgz') or relpath.endswith('.tar.gz'):</span><br><span style="color: hsl(120, 100%, 40%);">+ (label, name) = os.path.split(relpath)</span><br><span style="color: hsl(120, 100%, 40%);">+ #print('label: %s, name: %s' % (label, name))</span><br><span style="color: hsl(120, 100%, 40%);">+ li = self.bin_tars.get(label, [])</span><br><span style="color: hsl(120, 100%, 40%);">+ li.append(name)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.bin_tars[label] = li</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def has_bin_tar(self, bin_name):</span><br><span style="color: hsl(120, 100%, 40%);">+ def has_bin_tar(self, bin_name, run_label):</span><br><span> bin_tar_start = '%s.' % bin_name</span><br><span style="color: hsl(0, 100%, 40%);">- matches = [t for t in self.bin_tars if t.startswith(bin_tar_start)]</span><br><span style="color: hsl(0, 100%, 40%);">- self.dbg(bin_name=bin_name, matches=matches)</span><br><span style="color: hsl(120, 100%, 40%);">+ matches = [t for t in self.bin_tars[run_label] if t.startswith(bin_tar_start)]</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg('has bin_tar', run_label=run_label, bin_name=bin_name, matches=matches)</span><br><span> if not matches:</span><br><span> return None</span><br><span> if len(matches) > 1:</span><br><span style="color: hsl(0, 100%, 40%);">- raise RuntimeError('More than one match for bin name %r: %r' % (bin_name, matches))</span><br><span style="color: hsl(120, 100%, 40%);">+ raise RuntimeError('More than one match for bin name %r on run_label \'%s\': %r' % (bin_name, run_label, matches))</span><br><span> bin_tar = matches[0]</span><br><span style="color: hsl(0, 100%, 40%);">- bin_tar_path = self.dir.child(bin_tar)</span><br><span style="color: hsl(120, 100%, 40%);">+ bin_tar_path = self.dir.child(os.path.join(run_label, bin_tar))</span><br><span> if not os.path.isfile(bin_tar_path):</span><br><span> raise RuntimeError('Not a file or missing: %r' % bin_tar_path)</span><br><span> return bin_tar_path</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def get_inst(self, bin_name):</span><br><span style="color: hsl(0, 100%, 40%);">- bin_tar = self.has_bin_tar(bin_name)</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_inst(self, bin_name, run_label=None):</span><br><span style="color: hsl(120, 100%, 40%);">+ if run_label is None:</span><br><span style="color: hsl(120, 100%, 40%);">+ run_label = ''</span><br><span style="color: hsl(120, 100%, 40%);">+ bin_tar = self.has_bin_tar(bin_name, run_label)</span><br><span> if not bin_tar:</span><br><span> raise RuntimeError('No such binary available: %r' % bin_name)</span><br><span style="color: hsl(0, 100%, 40%);">- inst_dir = self.inst_dir.child(bin_name)</span><br><span style="color: hsl(120, 100%, 40%);">+ inst_dir = self.inst_dir.child(os.path.join(run_label, bin_name))</span><br><span> </span><br><span> if os.path.isdir(inst_dir):</span><br><span> # already unpacked</span><br><span>diff --git a/src/osmo_gsm_tester/obj/run_node.py b/src/osmo_gsm_tester/obj/run_node.py</span><br><span>index 26c85df..53bb3fe 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/run_node.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/run_node.py</span><br><span>@@ -35,12 +35,13 @@</span><br><span> T_LOCAL = 'local'</span><br><span> T_REM_SSH = 'ssh'</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def __init__(self, type=None, run_addr=None, ssh_user=None, ssh_addr=None):</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, type=None, run_addr=None, ssh_user=None, ssh_addr=None, run_label=None):</span><br><span> super().__init__(log.C_RUN, 'runnode')</span><br><span> self._type = type</span><br><span> self._run_addr = run_addr</span><br><span> self._ssh_user = ssh_user</span><br><span> self._ssh_addr = ssh_addr</span><br><span style="color: hsl(120, 100%, 40%);">+ self._run_label = run_label</span><br><span> if not self._type:</span><br><span> raise log.Error('run_type not set')</span><br><span> if not self._run_addr:</span><br><span>@@ -57,7 +58,9 @@</span><br><span> </span><br><span> @classmethod</span><br><span> def from_conf(cls, conf):</span><br><span style="color: hsl(0, 100%, 40%);">- return cls(conf.get('run_type', None), conf.get('run_addr', None), conf.get('ssh_user', None), conf.get('ssh_addr', None))</span><br><span style="color: hsl(120, 100%, 40%);">+ return cls(conf.get('run_type', None), conf.get('run_addr', None),</span><br><span style="color: hsl(120, 100%, 40%);">+ conf.get('ssh_user', None), conf.get('ssh_addr', None),</span><br><span style="color: hsl(120, 100%, 40%);">+ conf.get('run_label', None))</span><br><span> </span><br><span> def is_local(self):</span><br><span> return self._type == RunNode.T_LOCAL</span><br><span>@@ -77,4 +80,7 @@</span><br><span> def ssh_addr(self):</span><br><span> return self._ssh_addr</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def run_label(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._run_label</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # vim: expandtab tabstop=4 shiftwidth=4</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/18469">change 18469</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/18469"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: Iaf2e97da3aff693395f44f0e93b184d4846cf6da </div>
<div style="display:none"> Gerrit-Change-Number: 18469 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>