This is merely a historical archive of years 2008-2021, before the migration to mailman3.
A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.
pespin gerrit-no-reply at lists.osmocom.orgpespin has submitted this change. ( https://gerrit.osmocom.org/c/osmo-gsm-tester/+/18469 )
Change subject: Support identifying different tgz files based on run_label attribute
......................................................................
Support identifying different tgz files based on run_label attribute
tgz files in trials can be categorized in subdirectories, allowing to
select different bianry files at runtime based on the target run node
which is going to run them. This way for instance one can have a binary
linked against libs for eg. CentOS under run_label "centos/" or an ARM
target under "arm", and then use "run_label: arm" on the resource using
it.
Change-Id: Iaf2e97da3aff693395f44f0e93b184d4846cf6da
---
M .gitignore
M doc/manuals/chapters/trial.adoc
M selftest/Makefile
A selftest/trial_test/run_label/checksums.md5
A selftest/trial_test/run_label/foobar/sample.tar.gz
A selftest/trial_test/run_label/sample.tar.gz
M selftest/trial_test/trial_test.ok
M selftest/trial_test/trial_test.ok.ign
M selftest/trial_test/trial_test.py
M src/osmo_gsm_tester/core/trial.py
M src/osmo_gsm_tester/obj/run_node.py
11 files changed, 76 insertions(+), 21 deletions(-)
Approvals:
pespin: Looks good to me, approved
Jenkins Builder: Verified
diff --git a/.gitignore b/.gitignore
index ad9b423..ff09035 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,13 +4,16 @@
.version
_version.py
tags
-set_pythonpath
-test_work
state
*.pyc
sysmocom/resources.conf
sysmocom/ttcn3/resources.conf
+# selftest:
+set_pythonpath
+test_work
+selftest/trial_test/run_label/inst/
+
# manuals
doc/manuals/*.html
doc/manuals/*.svg
diff --git a/doc/manuals/chapters/trial.adoc b/doc/manuals/chapters/trial.adoc
index 4b52608..3ee6ae6 100644
--- a/doc/manuals/chapters/trial.adoc
+++ b/doc/manuals/chapters/trial.adoc
@@ -48,13 +48,22 @@
{app-name} tests create objects to manage the allocated resources during test
lifetime. These objects, in turn, usually run and manage processes started from
-the trail's sysroot binaries. {app-name} provide APIs for those object classes
+the trial's sysroot binaries. {app-name} provide APIs for those object classes
to discover, unpack and run those binaries. An object class simply needs to
request the name of the sysroot it wants to use (for instance 'osmo-bsc'), and
{app-name} will take care of preparing everything and providing the sysroot path
to it. It's a duty of the resource class to copy over the sysroot to the
destination if the intention is to run the binary remotely on another host.
+Moreover, {app-name} supports working with several different versions of a given
+sysroot by means of storing them in different subdirectories, which are later
+referenced by an object's class 'run_label' attribute named after that
+subdirectory. This way, for instance, a sysroot can be provided containing
+binaries linked against libraries present on a CentOS distribution, and other
+sysroots with the same name can also be provided which are linked against
+different versions of CentOS, or a different distro like Debian, or even a
+different arch like ARM.
+
When seeking a sysroot of a given name '<inst-name>' in the 'inst/' directory,
{app-name} will look for 'tgz' files starting with the pattern '<inst-name>.'
(up to the first dot). That means, suffixes are available for {app-name} user to
@@ -63,3 +72,6 @@
selected by {app-name} for 'osmo-bsc': 'osmo-bsc.tgz', 'osmo-bsc.build-23.tgz',
'osmo-bsc.5f3e0dd2.tgz', 'osmo-bsc.armv7.build-2.tgz'. If either none or more
than one valid file is found matching the pattern, an exception will be thrown.
+If a 'run_label=foobar' is provided, {app-name} will look up for the 'tgz' in
+exactly the same way, but this time in 'inst/foobar/' directory instead of
+'inst/'.
diff --git a/selftest/Makefile b/selftest/Makefile
index fb8618c..53d882c 100644
--- a/selftest/Makefile
+++ b/selftest/Makefile
@@ -12,6 +12,7 @@
clean:
@find . -name "*__pycache__" -type d -print0 | xargs -0 rm -rvf
@find . -name "*test_work" -type d -print0 | xargs -0 rm -rvf
+ @rm -rfv ./trial_test/run_label/inst
@rm -fv ./set_pythonpath
# vim: noexpandtab tabstop=8 shiftwidth=8
diff --git a/selftest/trial_test/run_label/checksums.md5 b/selftest/trial_test/run_label/checksums.md5
new file mode 100644
index 0000000..a162c56
--- /dev/null
+++ b/selftest/trial_test/run_label/checksums.md5
@@ -0,0 +1,2 @@
+b13c9c94d41c45caaa08ad9ad3ee0f11 sample.tar.gz
+430ec62329747b5f15fe6e2cb141b909 foobar/sample.tar.gz
diff --git a/selftest/trial_test/run_label/foobar/sample.tar.gz b/selftest/trial_test/run_label/foobar/sample.tar.gz
new file mode 100644
index 0000000..2c1b029
--- /dev/null
+++ b/selftest/trial_test/run_label/foobar/sample.tar.gz
Binary files differ
diff --git a/selftest/trial_test/run_label/sample.tar.gz b/selftest/trial_test/run_label/sample.tar.gz
new file mode 100644
index 0000000..3932964
--- /dev/null
+++ b/selftest/trial_test/run_label/sample.tar.gz
Binary files differ
diff --git a/selftest/trial_test/trial_test.ok b/selftest/trial_test/trial_test.ok
index 8c6a567..75bd1a7 100644
--- a/selftest/trial_test/trial_test.ok
+++ b/selftest/trial_test/trial_test.ok
@@ -14,3 +14,12 @@
ok, got RuntimeError: Checksum mismatch for '[PATH]/trial_test/invalid_checksum/file2' vs. '[PATH]/trial_test/invalid_checksum/checksums.md5' line 2
- detect missing file
ok, got RuntimeError: File listed in checksums file but missing in trials dir: '[PATH]/trial_test/missing_file/file2' vs. '[PATH]/trial_test/missing_file/checksums.md5' line 2
+- Verify trials based on run_label
+tst run_label: DBG: has bin_tar {bin_name='sample', matches=['sample.tar.gz'], run_label='foobar'}
+inst: [PATH]/trial_test/run_label/inst/foobar/sample
+content file2: subhello
+
+tst run_label: DBG: has bin_tar {bin_name='sample', matches=['sample.tar.gz'], run_label=''}
+inst: [PATH]/trial_test/run_label/inst/sample
+content file1: hello
+
diff --git a/selftest/trial_test/trial_test.ok.ign b/selftest/trial_test/trial_test.ok.ign
index 1a969de..46c3943 100644
--- a/selftest/trial_test/trial_test.ok.ign
+++ b/selftest/trial_test/trial_test.ok.ign
@@ -1,3 +1,3 @@
/tmp/[^/]* [TMP]
....-..-.._..-..-.. [TIMESTAMP]
-'[^']*/trial_test '[PATH]/trial_test
+/[^ ]*/trial_test/ [PATH]/trial_test/
diff --git a/selftest/trial_test/trial_test.py b/selftest/trial_test/trial_test.py
index a99428a..cf91a85 100755
--- a/selftest/trial_test/trial_test.py
+++ b/selftest/trial_test/trial_test.py
@@ -1,8 +1,10 @@
#!/usr/bin/env python3
-import time
import _prep
+
+import time
import os
+
from osmo_gsm_tester.core import util
from osmo_gsm_tester.core.trial import Trial
@@ -46,4 +48,17 @@
except RuntimeError as e:
print('ok, got RuntimeError: %s' % str(e))
+print('- Verify trials based on run_label')
+d = util.Dir('trial_test')
+t = Trial(d.child('run_label'))
+t.verify()
+inst = util.Dir(t.get_inst('sample', 'foobar'))
+print('inst: ' + str(inst))
+with open(inst.child('file2'), 'r') as f:
+ print('content file2: %s' % f.read())
+inst = util.Dir( t.get_inst('sample'))
+print('inst: ' + str(inst))
+with open(inst.child('file1'), 'r') as f:
+ print('content file1: %s' % f.read())
+
# vim: expandtab tabstop=4 shiftwidth=4
diff --git a/src/osmo_gsm_tester/core/trial.py b/src/osmo_gsm_tester/core/trial.py
index 83b0f11..eaf18c3 100644
--- a/src/osmo_gsm_tester/core/trial.py
+++ b/src/osmo_gsm_tester/core/trial.py
@@ -57,7 +57,7 @@
super().__init__(log.C_TST, os.path.basename(self.path))
self.dir = util.Dir(self.path)
self.inst_dir = util.Dir(self.dir.child('inst'))
- self.bin_tars = []
+ self.bin_tars = {}
self.suites = []
self.status = Trial.UNKNOWN
self._run_dir = None
@@ -125,10 +125,10 @@
line_nr += 1
if not line:
continue
- md5, filename = line.split(' ')
- file_path = self.dir.child(filename)
+ md5, relpath = line.split(' ')
+ file_path = self.dir.child(relpath)
- if not self.dir.isfile(filename):
+ if not self.dir.isfile(relpath):
raise RuntimeError('File listed in checksums file but missing in trials dir:'
' %r vs. %r line %d' % (file_path, checksums, line_nr))
@@ -136,28 +136,34 @@
raise RuntimeError('Checksum mismatch for %r vs. %r line %d'
% (file_path, checksums, line_nr))
- if filename.endswith('.tgz'):
- self.bin_tars.append(filename)
+ if relpath.endswith('.tgz') or relpath.endswith('.tar.gz'):
+ (label, name) = os.path.split(relpath)
+ #print('label: %s, name: %s' % (label, name))
+ li = self.bin_tars.get(label, [])
+ li.append(name)
+ self.bin_tars[label] = li
- def has_bin_tar(self, bin_name):
+ def has_bin_tar(self, bin_name, run_label):
bin_tar_start = '%s.' % bin_name
- matches = [t for t in self.bin_tars if t.startswith(bin_tar_start)]
- self.dbg(bin_name=bin_name, matches=matches)
+ matches = [t for t in self.bin_tars[run_label] if t.startswith(bin_tar_start)]
+ self.dbg('has bin_tar', run_label=run_label, bin_name=bin_name, matches=matches)
if not matches:
return None
if len(matches) > 1:
- raise RuntimeError('More than one match for bin name %r: %r' % (bin_name, matches))
+ raise RuntimeError('More than one match for bin name %r on run_label \'%s\': %r' % (bin_name, run_label, matches))
bin_tar = matches[0]
- bin_tar_path = self.dir.child(bin_tar)
+ bin_tar_path = self.dir.child(os.path.join(run_label, bin_tar))
if not os.path.isfile(bin_tar_path):
raise RuntimeError('Not a file or missing: %r' % bin_tar_path)
return bin_tar_path
- def get_inst(self, bin_name):
- bin_tar = self.has_bin_tar(bin_name)
+ def get_inst(self, bin_name, run_label=None):
+ if run_label is None:
+ run_label = ''
+ bin_tar = self.has_bin_tar(bin_name, run_label)
if not bin_tar:
raise RuntimeError('No such binary available: %r' % bin_name)
- inst_dir = self.inst_dir.child(bin_name)
+ inst_dir = self.inst_dir.child(os.path.join(run_label, bin_name))
if os.path.isdir(inst_dir):
# already unpacked
diff --git a/src/osmo_gsm_tester/obj/run_node.py b/src/osmo_gsm_tester/obj/run_node.py
index 26c85df..bd502af 100644
--- a/src/osmo_gsm_tester/obj/run_node.py
+++ b/src/osmo_gsm_tester/obj/run_node.py
@@ -26,6 +26,7 @@
'run_addr': schema.IPV4,
'ssh_user': schema.STR,
'ssh_addr': schema.IPV4,
+ 'run_label': schema.STR,
}
schema.register_resource_schema('run_node', resource_schema)
@@ -35,12 +36,13 @@
T_LOCAL = 'local'
T_REM_SSH = 'ssh'
- def __init__(self, type=None, run_addr=None, ssh_user=None, ssh_addr=None):
+ def __init__(self, type=None, run_addr=None, ssh_user=None, ssh_addr=None, run_label=None):
super().__init__(log.C_RUN, 'runnode')
self._type = type
self._run_addr = run_addr
self._ssh_user = ssh_user
self._ssh_addr = ssh_addr
+ self._run_label = run_label
if not self._type:
raise log.Error('run_type not set')
if not self._run_addr:
@@ -57,7 +59,9 @@
@classmethod
def from_conf(cls, conf):
- return cls(conf.get('run_type', None), conf.get('run_addr', None), conf.get('ssh_user', None), conf.get('ssh_addr', None))
+ return cls(conf.get('run_type', None), conf.get('run_addr', None),
+ conf.get('ssh_user', None), conf.get('ssh_addr', None),
+ conf.get('run_label', None))
def is_local(self):
return self._type == RunNode.T_LOCAL
@@ -77,4 +81,7 @@
def ssh_addr(self):
return self._ssh_addr
+ def run_label(self):
+ return self._run_label
+
# vim: expandtab tabstop=4 shiftwidth=4
--
To view, visit https://gerrit.osmocom.org/c/osmo-gsm-tester/+/18469
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: osmo-gsm-tester
Gerrit-Branch: master
Gerrit-Change-Id: Iaf2e97da3aff693395f44f0e93b184d4846cf6da
Gerrit-Change-Number: 18469
Gerrit-PatchSet: 4
Gerrit-Owner: pespin <pespin at sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: pespin <pespin at sysmocom.de>
Gerrit-MessageType: merged
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/gerrit-log/attachments/20200528/beeb577e/attachment.htm>