<p>pespin has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/17291">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">Introduce parametrized scenario files support<br><br>The idea is to have something similar to systemd template unit files:<br>https://fedoramagazine.org/systemd-template-unit-files/<br><br>Specially for modifiers, one finds the situation where same scenario structure<br>has to be created with lots of different values.<br>For instance, let's say we want to test with different eNodeB num_prb values:<br>[6, 15, 25, 50, 75,100]<br>Right now we'd need to create one scenario file for each of them, for instance:<br>mod-enb-nprb6.conf<br>mod-enb-nprb15.conf<br>mod-enb-nprb25.conf<br>mod-enb-nprb50.conf<br>mod-enb-nprb75.conf<br>mod-enb-nprb100.conf<br><br>And each of them containing something like (changing the num_prb value):<br>"""<br>modifiers:<br> enb:<br> - num_prb: 75<br>"""<br><br>Instead, we can now have one unique file mod-enb-nprb@.conf:<br>"""<br>modifiers:<br> enb:<br> - num_prb: ${param1}<br>"""<br>The general syntax is: "scenario-name@param1,param2,param3".<br>So "@" splits between scenario name and parameter list, and "," splits<br>between parameters.<br><br>For instance, one can now run following suite with scenario:<br>"4g:srsenb-rftype-uhd+srsue-rftype-uhd+mod-enb-nprb@75"<br><br>Related: OS#4424<br>Change-Id: Icfcba15b937225aa4b1f322a8005fcd57db1d1ca<br>---<br>M src/osmo_gsm_tester/config.py<br>M src/osmo_gsm_tester/template.py<br>2 files changed, 47 insertions(+), 7 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/91/17291/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/config.py b/src/osmo_gsm_tester/config.py</span><br><span>index 87b3da4..c9b94c5 100644</span><br><span>--- a/src/osmo_gsm_tester/config.py</span><br><span>+++ b/src/osmo_gsm_tester/config.py</span><br><span>@@ -54,7 +54,7 @@</span><br><span> import os</span><br><span> import copy</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-from . import log, schema, util</span><br><span style="color: hsl(120, 100%, 40%);">+from . import log, schema, util, template</span><br><span> from .util import is_dict, is_list, Dir, get_tempdir</span><br><span> </span><br><span> ENV_PREFIX = 'OSMO_GSM_TESTER_'</span><br><span>@@ -193,19 +193,53 @@</span><br><span> return defaults.get(for_kind, {})</span><br><span> </span><br><span> class Scenario(log.Origin, dict):</span><br><span style="color: hsl(0, 100%, 40%);">- def __init__(self, name, path):</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, name, path, param_list):</span><br><span> super().__init__(log.C_TST, name)</span><br><span> self.path = path</span><br><span style="color: hsl(120, 100%, 40%);">+ self.param_list = param_list</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def read_from_file(self, validation_schema):</span><br><span style="color: hsl(120, 100%, 40%);">+ with open(self.path, 'r') as f:</span><br><span style="color: hsl(120, 100%, 40%);">+ config_str = f.read()</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(self.param_list) != 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ param_dict = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ i = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ for param in self.param_list:</span><br><span style="color: hsl(120, 100%, 40%);">+ param_dict['param' + str(i)] = param</span><br><span style="color: hsl(120, 100%, 40%);">+ i += 1</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg(param_dict=param_dict)</span><br><span style="color: hsl(120, 100%, 40%);">+ config_str = template.render_strbuf_inline(config_str, param_dict)</span><br><span style="color: hsl(120, 100%, 40%);">+ config = yaml.safe_load(config_str)</span><br><span style="color: hsl(120, 100%, 40%);">+ config = _standardize(config)</span><br><span style="color: hsl(120, 100%, 40%);">+ if validation_schema:</span><br><span style="color: hsl(120, 100%, 40%);">+ schema.validate(config, validation_schema)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.update(config)</span><br><span> </span><br><span> def get_scenario(name, validation_schema=None):</span><br><span> scenarios_dir = get_scenarios_dir()</span><br><span> if not name.endswith('.conf'):</span><br><span> name = name + '.conf'</span><br><span style="color: hsl(120, 100%, 40%);">+ is_parametrized_file = '@' in name</span><br><span style="color: hsl(120, 100%, 40%);">+ param_list = []</span><br><span> path = scenarios_dir.child(name)</span><br><span style="color: hsl(0, 100%, 40%);">- if not os.path.isfile(path):</span><br><span style="color: hsl(0, 100%, 40%);">- raise RuntimeError('No such scenario file: %r' % path)</span><br><span style="color: hsl(0, 100%, 40%);">- sc = Scenario(name, path)</span><br><span style="color: hsl(0, 100%, 40%);">- sc.update(read(path, validation_schema=validation_schema))</span><br><span style="color: hsl(120, 100%, 40%);">+ if not is_parametrized_file:</span><br><span style="color: hsl(120, 100%, 40%);">+ if not os.path.isfile(path):</span><br><span style="color: hsl(120, 100%, 40%);">+ raise RuntimeError('No such scenario file: %r' % path)</span><br><span style="color: hsl(120, 100%, 40%);">+ else: # parametrized scenario:</span><br><span style="color: hsl(120, 100%, 40%);">+ # Allow first matching complete matching names (eg: scenario@param1,param2.conf),</span><br><span style="color: hsl(120, 100%, 40%);">+ # this allows setting specific content in different files for specific values.</span><br><span style="color: hsl(120, 100%, 40%);">+ if not os.path.isfile(path):</span><br><span style="color: hsl(120, 100%, 40%);">+ # get "scenario@.conf" from "scenario@param1,param2.conf":</span><br><span style="color: hsl(120, 100%, 40%);">+ prefix_name = name[:name.index("@")+1] + '.conf'</span><br><span style="color: hsl(120, 100%, 40%);">+ path = scenarios_dir.child(prefix_name)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not os.path.isfile(path):</span><br><span style="color: hsl(120, 100%, 40%);">+ raise RuntimeError('No such scenario file: %r (nor %s)' % (path, name))</span><br><span style="color: hsl(120, 100%, 40%);">+ # At this point, we have existing file path. Let's now scrap the parameter(s):</span><br><span style="color: hsl(120, 100%, 40%);">+ # get param1,param2 str from scenario@param1,param2.conf</span><br><span style="color: hsl(120, 100%, 40%);">+ param_list_str = name.split('@', 1)[1][:-len('.conf')]</span><br><span style="color: hsl(120, 100%, 40%);">+ param_list = param_list_str.split(',')</span><br><span style="color: hsl(120, 100%, 40%);">+ sc = Scenario(name, path, param_list)</span><br><span style="color: hsl(120, 100%, 40%);">+ sc.read_from_file(validation_schema)</span><br><span> return sc</span><br><span> </span><br><span> def add(dest, src):</span><br><span>diff --git a/src/osmo_gsm_tester/template.py b/src/osmo_gsm_tester/template.py</span><br><span>index 6ce7ebd..dac89f2 100644</span><br><span>--- a/src/osmo_gsm_tester/template.py</span><br><span>+++ b/src/osmo_gsm_tester/template.py</span><br><span>@@ -19,7 +19,7 @@</span><br><span> # along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span> </span><br><span> import os</span><br><span style="color: hsl(0, 100%, 40%);">-from mako.lookup import TemplateLookup</span><br><span style="color: hsl(120, 100%, 40%);">+from mako.lookup import TemplateLookup, Template</span><br><span> </span><br><span> from . import log</span><br><span> from .util import dict2obj</span><br><span>@@ -53,4 +53,10 @@</span><br><span> </span><br><span> return template.render(**dict2obj(values))</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+def render_strbuf_inline(strbuf, values):</span><br><span style="color: hsl(120, 100%, 40%);">+ '''Receive a string containing template syntax, and generate output using</span><br><span style="color: hsl(120, 100%, 40%);">+ passed values.'''</span><br><span style="color: hsl(120, 100%, 40%);">+ mytemplate = Template(strbuf)</span><br><span style="color: hsl(120, 100%, 40%);">+ return mytemplate.render(**dict2obj(values))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # vim: expandtab tabstop=4 shiftwidth=4</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/17291">change 17291</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/17291"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: Icfcba15b937225aa4b1f322a8005fcd57db1d1ca </div>
<div style="display:none"> Gerrit-Change-Number: 17291 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>