<p>Holger Freyther has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/13805">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">virtual: Have a single result class that can store data<br><br>We want to have LU, SMS and other tests run at the same time. Begin<br>by creating a single result where testcases can store additional data.<br><br>Move the stats code into the UL test case handling and out of the<br>suite.<br><br>Change-Id: Ie99351bee1515de8cf6870467f08256a53701907<br>---<br>M src/osmo_gsm_tester/ms_driver.py<br>M src/osmo_ms_driver/location_update_test.py<br>M src/osmo_ms_driver/test_support.py<br>M suites/nitb_netreg_mass/register_default_mass.py<br>4 files changed, 69 insertions(+), 37 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/05/13805/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/ms_driver.py b/src/osmo_gsm_tester/ms_driver.py</span><br><span>index 96b907a..3cfcad6 100644</span><br><span>--- a/src/osmo_gsm_tester/ms_driver.py</span><br><span>+++ b/src/osmo_gsm_tester/ms_driver.py</span><br><span>@@ -130,6 +130,12 @@</span><br><span> """</span><br><span> return self._test_case.get_result_values()</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def lus_less_than(self, acceptable_delay):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Returns the results that completed their LU within the acceptable delay.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._test_case.lus_less_than(acceptable_delay)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> def cleanup(self):</span><br><span> """</span><br><span> Cleans up the driver (e.g. AF_UNIX files).</span><br><span>diff --git a/src/osmo_ms_driver/location_update_test.py b/src/osmo_ms_driver/location_update_test.py</span><br><span>index 82c1cb3..6681d85 100644</span><br><span>--- a/src/osmo_ms_driver/location_update_test.py</span><br><span>+++ b/src/osmo_ms_driver/location_update_test.py</span><br><span>@@ -19,32 +19,39 @@</span><br><span> from copy import copy</span><br><span> from osmo_gsm_tester import log</span><br><span> from .starter import OsmoVirtPhy, OsmoMobile</span><br><span style="color: hsl(0, 100%, 40%);">-from .test_support import Results</span><br><span style="color: hsl(120, 100%, 40%);">+from .test_support import Result</span><br><span> </span><br><span> from datetime import timedelta</span><br><span> </span><br><span> import collections</span><br><span> import time</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-class LUResult(Results):</span><br><span style="color: hsl(0, 100%, 40%);">- """Representation of a Location Updating Result."""</span><br><span style="color: hsl(120, 100%, 40%);">+# Key used for the result dictionary</span><br><span style="color: hsl(120, 100%, 40%);">+LU_RESULT_NAME = 'lu_time'</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def __init__(self, name):</span><br><span style="color: hsl(0, 100%, 40%);">- super().__init__(name)</span><br><span style="color: hsl(0, 100%, 40%);">- self._time_of_lu = None</span><br><span style="color: hsl(120, 100%, 40%);">+def has_lu_time(result):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Returns true if a LU occurred.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ return result.has_result(LU_RESULT_NAME)</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def set_lu_time(self, time):</span><br><span style="color: hsl(0, 100%, 40%);">- assert self._time_of_lu is None</span><br><span style="color: hsl(0, 100%, 40%);">- self._time_of_lu = time</span><br><span style="color: hsl(120, 100%, 40%);">+def lu_time(result):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Returns the time of the LU occurrence.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ return result.get_result(LU_RESULT_NAME, default=0)</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def has_lu_time(self):</span><br><span style="color: hsl(0, 100%, 40%);">- return self._time_of_lu is not None</span><br><span style="color: hsl(120, 100%, 40%);">+def lu_delay(result):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Returns the delay from LU success to MS start time.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ return lu_time(result) - result.start_time()</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def lu_time(self):</span><br><span style="color: hsl(0, 100%, 40%);">- return self._time_of_lu or 0</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- def lu_delay(self):</span><br><span style="color: hsl(0, 100%, 40%);">- return self.lu_time() - self.start_time()</span><br><span style="color: hsl(120, 100%, 40%);">+def set_lu_time(result, time):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Sets/Overrides the time of the LU success for this MS.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ result.set_result(LU_RESULT_NAME, time)</span><br><span> </span><br><span> </span><br><span> LUStats = collections.namedtuple("LUStats", ["num_attempted", "num_completed",</span><br><span>@@ -106,7 +113,7 @@</span><br><span> self.TEMPLATE_CFG, self._subscribers[i],</span><br><span> phy.phy_filename(),</span><br><span> self._event_server.server_path())</span><br><span style="color: hsl(0, 100%, 40%);">- self._results[ms_name] = LUResult(ms_name)</span><br><span style="color: hsl(120, 100%, 40%);">+ self._results[ms_name] = Result(ms_name)</span><br><span> self._mobiles.append(launcher)</span><br><span> self._unstarted = copy(self._mobiles)</span><br><span> </span><br><span>@@ -204,10 +211,10 @@</span><br><span> elif data['type'] == 'event':</span><br><span> if data['data']['lu_done'] == 1:</span><br><span> ms = self._results[data['ms']]</span><br><span style="color: hsl(0, 100%, 40%);">- if not ms.has_lu_time():</span><br><span style="color: hsl(120, 100%, 40%);">+ if not has_lu_time(ms):</span><br><span> self._outstanding = self._outstanding - 1</span><br><span style="color: hsl(0, 100%, 40%);">- ms.set_lu_time(time)</span><br><span style="color: hsl(0, 100%, 40%);">- self.log("MS performed LU ", ms=ms, at=time, lu_delay=ms.lu_delay())</span><br><span style="color: hsl(120, 100%, 40%);">+ set_lu_time(ms, time)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log("MS performed LU ", ms=ms, at=time, lu_delay=lu_delay(ms))</span><br><span> else:</span><br><span> print(time, data)</span><br><span> raise Exception("Unknown event type..:" + _data.decode())</span><br><span>@@ -219,10 +226,10 @@</span><br><span> def find_min_max(self, results):</span><br><span> min_value = max_value = None</span><br><span> for result in results:</span><br><span style="color: hsl(0, 100%, 40%);">- if min_value is None or result.lu_delay() < min_value:</span><br><span style="color: hsl(0, 100%, 40%);">- min_value = result.lu_delay()</span><br><span style="color: hsl(0, 100%, 40%);">- if max_value is None or result.lu_delay() > max_value:</span><br><span style="color: hsl(0, 100%, 40%);">- max_value = result.lu_delay()</span><br><span style="color: hsl(120, 100%, 40%);">+ if min_value is None or lu_delay(result) < min_value:</span><br><span style="color: hsl(120, 100%, 40%);">+ min_value = lu_delay(result)</span><br><span style="color: hsl(120, 100%, 40%);">+ if max_value is None or lu_delay(result) > max_value:</span><br><span style="color: hsl(120, 100%, 40%);">+ max_value = lu_delay(result)</span><br><span> return min_value, max_value</span><br><span> </span><br><span> def get_result_values(self):</span><br><span>@@ -237,7 +244,7 @@</span><br><span> """</span><br><span> attempted = len(self._subscribers)</span><br><span> completed = attempted - self._outstanding</span><br><span style="color: hsl(0, 100%, 40%);">- min_latency, max_latency = self.find_min_max(filter(lambda x: x.has_lu_time(), self._results.values()))</span><br><span style="color: hsl(120, 100%, 40%);">+ min_latency, max_latency = self.find_min_max(filter(lambda x: has_lu_time(x), self._results.values()))</span><br><span> return LUStats(attempted, completed, min_latency, max_latency)</span><br><span> </span><br><span> def print_stats(self):</span><br><span>@@ -246,3 +253,16 @@</span><br><span> </span><br><span> self.log("Tests done", all_completed=all_completed,</span><br><span> min=stats.min_latency, max=stats.max_latency)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def lus_less_than(self, acceptable_delay):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Returns LUs that completed within the acceptable delay.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ res = []</span><br><span style="color: hsl(120, 100%, 40%);">+ for result in self._results.values():</span><br><span style="color: hsl(120, 100%, 40%);">+ if not has_lu_time(result):</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ if timedelta(seconds=lu_delay(result)) >= acceptable_delay:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ res.append(result)</span><br><span style="color: hsl(120, 100%, 40%);">+ return res</span><br><span>diff --git a/src/osmo_ms_driver/test_support.py b/src/osmo_ms_driver/test_support.py</span><br><span>index f1c34fb..0fd321c 100644</span><br><span>--- a/src/osmo_ms_driver/test_support.py</span><br><span>+++ b/src/osmo_ms_driver/test_support.py</span><br><span>@@ -26,15 +26,17 @@</span><br><span> yield ("%.15d" % n, "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00")</span><br><span> n += 1</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-class Results(log.Origin):</span><br><span style="color: hsl(120, 100%, 40%);">+class Result(log.Origin):</span><br><span> """</span><br><span style="color: hsl(0, 100%, 40%);">- A base class to collect results from tests.</span><br><span style="color: hsl(120, 100%, 40%);">+ The class for results. There should be one result class per test subject.</span><br><span style="color: hsl(120, 100%, 40%);">+ Specific tests can use add_result to add their outcome to this object.</span><br><span> """</span><br><span> </span><br><span> def __init__(self, name):</span><br><span> super().__init__(log.C_RUN, name)</span><br><span> self._time_of_registration = None</span><br><span> self._time_of_launch = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self._results = {}</span><br><span> </span><br><span> def set_start_time(self, time):</span><br><span> assert self._time_of_registration is None</span><br><span>@@ -49,3 +51,15 @@</span><br><span> </span><br><span> def launch_time(self):</span><br><span> return self._time_of_launch or 0</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def set_result(self, key, value):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Sets a result with the given key and value."""</span><br><span style="color: hsl(120, 100%, 40%);">+ self._results[key] = value</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_result(self, key, default=None):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Returns the result for the given key or default."""</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._results.get(key, default)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def has_result(self, key):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Returns true if there is a value for the key."""</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._results.get(key) is not None</span><br><span>diff --git a/suites/nitb_netreg_mass/register_default_mass.py b/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>index 306eb81..f4e5e80 100644</span><br><span>--- a/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>+++ b/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>@@ -46,15 +46,7 @@</span><br><span> </span><br><span> # Check how many results are below our threshold.</span><br><span> acceptable_delay = timedelta(seconds=30)</span><br><span style="color: hsl(0, 100%, 40%);">-results = ms_driver.get_result_values()</span><br><span style="color: hsl(0, 100%, 40%);">-quick_enough = 0</span><br><span style="color: hsl(0, 100%, 40%);">-for result in results:</span><br><span style="color: hsl(0, 100%, 40%);">- if not result.has_lu_time():</span><br><span style="color: hsl(0, 100%, 40%);">- continue</span><br><span style="color: hsl(0, 100%, 40%);">- if timedelta(seconds=result.lu_delay()) >= acceptable_delay:</span><br><span style="color: hsl(0, 100%, 40%);">- continue</span><br><span style="color: hsl(0, 100%, 40%);">- quick_enough = quick_enough + 1</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-latency_ratio = quick_enough / len(results)</span><br><span style="color: hsl(120, 100%, 40%);">+quick_enough = len(ms_driver.lus_less_than(acceptable_delay))</span><br><span style="color: hsl(120, 100%, 40%);">+latency_ratio = quick_enough / stats.num_attempted</span><br><span> if latency_ratio < 0.99:</span><br><span> raise Exception("Latency ratio of %f%% lower than threshold." % (latency_ratio * 100.0))</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/13805">change 13805</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/13805"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: Ie99351bee1515de8cf6870467f08256a53701907 </div>
<div style="display:none"> Gerrit-Change-Number: 13805 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Holger Freyther <holger@freyther.de> </div>