<p>Holger Freyther <strong>merged</strong> this change.</p><p><a href="https://gerrit.osmocom.org/13124">View Change</a></p><div style="white-space:pre-wrap">Approvals:
Holger Freyther: Looks good to me, approved
Jenkins Builder: Verified
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">Revert "nitb_netreg_mass: Add code to declare a run successful/failure"<br><br>This reverts commit b4ad8d72431df2336b9c7fbc6eb9597daa80a35c.<br><br>Reason for revert: There was a true dependency on earlier changes (len(mobiles)). Rollback until the resource configuration is merged.<br><br>Change-Id: If0f5e24cec2f0d1d144b464180b341cacbdf0ebb<br>---<br>M src/osmo_gsm_tester/ms_driver.py<br>M src/osmo_ms_driver/location_update_test.py<br>M suites/nitb_netreg_mass/register_default_mass.py<br>3 files changed, 7 insertions(+), 71 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/ms_driver.py b/src/osmo_gsm_tester/ms_driver.py</span><br><span>index de2f967..e8e543c 100644</span><br><span>--- a/src/osmo_gsm_tester/ms_driver.py</span><br><span>+++ b/src/osmo_gsm_tester/ms_driver.py</span><br><span>@@ -146,18 +146,6 @@</span><br><span> """</span><br><span> self._test_case.print_stats()</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def get_stats(self):</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- Returns a statistical summary of the test.</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- return self._test_case.get_stats()</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- def get_result_values(self):</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- Returns the raw result values of the test run in any order.</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- return self._test_case.get_result_values()</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> def cleanup(self):</span><br><span> """</span><br><span> Cleans up the driver (e.g. AF_UNIX files).</span><br><span>diff --git a/src/osmo_ms_driver/location_update_test.py b/src/osmo_ms_driver/location_update_test.py</span><br><span>index f14bd28..8500607 100644</span><br><span>--- a/src/osmo_ms_driver/location_update_test.py</span><br><span>+++ b/src/osmo_ms_driver/location_update_test.py</span><br><span>@@ -23,11 +23,9 @@</span><br><span> </span><br><span> from datetime import timedelta</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-import collections</span><br><span> import time</span><br><span> </span><br><span> class LUResult(Results):</span><br><span style="color: hsl(0, 100%, 40%);">- """Representation of a Location Updating Result."""</span><br><span> </span><br><span> def __init__(self, name):</span><br><span> super().__init__(name)</span><br><span>@@ -46,10 +44,6 @@</span><br><span> def lu_delay(self):</span><br><span> return self.lu_time() - self.start_time()</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-LUStats = collections.namedtuple("LUStats", ["num_attempted", "num_completed",</span><br><span style="color: hsl(0, 100%, 40%);">- "min_latency", "max_latency"])</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> class MassUpdateLocationTest(log.Origin):</span><br><span> """</span><br><span> A test to launch a configurable amount of MS and make them</span><br><span>@@ -213,24 +207,9 @@</span><br><span> max_value = result.lu_delay()</span><br><span> return min_value, max_value</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def get_result_values(self):</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- Returns the raw result values of the test run in any order.</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- return self._results.values()</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- def get_stats(self):</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- Returns a statistical summary of the test.</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- attempted = self._number_of_ms</span><br><span style="color: hsl(0, 100%, 40%);">- completed = attempted - self._outstanding</span><br><span style="color: hsl(0, 100%, 40%);">- min_latency, max_latency = self.find_min_max(filter(lambda x: x.has_lu_time(), self._results.values()))</span><br><span style="color: hsl(0, 100%, 40%);">- return LUStats(attempted, completed, min_latency, max_latency)</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> def print_stats(self):</span><br><span style="color: hsl(0, 100%, 40%);">- stats = self.get_stats()</span><br><span style="color: hsl(0, 100%, 40%);">- all_completed = stats.num_attempted == stats.num_completed</span><br><span style="color: hsl(120, 100%, 40%);">+ all_completed = self.all_completed()</span><br><span style="color: hsl(120, 100%, 40%);">+ min_value, max_value = self.find_min_max(filter(lambda x: x.has_lu_time(), self._results.values()))</span><br><span> </span><br><span> self.log("Tests done", all_completed=all_completed,</span><br><span style="color: hsl(0, 100%, 40%);">- min=stats.min_latency, max=stats.max_latency)</span><br><span style="color: hsl(120, 100%, 40%);">+ min=min_value, max=max_value)</span><br><span>diff --git a/suites/nitb_netreg_mass/register_default_mass.py b/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>index 8259359..ede2b74 100644</span><br><span>--- a/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>+++ b/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>@@ -1,17 +1,16 @@</span><br><span> #!/usr/bin/env python3</span><br><span> """</span><br><span style="color: hsl(0, 100%, 40%);">-Runs a network registration with a 'massive' amount of MS</span><br><span style="color: hsl(120, 100%, 40%);">+Run a network registration with a 'massive' amount of MS</span><br><span> using the ms_driver infrastructure.</span><br><span> """</span><br><span> from osmo_gsm_tester.testenv import *</span><br><span style="color: hsl(0, 100%, 40%);">-from datetime import timedelta</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-print('Claiming resources for the test')</span><br><span style="color: hsl(120, 100%, 40%);">+print('use resources...')</span><br><span> nitb = suite.nitb()</span><br><span> bts = suite.bts()</span><br><span> ms_driver = suite.ms_driver()</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-print('Launching a simple network')</span><br><span style="color: hsl(120, 100%, 40%);">+print('start nitb and bts...')</span><br><span> nitb.bts_add(bts)</span><br><span> nitb.start()</span><br><span> bts.start()</span><br><span>@@ -24,35 +23,5 @@</span><br><span> # Run the base test.</span><br><span> ms_driver.run_test()</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-# Print the stats of the run.</span><br><span style="color: hsl(120, 100%, 40%);">+# Print stats</span><br><span> ms_driver.print_stats()</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-# Evaluate if this run was successful or not. Our initial acceptance criteria</span><br><span style="color: hsl(0, 100%, 40%);">-# is quite basic but it should allow us to scale to a larger number of MS and</span><br><span style="color: hsl(0, 100%, 40%);">-# reasons (e.g. have a full BCCH).</span><br><span style="color: hsl(0, 100%, 40%);">-#</span><br><span style="color: hsl(0, 100%, 40%);">-# 99% of LUs should complete</span><br><span style="color: hsl(0, 100%, 40%);">-# 99% of successful LUs should complete within 10s.</span><br><span style="color: hsl(0, 100%, 40%);">-stats = ms_driver.get_stats()</span><br><span style="color: hsl(0, 100%, 40%);">-if len(mobiles) > 0 and stats.num_completed < 1:</span><br><span style="color: hsl(0, 100%, 40%);">- raise Exception("No run completed.")</span><br><span style="color: hsl(0, 100%, 40%);">-completion_ratio = stats.num_attempted / stats.num_completed</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-# Verify that 99% of LUs completed.</span><br><span style="color: hsl(0, 100%, 40%);">-if completion_ratio < 0.99:</span><br><span style="color: hsl(0, 100%, 40%);">- raise Exception("Completion ratio of %f%% lower than threshold." % (completion_ratio * 100.0))</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-# Check how many results are below our threshold.</span><br><span style="color: hsl(0, 100%, 40%);">-acceptable_delay = timedelta(seconds=20)</span><br><span style="color: hsl(0, 100%, 40%);">-results = ms_driver.get_result_values()</span><br><span style="color: hsl(0, 100%, 40%);">-quick_enough = 0</span><br><span style="color: hsl(0, 100%, 40%);">-for result in results:</span><br><span style="color: hsl(0, 100%, 40%);">- if not result.has_lu_time():</span><br><span style="color: hsl(0, 100%, 40%);">- continue</span><br><span style="color: hsl(0, 100%, 40%);">- if timedelta(seconds=result.lu_delay()) >= acceptable_delay:</span><br><span style="color: hsl(0, 100%, 40%);">- continue</span><br><span style="color: hsl(0, 100%, 40%);">- quick_enough = quick_enough + 1</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-latency_ratio = quick_enough / len(results)</span><br><span style="color: hsl(0, 100%, 40%);">-if latency_ratio < 0.99:</span><br><span style="color: hsl(0, 100%, 40%);">- raise Exception("Latency ratio of %f%% lower than threshold." % (latency_ratio * 100.0))</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/13124">change 13124</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/13124"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: merged </div>
<div style="display:none"> Gerrit-Change-Id: If0f5e24cec2f0d1d144b464180b341cacbdf0ebb </div>
<div style="display:none"> Gerrit-Change-Number: 13124 </div>
<div style="display:none"> Gerrit-PatchSet: 2 </div>
<div style="display:none"> Gerrit-Owner: Holger Freyther <holger@freyther.de> </div>
<div style="display:none"> Gerrit-Reviewer: Holger Freyther <holger@freyther.de> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins Builder (1000002) </div>
<div style="display:none"> Gerrit-Reviewer: Pau Espin Pedrol <pespin@sysmocom.de> </div>