<p>Holger Freyther has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/13042">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">nitb_netreg_mass: Allow to define subscribers from the outside<br><br>Remove hardcoded configuration and replace it with a reservation in<br>the suite. In the future this can be used to specify MSISDN and KI<br>as well. We can now scale the subscriber count to something beyond<br>one can easily test by hand.<br><br>Change-Id: Ic9fd560c2924731e4fd6eea5aaf3ad565cb4ef52<br>---<br>M src/osmo_gsm_tester/ms_driver.py<br>M src/osmo_gsm_tester/suite.py<br>M src/osmo_ms_driver/__main__.py<br>M src/osmo_ms_driver/location_update_test.py<br>M src/osmo_ms_driver/starter.py<br>M suites/nitb_netreg_mass/register_default_mass.py<br>6 files changed, 77 insertions(+), 51 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/42/13042/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/ms_driver.py b/src/osmo_gsm_tester/ms_driver.py</span><br><span>index 6292d7d..c7f2540 100644</span><br><span>--- a/src/osmo_gsm_tester/ms_driver.py</span><br><span>+++ b/src/osmo_gsm_tester/ms_driver.py</span><br><span>@@ -21,7 +21,7 @@</span><br><span> from osmo_ms_driver.event_server import EventServer</span><br><span> from osmo_ms_driver.simple_loop import SimpleLoop</span><br><span> from osmo_ms_driver.location_update_test import MassUpdateLocationTest</span><br><span style="color: hsl(0, 100%, 40%);">-from osmo_ms_driver.starter import BinaryOptions, Subscriber</span><br><span style="color: hsl(120, 100%, 40%);">+from osmo_ms_driver.starter import BinaryOptions</span><br><span> </span><br><span> import os.path</span><br><span> import shutil</span><br><span>@@ -34,7 +34,6 @@</span><br><span> self._suite_run = suite_run</span><br><span> </span><br><span> # TODO: take config out of the test scenario</span><br><span style="color: hsl(0, 100%, 40%);">- self._num_ms = 10</span><br><span> self._time_start = timedelta(seconds=60)</span><br><span> self._time_step = timedelta(milliseconds=100)</span><br><span> self._test_duration = timedelta(seconds=120)</span><br><span>@@ -42,6 +41,7 @@</span><br><span> self._loop = SimpleLoop()</span><br><span> self._test_case = None</span><br><span> self.event_server_sk_tmp_dir = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self._subscribers = []</span><br><span> </span><br><span> if len(self.event_server_path().encode()) > 107:</span><br><span> raise log.Error('Path for event_server socket is longer than max allowed len for unix socket path (107):', self.event_server_path())</span><br><span>@@ -78,6 +78,10 @@</span><br><span> mobile = check_and_return_binary('mobile')</span><br><span> return BinaryOptions(virtphy, mobile, env)</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def subscriber_add(self, subscriber):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Adds a subscriber to the list of subscribers."""</span><br><span style="color: hsl(120, 100%, 40%);">+ self._subscribers.append(subscriber)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> def configure(self):</span><br><span> """</span><br><span> Configures the subscribers, tests and registration server. Needs to be</span><br><span>@@ -88,24 +92,14 @@</span><br><span> self._ev_server = EventServer("ev_server", event_server_path)</span><br><span> self._ev_server.listen(self._loop)</span><br><span> options = self.build_binary_options()</span><br><span style="color: hsl(0, 100%, 40%);">- self._test_case = MassUpdateLocationTest("mass", options, self._num_ms, self._cdf,</span><br><span style="color: hsl(120, 100%, 40%);">+ self._test_case = MassUpdateLocationTest("mass", options, self._cdf,</span><br><span> self._ev_server,</span><br><span> util.Dir(self.event_server_sk_tmp_dir),</span><br><span> suite_run=self._suite_run)</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- # TODO: We should pass subscribers down to the test and not get it from</span><br><span style="color: hsl(0, 100%, 40%);">- # there.</span><br><span style="color: hsl(0, 100%, 40%);">- self._subs = [Subscriber(imsi=mob.imsi(), ki=mob.ki()) for mob in self._test_case.mobiles()]</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- def ms_subscribers(self):</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- Returns a list of 'subscribers' that were configured in the</span><br><span style="color: hsl(0, 100%, 40%);">- current scenario.</span><br><span style="color: hsl(0, 100%, 40%);">- """</span><br><span style="color: hsl(0, 100%, 40%);">- if not hasattr(self, '_subs'):</span><br><span style="color: hsl(0, 100%, 40%);">- self.configure()</span><br><span style="color: hsl(0, 100%, 40%);">- return self._subs</span><br><span style="color: hsl(120, 100%, 40%);">+ for sub in self._subscribers:</span><br><span style="color: hsl(120, 100%, 40%);">+ self._test_case.subscriber_add(sub)</span><br><span style="color: hsl(120, 100%, 40%);">+ self._configured = True</span><br><span> </span><br><span> def run_test(self):</span><br><span> """</span><br><span>@@ -113,7 +107,7 @@</span><br><span> devices according to their schedule. Returns once all tests succeeded</span><br><span> or the configured timeout has passed.</span><br><span> """</span><br><span style="color: hsl(0, 100%, 40%);">- if not hasattr(self, '_subs'):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not hasattr(self, '_configured'):</span><br><span> self.configure()</span><br><span> self._test_case.run_test(self._loop, self._test_duration)</span><br><span> </span><br><span>diff --git a/src/osmo_gsm_tester/suite.py b/src/osmo_gsm_tester/suite.py</span><br><span>index 71e7c04..14a205d 100644</span><br><span>--- a/src/osmo_gsm_tester/suite.py</span><br><span>+++ b/src/osmo_gsm_tester/suite.py</span><br><span>@@ -25,8 +25,7 @@</span><br><span> from . import config, log, util, resource, test</span><br><span> from .event_loop import MainLoop</span><br><span> from . import osmo_nitb, osmo_hlr, osmo_mgcpgw, osmo_mgw, osmo_msc, osmo_bsc, osmo_stp, osmo_ggsn, osmo_sgsn, modem, esme, osmocon, ms_driver, iperf3</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-Mobile = collections.namedtuple("Mobile", ["imsi"])</span><br><span style="color: hsl(120, 100%, 40%);">+from osmo_ms_driver.starter import Subscriber</span><br><span> </span><br><span> class Timeout(Exception):</span><br><span> pass</span><br><span>@@ -321,7 +320,18 @@</span><br><span> </span><br><span> def mobile(self):</span><br><span> mobile_cfg = self.reserved_resources.get(resource.R_MOBILE)</span><br><span style="color: hsl(0, 100%, 40%);">- return Mobile(mobile_cfg['imsi'])</span><br><span style="color: hsl(120, 100%, 40%);">+ # TODO(zecke): Pick a random KI and convert to hex.</span><br><span style="color: hsl(120, 100%, 40%);">+ return Subscriber(mobile_cfg['imsi'],</span><br><span style="color: hsl(120, 100%, 40%);">+ "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def all_mobiles(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Returns all mobiles reserved."""</span><br><span style="color: hsl(120, 100%, 40%);">+ mobiles = []</span><br><span style="color: hsl(120, 100%, 40%);">+ while True:</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ mobiles.append(self.mobile())</span><br><span style="color: hsl(120, 100%, 40%);">+ except resource.NoResourceExn:</span><br><span style="color: hsl(120, 100%, 40%);">+ return mobiles</span><br><span> </span><br><span> def bts(self, specifics=None):</span><br><span> bts = bts_obj(self, self.reserved_resources.get(resource.R_BTS, specifics=specifics))</span><br><span>diff --git a/src/osmo_ms_driver/__main__.py b/src/osmo_ms_driver/__main__.py</span><br><span>index 7e1afa8..bbade48 100644</span><br><span>--- a/src/osmo_ms_driver/__main__.py</span><br><span>+++ b/src/osmo_ms_driver/__main__.py</span><br><span>@@ -20,7 +20,8 @@</span><br><span> from .simple_loop import SimpleLoop</span><br><span> from .location_update_test import MassUpdateLocationTest</span><br><span> from .cdf import cdfs</span><br><span style="color: hsl(0, 100%, 40%);">-from .starter import BinaryOptions</span><br><span style="color: hsl(120, 100%, 40%);">+from .starter import BinaryOptions, Subscriber</span><br><span style="color: hsl(120, 100%, 40%);">+from .test_support import imsi_ki_gen</span><br><span> from osmo_gsm_tester import log, util</span><br><span> </span><br><span> # System modules</span><br><span>@@ -84,7 +85,14 @@</span><br><span> </span><br><span> # Just a single test for now.</span><br><span> options = BinaryOptions("virtphy", "mobile", os.environ)</span><br><span style="color: hsl(0, 100%, 40%);">- test = MassUpdateLocationTest("lu_test", options, args.num_ms, cdf, ev_server, tmp_dir)</span><br><span style="color: hsl(120, 100%, 40%);">+ test = MassUpdateLocationTest("lu_test", options, cdf, ev_server, tmp_dir)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Add subscribers to the test.</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi_gen = imsi_ki_gen()</span><br><span style="color: hsl(120, 100%, 40%);">+ for i in range(0, args.num_ms):</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi, ki = next(imsi_gen)</span><br><span style="color: hsl(120, 100%, 40%);">+ test.subscriber_add(Subscriber(imsi, ki))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> atexit.register(test.stop_all)</span><br><span> </span><br><span> # Run until everything has been launched</span><br><span>diff --git a/src/osmo_ms_driver/location_update_test.py b/src/osmo_ms_driver/location_update_test.py</span><br><span>index f14bd28..5ff2199 100644</span><br><span>--- a/src/osmo_ms_driver/location_update_test.py</span><br><span>+++ b/src/osmo_ms_driver/location_update_test.py</span><br><span>@@ -19,7 +19,7 @@</span><br><span> from copy import copy</span><br><span> from osmo_gsm_tester import log</span><br><span> from .starter import OsmoVirtPhy, OsmoMobile</span><br><span style="color: hsl(0, 100%, 40%);">-from .test_support import imsi_ki_gen, Results</span><br><span style="color: hsl(120, 100%, 40%);">+from .test_support import Results</span><br><span> </span><br><span> from datetime import timedelta</span><br><span> </span><br><span>@@ -62,42 +62,53 @@</span><br><span> TEMPLATE_LUA = "osmo-mobile-lu.lua"</span><br><span> TEMPLATE_CFG = "osmo-mobile.cfg"</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def __init__(self, name, options, number_of_ms, cdf_function,</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, name, options, cdf_function,</span><br><span> event_server, tmp_dir, suite_run=None):</span><br><span> super().__init__(log.C_RUN, name)</span><br><span> self._binary_options = options</span><br><span style="color: hsl(0, 100%, 40%);">- self._number_of_ms = number_of_ms</span><br><span> self._cdf = cdf_function</span><br><span style="color: hsl(0, 100%, 40%);">- self._cdf.set_target(number_of_ms)</span><br><span> self._suite_run = suite_run</span><br><span style="color: hsl(120, 100%, 40%);">+ self._tmp_dir = tmp_dir</span><br><span> self._unstarted = []</span><br><span> self._mobiles = []</span><br><span> self._phys = []</span><br><span> self._results = {}</span><br><span style="color: hsl(0, 100%, 40%);">- imsi_gen = imsi_ki_gen()</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- self._outstanding = number_of_ms</span><br><span style="color: hsl(0, 100%, 40%);">- for i in range(0, number_of_ms):</span><br><span style="color: hsl(0, 100%, 40%);">- ms_name = "%.5d" % i</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- phy = OsmoVirtPhy(options.virtphy, options.env,</span><br><span style="color: hsl(0, 100%, 40%);">- ms_name, tmp_dir)</span><br><span style="color: hsl(0, 100%, 40%);">- self._phys.append(phy)</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- launcher = OsmoMobile(options.mobile, options.env,</span><br><span style="color: hsl(0, 100%, 40%);">- ms_name, tmp_dir, self.TEMPLATE_LUA,</span><br><span style="color: hsl(0, 100%, 40%);">- self.TEMPLATE_CFG, imsi_gen,</span><br><span style="color: hsl(0, 100%, 40%);">- phy.phy_filename(),</span><br><span style="color: hsl(0, 100%, 40%);">- event_server.server_path())</span><br><span style="color: hsl(0, 100%, 40%);">- self._results[ms_name] = LUResult(ms_name)</span><br><span style="color: hsl(0, 100%, 40%);">- self._mobiles.append(launcher)</span><br><span> self._event_server = event_server</span><br><span> self._event_server.register(self.handle_msg)</span><br><span style="color: hsl(0, 100%, 40%);">- self._unstarted = copy(self._mobiles)</span><br><span> self._started = []</span><br><span style="color: hsl(120, 100%, 40%);">+ self._subscribers = []</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def mobiles(self):</span><br><span style="color: hsl(0, 100%, 40%);">- return self._mobiles</span><br><span style="color: hsl(120, 100%, 40%);">+ def subscriber_add(self, subscriber):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Adds a subscriber to the list of subscribers.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ Must be called before starting the testcase.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ self._subscribers.append(subscriber)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def configure_tasks(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Sets up the test run."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self._cdf.set_target(len(self._subscribers))</span><br><span style="color: hsl(120, 100%, 40%);">+ self._outstanding = len(self._subscribers)</span><br><span style="color: hsl(120, 100%, 40%);">+ for i in range(0, self._outstanding):</span><br><span style="color: hsl(120, 100%, 40%);">+ ms_name = "%.5d" % i</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ phy = OsmoVirtPhy(self._binary_options.virtphy,</span><br><span style="color: hsl(120, 100%, 40%);">+ self._binary_options.env,</span><br><span style="color: hsl(120, 100%, 40%);">+ ms_name, self._tmp_dir)</span><br><span style="color: hsl(120, 100%, 40%);">+ self._phys.append(phy)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ launcher = OsmoMobile(self._binary_options.mobile,</span><br><span style="color: hsl(120, 100%, 40%);">+ self._binary_options.env,</span><br><span style="color: hsl(120, 100%, 40%);">+ ms_name, self._tmp_dir, self.TEMPLATE_LUA,</span><br><span style="color: hsl(120, 100%, 40%);">+ self.TEMPLATE_CFG, self._subscribers[i],</span><br><span style="color: hsl(120, 100%, 40%);">+ phy.phy_filename(),</span><br><span style="color: hsl(120, 100%, 40%);">+ self._event_server.server_path())</span><br><span style="color: hsl(120, 100%, 40%);">+ self._results[ms_name] = LUResult(ms_name)</span><br><span style="color: hsl(120, 100%, 40%);">+ self._mobiles.append(launcher)</span><br><span style="color: hsl(120, 100%, 40%);">+ self._unstarted = copy(self._mobiles)</span><br><span> </span><br><span> def pre_launch(self, loop):</span><br><span> """</span><br><span>@@ -117,6 +128,7 @@</span><br><span> def prepare(self, loop):</span><br><span> self.log("Starting testcase")</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ self.configure_tasks()</span><br><span> self.pre_launch(loop)</span><br><span> </span><br><span> self._start_time = time.clock_gettime(time.CLOCK_MONOTONIC)</span><br><span>@@ -223,7 +235,7 @@</span><br><span> """</span><br><span> Returns a statistical summary of the test.</span><br><span> """</span><br><span style="color: hsl(0, 100%, 40%);">- attempted = self._number_of_ms</span><br><span style="color: hsl(120, 100%, 40%);">+ attempted = len(self._subscribers)</span><br><span> completed = attempted - self._outstanding</span><br><span> min_latency, max_latency = self.find_min_max(filter(lambda x: x.has_lu_time(), self._results.values()))</span><br><span> return LUStats(attempted, completed, min_latency, max_latency)</span><br><span>diff --git a/src/osmo_ms_driver/starter.py b/src/osmo_ms_driver/starter.py</span><br><span>index 7710766..bc8548f 100644</span><br><span>--- a/src/osmo_ms_driver/starter.py</span><br><span>+++ b/src/osmo_ms_driver/starter.py</span><br><span>@@ -93,14 +93,14 @@</span><br><span> self._vphy_proc.terminate()</span><br><span> </span><br><span> class OsmoMobile(Launcher):</span><br><span style="color: hsl(0, 100%, 40%);">- def __init__(self, binary, env, name_number, tmp_dir, lua_tmpl, cfg_tmpl, imsi_ki_generator, phy_filename, ev_server_path):</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, binary, env, name_number, tmp_dir, lua_tmpl, cfg_tmpl, subscriber, phy_filename, ev_server_path):</span><br><span> super().__init__(binary, env, "osmo-ms-mob", name_number, tmp_dir)</span><br><span> self._lua_template = lua_tmpl</span><br><span> self._cfg_template = cfg_tmpl</span><br><span style="color: hsl(0, 100%, 40%);">- self._imsi_ki_generator = imsi_ki_generator</span><br><span> self._phy_filename = phy_filename</span><br><span> self._ev_server_path = ev_server_path</span><br><span style="color: hsl(0, 100%, 40%);">- self._imsi, self._ki = next(self._imsi_ki_generator)</span><br><span style="color: hsl(120, 100%, 40%);">+ self._imsi = subscriber.imsi()</span><br><span style="color: hsl(120, 100%, 40%);">+ self._ki = subscriber.ki()</span><br><span> self._omob_proc = None</span><br><span> </span><br><span> def imsi(self):</span><br><span>diff --git a/suites/nitb_netreg_mass/register_default_mass.py b/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>index 9772197..cec8c65 100644</span><br><span>--- a/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>+++ b/suites/nitb_netreg_mass/register_default_mass.py</span><br><span>@@ -10,6 +10,7 @@</span><br><span> nitb = suite.nitb()</span><br><span> bts = suite.bts()</span><br><span> ms_driver = suite.ms_driver()</span><br><span style="color: hsl(120, 100%, 40%);">+mobiles = suite.all_mobiles()</span><br><span> </span><br><span> print('Launching a simple network')</span><br><span> nitb.bts_add(bts)</span><br><span>@@ -18,8 +19,9 @@</span><br><span> wait(nitb.bts_is_connected, bts)</span><br><span> </span><br><span> # Configure all MS that the MS driver knows about.</span><br><span style="color: hsl(0, 100%, 40%);">-for ms in ms_driver.ms_subscribers():</span><br><span style="color: hsl(0, 100%, 40%);">- nitb.subscriber_add(ms)</span><br><span style="color: hsl(120, 100%, 40%);">+for mobile in mobiles:</span><br><span style="color: hsl(120, 100%, 40%);">+ nitb.subscriber_add(mobile)</span><br><span style="color: hsl(120, 100%, 40%);">+ ms_driver.subscriber_add(mobile)</span><br><span> </span><br><span> # Run the base test.</span><br><span> ms_driver.run_test()</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/13042">change 13042</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/13042"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: Ic9fd560c2924731e4fd6eea5aaf3ad565cb4ef52 </div>
<div style="display:none"> Gerrit-Change-Number: 13042 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Holger Freyther <holger@freyther.de> </div>