<p>pespin <strong>submitted</strong> this change.</p><p><a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/22600">View Change</a></p><div style="white-space:pre-wrap">Approvals:
pespin: Looks good to me, approved; Verified
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">enb,epc,ms: refactor KPI API<br><br>we previously mixed component specific and component agnostic APIs<br>(stdout vs. log file for example) for setting and retrieving KPI.<br><br>This patch propose to use a single abstract get_kpis() method for<br>all components that can be enriched with component-specific<br>stuff as desired.<br><br>In the case of srsLTE blocks, the main implementation will<br>remain in srslte_common() and is shared among srsENB/srsUE/srsEPC.<br><br>The KPI analyzer in srslte_common() extract and also manages<br>all three KPI sources (log, csv and stdout) independently.<br><br>In addition to the get_kpis() method that always returns a flat<br>dictionary, it also exposes get_kpi_tree() that return<br>a dict of KPI dicts that will be used for the Junit.xml generation.<br><br>Change-Id: I4bacc6b8a0cb92a581edfb947100b57022265265<br>---<br>M src/osmo_gsm_tester/obj/enb.py<br>M src/osmo_gsm_tester/obj/enb_amarisoft.py<br>M src/osmo_gsm_tester/obj/enb_srs.py<br>M src/osmo_gsm_tester/obj/epc.py<br>M src/osmo_gsm_tester/obj/epc_amarisoft.py<br>M src/osmo_gsm_tester/obj/epc_srs.py<br>M src/osmo_gsm_tester/obj/ms_srs.py<br>M src/osmo_gsm_tester/obj/srslte_common.py<br>8 files changed, 56 insertions(+), 45 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/obj/enb.py b/src/osmo_gsm_tester/obj/enb.py</span><br><span>index 32ead69..15a0033 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/enb.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/enb.py</span><br><span>@@ -356,4 +356,8 @@</span><br><span> def get_counter(self, counter_name):</span><br><span> pass</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-# vim: expandtab tabstop=4 shiftwidth=4</span><br><span style="color: hsl(120, 100%, 40%);">+ @abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_kpis(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# vim: expandtab tabstop=4 shiftwidth=4</span><br><span>\ No newline at end of file</span><br><span>diff --git a/src/osmo_gsm_tester/obj/enb_amarisoft.py b/src/osmo_gsm_tester/obj/enb_amarisoft.py</span><br><span>index e97bb90..01aed18 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/enb_amarisoft.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/enb_amarisoft.py</span><br><span>@@ -260,6 +260,9 @@</span><br><span> return self.process.get_counter_stdout('PRACH:')</span><br><span> raise log.Error('counter %s not implemented!' % counter_name)</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def get_kpis(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return {}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> def get_rfemu(self, cell=0, dl=True):</span><br><span> cell_list = self.gen_conf['enb'].get('cell_list', None)</span><br><span> if cell_list is None or len(cell_list) < cell + 1:</span><br><span>diff --git a/src/osmo_gsm_tester/obj/enb_srs.py b/src/osmo_gsm_tester/obj/enb_srs.py</span><br><span>index aee3f61..83df5ed 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/enb_srs.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/enb_srs.py</span><br><span>@@ -101,7 +101,7 @@</span><br><span> self.log(repr(e))</span><br><span> </span><br><span> # Collect KPIs for each TC</span><br><span style="color: hsl(0, 100%, 40%);">- self.testenv.test().set_kpis(self.get_kpis())</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.test().set_kpis(self.get_kpi_tree())</span><br><span> # Clean up for parent class:</span><br><span> super().cleanup()</span><br><span> </span><br><span>@@ -267,6 +267,9 @@</span><br><span> return self.process.get_counter_stdout('RACH:')</span><br><span> raise log.Error('counter %s not implemented!' % counter_name)</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def get_kpis(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return srslte_common.get_kpis(self)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> def get_rfemu(self, cell=0, dl=True):</span><br><span> cell_list = self.gen_conf['enb'].get('cell_list', None)</span><br><span> if cell_list is None or len(cell_list) < cell + 1:</span><br><span>diff --git a/src/osmo_gsm_tester/obj/epc.py b/src/osmo_gsm_tester/obj/epc.py</span><br><span>index 6f056fc..aaa96b7 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/epc.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/epc.py</span><br><span>@@ -116,4 +116,8 @@</span><br><span> def run_node(self):</span><br><span> return self._run_node</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ @abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_kpis(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # vim: expandtab tabstop=4 shiftwidth=4</span><br><span>diff --git a/src/osmo_gsm_tester/obj/epc_amarisoft.py b/src/osmo_gsm_tester/obj/epc_amarisoft.py</span><br><span>index 1291891..4c3bf07 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/epc_amarisoft.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/epc_amarisoft.py</span><br><span>@@ -199,4 +199,7 @@</span><br><span> # TODO: set proper addr</span><br><span> return '192.168.4.1'</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def get_kpis(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return {}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # vim: expandtab tabstop=4 shiftwidth=4</span><br><span>diff --git a/src/osmo_gsm_tester/obj/epc_srs.py b/src/osmo_gsm_tester/obj/epc_srs.py</span><br><span>index 6a7a20e..6a0a7bb 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/epc_srs.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/epc_srs.py</span><br><span>@@ -219,4 +219,7 @@</span><br><span> def tun_addr(self):</span><br><span> return '172.16.0.1'</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def get_kpis(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return srslte_common.get_kpis(self)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # vim: expandtab tabstop=4 shiftwidth=4</span><br><span>diff --git a/src/osmo_gsm_tester/obj/ms_srs.py b/src/osmo_gsm_tester/obj/ms_srs.py</span><br><span>index 2f19f3f..aaeeca5 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/ms_srs.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/ms_srs.py</span><br><span>@@ -126,7 +126,7 @@</span><br><span> self.log(repr(e))</span><br><span> </span><br><span> # Collect KPIs for each TC</span><br><span style="color: hsl(0, 100%, 40%);">- self.testenv.test().set_kpis(self.get_kpis())</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.test().set_kpis(self.get_kpi_tree())</span><br><span> </span><br><span> def features(self):</span><br><span> return self._conf.get('features', [])</span><br><span>diff --git a/src/osmo_gsm_tester/obj/srslte_common.py b/src/osmo_gsm_tester/obj/srslte_common.py</span><br><span>index 21001b7..cbc360f 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/srslte_common.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/srslte_common.py</span><br><span>@@ -27,7 +27,9 @@</span><br><span> self.process = None</span><br><span> self.metrics_file = None</span><br><span> self.stop_sleep_time = 6 # We require at most 5s to stop</span><br><span style="color: hsl(0, 100%, 40%);">- self.kpis = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log_kpi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.stdout_kpi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.csv_kpi = None</span><br><span> </span><br><span> def sleep_after_stop(self):</span><br><span> # Only sleep once</span><br><span>@@ -42,61 +44,50 @@</span><br><span> self.sleep_after_stop()</span><br><span> </span><br><span> def get_kpis(self):</span><br><span style="color: hsl(0, 100%, 40%);">- ''' Return all KPI '''</span><br><span style="color: hsl(0, 100%, 40%);">- if self.kpis is None:</span><br><span style="color: hsl(0, 100%, 40%);">- self.extract_kpis()</span><br><span style="color: hsl(0, 100%, 40%);">- return self.kpis</span><br><span style="color: hsl(120, 100%, 40%);">+ ''' Merge all KPI and return as flat dict '''</span><br><span style="color: hsl(120, 100%, 40%);">+ self.extract_kpis()</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_flat = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_flat.update(self.log_kpi)</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_flat.update(self.stdout_kpi)</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_flat.update(self.csv_kpi)</span><br><span style="color: hsl(120, 100%, 40%);">+ return kpi_flat</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def get_log_kpis(self):</span><br><span style="color: hsl(0, 100%, 40%);">- ''' Return KPIs extracted from log '''</span><br><span style="color: hsl(0, 100%, 40%);">- if self.kpis is None:</span><br><span style="color: hsl(0, 100%, 40%);">- self.extract_kpis()</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- # Use log KPIs if they exist for this node</span><br><span style="color: hsl(0, 100%, 40%);">- if "log_" + self.name() in self.kpis:</span><br><span style="color: hsl(0, 100%, 40%);">- log_kpi = self.kpis["log_" + self.name()]</span><br><span style="color: hsl(0, 100%, 40%);">- else:</span><br><span style="color: hsl(0, 100%, 40%);">- log_kpi = {}</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- # Make sure we have the errors and warnings counter in the dict</span><br><span style="color: hsl(0, 100%, 40%);">- if 'total_errors' not in log_kpi:</span><br><span style="color: hsl(0, 100%, 40%);">- log_kpi['total_errors'] = 0</span><br><span style="color: hsl(0, 100%, 40%);">- if 'total_warnings' not in log_kpi:</span><br><span style="color: hsl(0, 100%, 40%);">- log_kpi['total_warnings'] = 0</span><br><span style="color: hsl(0, 100%, 40%);">- return log_kpi</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_kpi_tree(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ ''' Return all KPI as dict of dict in which the source (e.g. stdout_srsue1) is the key of the first dict '''</span><br><span style="color: hsl(120, 100%, 40%);">+ self.extract_kpis()</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_tree = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_tree["log_" + self.name()] = self.log_kpi</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_tree["csv_" + self.name()] = self.csv_kpi</span><br><span style="color: hsl(120, 100%, 40%);">+ kpi_tree["stdout_" + self.name()] = self.stdout_kpi</span><br><span style="color: hsl(120, 100%, 40%);">+ return kpi_tree</span><br><span> </span><br><span> def extract_kpis(self):</span><br><span> ''' Use the srsLTE KPI analyzer module (part of srsLTE.git) if available to collect KPIs '''</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ # Make sure this only runs once</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.csv_kpi is not None or self.log_kpi is not None or self.stdout_kpi is not None:</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Start with empty KPIs</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log_kpi = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ self.stdout_kpi = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ self.csv_kpi = {}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # Stop application, copy back logs and process them</span><br><span> if self.running():</span><br><span> self.stop()</span><br><span> self.cleanup()</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- self.kpis = {}</span><br><span> try:</span><br><span> # Please make sure the srsLTE scripts folder is included in your PYTHONPATH env variable</span><br><span> from kpi_analyzer import kpi_analyzer</span><br><span> analyzer = kpi_analyzer(self.name())</span><br><span> if self.log_file is not None:</span><br><span style="color: hsl(0, 100%, 40%);">- self.kpis["log_" + self.name()] = analyzer.get_kpi_from_logfile(self.log_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log_kpi = analyzer.get_kpi_from_logfile(self.log_file)</span><br><span> if self.process.get_output_file('stdout') is not None:</span><br><span style="color: hsl(0, 100%, 40%);">- self.kpis["stdout_" + self.name()] = analyzer.get_kpi_from_stdout(self.process.get_output_file('stdout'))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.stdout_kpi = analyzer.get_kpi_from_stdout(self.process.get_output_file('stdout'))</span><br><span> if self.metrics_file is not None:</span><br><span style="color: hsl(0, 100%, 40%);">- self.kpis["csv_" + self.name()] = analyzer.get_kpi_from_csv(self.metrics_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.csv_kpi = analyzer.get_kpi_from_csv(self.metrics_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ # PHY errors for either UE or eNB components from parsed KPI vector as extra entry in dict</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log_kpi["num_phy_errors"] = analyzer.get_num_phy_errors(self.log_kpi)</span><br><span> except ImportError:</span><br><span style="color: hsl(0, 100%, 40%);">- self.log("Can't load KPI analyzer module.")</span><br><span style="color: hsl(0, 100%, 40%);">- self.kpis = {}</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- return self.kpis</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- def get_num_phy_errors(self, kpi):</span><br><span style="color: hsl(0, 100%, 40%);">- """ Use KPI analyzer to calculate the number PHY errors for either UE or eNB components from parsed KPI vector """</span><br><span style="color: hsl(0, 100%, 40%);">- try:</span><br><span style="color: hsl(0, 100%, 40%);">- # Same as above, make sure the srsLTE scripts folder is included in your PYTHONPATH env variable</span><br><span style="color: hsl(0, 100%, 40%);">- from kpi_analyzer import kpi_analyzer</span><br><span style="color: hsl(0, 100%, 40%);">- analyzer = kpi_analyzer(self.name())</span><br><span style="color: hsl(0, 100%, 40%);">- return analyzer.get_num_phy_errors(kpi)</span><br><span style="color: hsl(0, 100%, 40%);">- except ImportError:</span><br><span style="color: hsl(0, 100%, 40%);">- self.log("Can't load KPI analyzer module.")</span><br><span style="color: hsl(0, 100%, 40%);">- return 0</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log("Can't load KPI analyzer module.")</span><br><span>\ No newline at end of file</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/22600">change 22600</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/22600"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I4bacc6b8a0cb92a581edfb947100b57022265265 </div>
<div style="display:none"> Gerrit-Change-Number: 22600 </div>
<div style="display:none"> Gerrit-PatchSet: 8 </div>
<div style="display:none"> Gerrit-Owner: srs_andre <andre@softwareradiosystems.com> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins Builder </div>
<div style="display:none"> Gerrit-Reviewer: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>