<p>ninjab3s has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21302">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">Introduce Android UEs as new modems<br><br>To expand the test capacities we would like to introduce<br>Android UEs as new modems. Currently the following tests<br>are supported:<br>- Ping<br>- iPerf3 DL/UL<br>- RRC Mobile MT Ping<br><br>In the following is a small description.<br><br>Prerequisites:<br> - Android UE<br> - Rooted (Ping, iPerf, RRC Idle MT Ping)<br> - Qualcomm baseband with working diag_mdlog (RRC Idle MT Ping)<br> - iPerf3<br> - Dropbear<br> - OGT Slave Unit<br> - ADB (sudo apt-get install adb)<br> - Pycrate (https://github.com/P1sec/pycrate)<br> - SCAT<br> clone https://github.com/bedrankara/scat/ & install dependencies<br> checkout branch ogt<br> symlink scat (ln -s ~/scat/scat.py /usr/local/bin/scat)<br><br>Infrastructure explaination:<br>The Android UEs are connected to the OGT Units via USB. We<br>activate tethering and set up a SSH server (with Dropbear).<br>We chose tethering over WiFi to have a more stable route<br>for the ssh connection. We forward incoming connections to<br>the OGT unit hosting the Android UE(s) on specific ports<br>to the UEs via iptables. This enables OGT to issue commands<br>directly to the UEs. In case of local execution we use ADB<br>to issue commands to the AndroidUE. The set up was tested<br>with 5 Android UEs connected in parallel but it should be<br>scalable to the number of available IPs in the respective<br>subnet. Furthermore, we need to cross compile Dropbear<br>and iPerf3 to use them on the UEs. These tools have to be<br>added to the $PATH variable of the UEs.<br><br>Examplary set up:<br>In this example we have two separate OGT units (master<br>and slave) and two Android UEs that are connected to the<br>slave unit. An illustration may be found here: https://ibb.co/6BXSP2C<br><br>On UE 1:<br>ip address add 192.168.42.130/24 dev rndis0<br>ip route add 192.168.42.0/24 dev rndis0 table local_network<br>dropbearmulti dropbear -F -E -p 130 -R -T /data/local/tmp/authorized_keys -U 0 -G 0 -N root -A<br><br>On UE 2:<br>ip address add 192.168.42.131/24 dev rndis0<br>ip route add 192.168.42.0/24 dev rndis0 table local_network<br>dropbearmulti dropbear -F -E -p 131 -R -T /data/local/tmp/authorized_keys -U 0 -G 0 -N root -A<br><br>On OGT slave unit:<br>sudo ip link add name ogt type bridge<br>sudo ip l set eth0 master ogt<br>sudo ip l set enp0s20f0u1 master ogt<br>sudo ip l set enp0s20f0u2 master ogt<br>sudo ip a a 192.168.42.1/24 dev ogt<br>sudo ip link set ogt up<br><br>Now we have to manually connect to every UE from OGT Master<br>to set up SSH keys and verify that the setup works.<br>Therefore, use:<br>ssh -p [UE-PORT] root@[OGT SLAVE UNIT's IP]<br><br>Finally, to finish the setup procedure create the<br>remote_run_dir for Android UEs on the slave unit like<br>following:<br>mkdir /osmo-gsm-tester-androidue<br>chown jenkins /osmo-gsm-tester-androidue<br><br>Example for modem in resource.conf:<br>- label: mi5g<br> type: androidue<br> imsi: '901700000034757'<br> ki: '85E9E9A947B9ACBB966ED7113C7E1B8A'<br> opc: '3E1C73A29B9C293DC5A763E42C061F15'<br> ue_serial: '8d9d79a9'<br> apn_name: 'srsapn'<br> apn: 'srsapn'<br> apn_mcc: '901'<br> apn_mnc: '70'<br> sel_apn: 'True'<br> auth_algo: 'milenage'<br> features: ['4g', 'dl_qam256', 'qc_diag']<br> run_node:<br> run_type: ssh<br> run_addr: 100.113.1.170<br> ssh_user: jenkins<br> ssh_addr: 100.113.1.170<br> ue_ssh_port: 130<br><br>Example for default-suites.conf:<br>- 4g:ms-label@mi5g+srsenb-rftype@uhd+mod-enb-nprb@25+mod-enb-txmode@1<br><br>Change-Id: I79a5d803e869a868d4dac5e0d4c2feb38038dc5c<br>---<br>M src/osmo_gsm_tester/core/schema.py<br>A src/osmo_gsm_tester/obj/bitrate_monitor.py<br>M src/osmo_gsm_tester/obj/iperf3.py<br>M src/osmo_gsm_tester/obj/ms.py<br>A src/osmo_gsm_tester/obj/ms_android.py<br>A src/osmo_gsm_tester/obj/qc_diag.py<br>A sysmocom/scenarios/ms-label.conf<br>M sysmocom/suites/4g/iperf3_dl.py<br>M sysmocom/suites/4g/iperf3_ul.py<br>A utils/bin/osmo-gsm-tester_androidue_conn_chk.sh<br>A utils/bin/osmo-gsm-tester_androidue_diag_parser.sh<br>11 files changed, 801 insertions(+), 9 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/02/21302/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/core/schema.py b/src/osmo_gsm_tester/core/schema.py</span><br><span>index 9d26b0f..96d85f5 100644</span><br><span>--- a/src/osmo_gsm_tester/core/schema.py</span><br><span>+++ b/src/osmo_gsm_tester/core/schema.py</span><br><span>@@ -111,7 +111,7 @@</span><br><span> raise ValueError('Unknown Cipher value: %r' % val)</span><br><span> </span><br><span> def modem_feature(val):</span><br><span style="color: hsl(0, 100%, 40%);">- if val in ('sms', 'gprs', 'voice', 'ussd', 'sim', '2g', '3g', '4g', 'dl_qam256'):</span><br><span style="color: hsl(120, 100%, 40%);">+ if val in ('sms', 'gprs', 'voice', 'ussd', 'sim', '2g', '3g', '4g', 'dl_qam256', 'qc_diag'):</span><br><span> return True</span><br><span> raise ValueError('Unknown Modem Feature: %r' % val)</span><br><span> </span><br><span>diff --git a/src/osmo_gsm_tester/obj/bitrate_monitor.py b/src/osmo_gsm_tester/obj/bitrate_monitor.py</span><br><span>new file mode 100644</span><br><span>index 0000000..299a052</span><br><span>--- /dev/null</span><br><span>+++ b/src/osmo_gsm_tester/obj/bitrate_monitor.py</span><br><span>@@ -0,0 +1,145 @@</span><br><span style="color: hsl(120, 100%, 40%);">+# osmo_gsm_tester: specifics for running an AndroidUE modem</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright (C) 2020 by sysmocom - s.f.m.c. GmbH</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com></span><br><span style="color: hsl(120, 100%, 40%);">+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com></span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is free software: you can redistribute it and/or modify</span><br><span style="color: hsl(120, 100%, 40%);">+# it under the terms of the GNU General Public License as</span><br><span style="color: hsl(120, 100%, 40%);">+# published by the Free Software Foundation, either version 3 of the</span><br><span style="color: hsl(120, 100%, 40%);">+# License, or (at your option) any later version.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is distributed in the hope that it will be useful,</span><br><span style="color: hsl(120, 100%, 40%);">+# but WITHOUT ANY WARRANTY; without even the implied warranty of</span><br><span style="color: hsl(120, 100%, 40%);">+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span><br><span style="color: hsl(120, 100%, 40%);">+# GNU General Public License for more details.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# You should have received a copy of the GNU General Public License</span><br><span style="color: hsl(120, 100%, 40%);">+# along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from ..core import log, process</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class BitRateMonitor(log.Origin):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+##############</span><br><span style="color: hsl(120, 100%, 40%);">+# PROTECTED</span><br><span style="color: hsl(120, 100%, 40%);">+##############</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, testenv, run_dir, run_node, rem_host, ue_serial, ue_data_intf, remote_port):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__(log.C_TST, 'BitRateMonitor_%s' % ue_serial)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv = testenv</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host = rem_host</span><br><span style="color: hsl(120, 100%, 40%);">+ self._run_node = run_node</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_dir = run_dir</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_serial = ue_serial</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_data_intf = ue_data_intf</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_port = remote_port</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rx_monitor_proc = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tx_monitor_proc = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+########################</span><br><span style="color: hsl(120, 100%, 40%);">+# PUBLIC - INTERNAL API</span><br><span style="color: hsl(120, 100%, 40%);">+########################</span><br><span style="color: hsl(120, 100%, 40%);">+ def run_androidue_cmd(self, name, popen_args, sync=True):</span><br><span style="color: hsl(120, 100%, 40%);">+ # This function executes the given command directly on the Android UE. Therefore,</span><br><span style="color: hsl(120, 100%, 40%);">+ # ADB is used to execute commands locally and ssh for remote execution. Make sure</span><br><span style="color: hsl(120, 100%, 40%);">+ # ADB is installed</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ # use adb locally instead of ssh</span><br><span style="color: hsl(120, 100%, 40%);">+ adb_cmd_pref = ['adb', '-s', self.ue_serial, 'exec-out', 'su', '-c']</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args = adb_cmd_pref + list(popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local_sync(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ old_user = self.rem_host.remote_user</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.remote_user = 'root'</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.rem_host.RemoteProcess(name, popen_args, remote_env={}, remote_port=self._run_node.remote_port())</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch_sync()</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.remote_user = old_user</span><br><span style="color: hsl(120, 100%, 40%);">+ return proc</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+###################</span><br><span style="color: hsl(120, 100%, 40%);">+# PUBLIC (test API included)</span><br><span style="color: hsl(120, 100%, 40%);">+###################</span><br><span style="color: hsl(120, 100%, 40%);">+ def start(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # start bit rate monitoring on Android UE</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_rx_mon = ['while', 'true;', 'do',</span><br><span style="color: hsl(120, 100%, 40%);">+ 'echo', '`cat', '/sys/class/net/' + self.ue_data_intf + '/statistics/rx_bytes`;',</span><br><span style="color: hsl(120, 100%, 40%);">+ 'sleep', '1;', 'done']</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_tx_mon = ['while', 'true;', 'do',</span><br><span style="color: hsl(120, 100%, 40%);">+ 'echo', '`cat', '/sys/class/net/' + self.ue_data_intf + '/statistics/tx_bytes`;',</span><br><span style="color: hsl(120, 100%, 40%);">+ 'sleep', '1;', 'done']</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rx_monitor_proc = self.run_androidue_cmd("rx-monitor", popen_args_rx_mon, sync=False)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.remember_to_stop(self.rx_monitor_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tx_monitor_proc = self.run_androidue_cmd("tx-monitor", popen_args_tx_mon, sync=False)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.remember_to_stop(self.tx_monitor_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def stop(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.stop_process(self.rx_monitor_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.stop_process(self.tx_monitor_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def save_metrics(self, metrics_file):</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_rx_raw = self.rx_monitor_proc.get_stdout().split('\n')</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_tx_raw = self.tx_monitor_proc.get_stdout().split('\n')</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_rx_raw.remove('')</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_tx_raw.remove('')</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_rx_l = brate_rx_raw[1:]</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_tx_l = brate_tx_raw[1:]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(brate_rx_l) < 2 or len(brate_tx_l) < 2:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error("Insufficient data available to write metrics file")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # cut of elements if lists don't have the same length</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(brate_rx_l) > len(brate_tx_l):</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_rx_l = brate_rx_l[:len(brate_tx_l) - len(brate_rx_l)]</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(brate_rx_l) < len(brate_tx_l):</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_tx_l = brate_tx_l[:len(brate_rx_l) - len(brate_tx_l)]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # get start value</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_rx_last = int(brate_rx_l[0])</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_tx_last = int(brate_tx_l[0])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ with open(metrics_file, "w") as ue_metrics_fh:</span><br><span style="color: hsl(120, 100%, 40%);">+ ue_metrics_fh.write("time;cc;earfcn;pci;rsrp;pl;cfo;pci_neigh;rsrp_neigh;cfo_neigh;"</span><br><span style="color: hsl(120, 100%, 40%);">+ + "dl_mcs;dl_snr;dl_turbo;dl_brate;dl_bler;"</span><br><span style="color: hsl(120, 100%, 40%);">+ + "ul_ta;ul_mcs;ul_buff;ul_brate;ul_bler;rf_o;rf_u;rf_l;"</span><br><span style="color: hsl(120, 100%, 40%);">+ + "is_attached\n")</span><br><span style="color: hsl(120, 100%, 40%);">+ for i in range(1, len(brate_rx_l)):</span><br><span style="color: hsl(120, 100%, 40%);">+ time = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ cc = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ earfcn = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ pci = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ rsrp = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ pl = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ cfo = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ pci_neigh = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ rsrp_neigh = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ cfo_neigh = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ dl_mcs = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ dl_snr = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ dl_turbo = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ dl_brate = str((int(brate_rx_l[i]) - brate_rx_last) * 8)</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_rx_last = int(brate_rx_l[i])</span><br><span style="color: hsl(120, 100%, 40%);">+ dl_bler = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ ul_ta = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ ul_mcs = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ ul_buff = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ ul_brate = str((int(brate_tx_l[i]) - brate_tx_last) * 8)</span><br><span style="color: hsl(120, 100%, 40%);">+ brate_tx_last = int(brate_tx_l[i])</span><br><span style="color: hsl(120, 100%, 40%);">+ ul_bler = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ rf_o = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ rf_u = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ rf_l = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+ is_attached = "0"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ line = time + ";" + cc + ";" + earfcn + ";" + pci + ";" + rsrp + ";" + pl + ";" + cfo + ";" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + pci_neigh + ";" + rsrp_neigh + ";" + cfo_neigh + ";" + dl_mcs + ";" + dl_snr + ";" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + dl_turbo + ";" + dl_brate + ";" + dl_bler + ";" + ul_ta + ";" + ul_mcs + ";" + ul_buff + ";" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + ul_brate + ";" + ul_bler + ";" + rf_o + ";" + rf_u + ";" + rf_l + ";" + is_attached</span><br><span style="color: hsl(120, 100%, 40%);">+ ue_metrics_fh.write(line + "\n")</span><br><span>diff --git a/src/osmo_gsm_tester/obj/iperf3.py b/src/osmo_gsm_tester/obj/iperf3.py</span><br><span>index 2039a9b..5fe51d1 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/iperf3.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/iperf3.py</span><br><span>@@ -232,7 +232,7 @@</span><br><span> locally = not self._run_node or self._run_node.is_local()</span><br><span> return locally</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def prepare_test_proc(self, dir=None, netns=None, time_sec=None, proto=None, bitrate=0, tos=None):</span><br><span style="color: hsl(120, 100%, 40%);">+ def prepare_test_proc(self, dir=None, netns=None, time_sec=None, proto=None, bitrate=0, tos=None, ue=None):</span><br><span> values = config.get_defaults('iperf3cli')</span><br><span> config.overlay(values, self.testenv.suite().config().get('iperf3cli', {}))</span><br><span> </span><br><span>@@ -264,6 +264,7 @@</span><br><span> popen_args = ('iperf3', '-c', self.server.addr(),</span><br><span> '-p', str(self.server.port()), '-J',</span><br><span> '-t', str(time_sec))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> if dir == IPerf3Client.DIR_DL:</span><br><span> popen_args += ('-R',)</span><br><span> elif dir == IPerf3Client.DIR_BI:</span><br><span>@@ -273,14 +274,20 @@</span><br><span> if tos is not None:</span><br><span> popen_args += ('-S', str(tos))</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ ue_serial = None</span><br><span style="color: hsl(120, 100%, 40%);">+ remote_port = None</span><br><span style="color: hsl(120, 100%, 40%);">+ if ue and ue.__class__.__name__ == 'AndroidUE':</span><br><span style="color: hsl(120, 100%, 40%);">+ ue_serial = ue.ue_serial</span><br><span style="color: hsl(120, 100%, 40%);">+ remote_port = ue.get_remote_port()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> if self.runs_locally():</span><br><span style="color: hsl(0, 100%, 40%);">- proc = self.prepare_test_proc_locally(netns, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.prepare_test_proc_locally(netns, popen_args, ue_serial)</span><br><span> else:</span><br><span style="color: hsl(0, 100%, 40%);">- proc = self.prepare_test_proc_remotely(netns, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.prepare_test_proc_remotely(netns, popen_args, remote_port)</span><br><span> proc.set_default_wait_timeout(time_sec + 120) # leave extra time for remote run, ctrl conn establishment, buffer draining, etc.</span><br><span> return proc</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def prepare_test_proc_remotely(self, netns, popen_args):</span><br><span style="color: hsl(120, 100%, 40%);">+ def prepare_test_proc_remotely(self, netns, popen_args, remote_port):</span><br><span> self.rem_host = remote.RemoteHost(self.run_dir, self._run_node.ssh_user(), self._run_node.ssh_addr())</span><br><span> </span><br><span> remote_prefix_dir = util.Dir(IPerf3Client.REMOTE_DIR)</span><br><span>@@ -295,16 +302,21 @@</span><br><span> if netns:</span><br><span> self.process = self.rem_host.RemoteNetNSProcess(self.name(), netns, popen_args, env={})</span><br><span> else:</span><br><span style="color: hsl(0, 100%, 40%);">- self.process = self.rem_host.RemoteProcess(self.name(), popen_args, env={})</span><br><span style="color: hsl(120, 100%, 40%);">+ self.process = self.rem_host.RemoteProcess(self.name(), popen_args, remote_port=remote_port, env={})</span><br><span> return self.process</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def prepare_test_proc_locally(self, netns, popen_args):</span><br><span style="color: hsl(120, 100%, 40%);">+ def prepare_test_proc_locally(self, netns, popen_args, ue_serial):</span><br><span> pcap_recorder.PcapRecorder(self.testenv, self.run_dir.new_dir('pcap'), None,</span><br><span> 'host %s and port not 22' % self.server.addr(), netns)</span><br><span> </span><br><span> if self.logfile_supported:</span><br><span> popen_args += ('--logfile', os.path.abspath(self.log_file),)</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ # In case of an Android UE we need to extend the command to execute it with ADB</span><br><span style="color: hsl(120, 100%, 40%);">+ if ue_serial:</span><br><span style="color: hsl(120, 100%, 40%);">+ adb_cmd_pref = ['adb', '-s', ue_serial, 'exec-out', 'su', '-c']</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args = adb_cmd_pref + list(popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> if netns:</span><br><span> self.process = process.NetNSProcess(self.name(), self.run_dir, netns, popen_args, env={})</span><br><span> else:</span><br><span>diff --git a/src/osmo_gsm_tester/obj/ms.py b/src/osmo_gsm_tester/obj/ms.py</span><br><span>index 70ce558..bfd3f3b 100644</span><br><span>--- a/src/osmo_gsm_tester/obj/ms.py</span><br><span>+++ b/src/osmo_gsm_tester/obj/ms.py</span><br><span>@@ -72,6 +72,9 @@</span><br><span> elif ms_type == 'srsue':</span><br><span> from .ms_srs import srsUE</span><br><span> ms_class = srsUE</span><br><span style="color: hsl(120, 100%, 40%);">+ elif ms_type == 'androidue':</span><br><span style="color: hsl(120, 100%, 40%);">+ from .ms_android import AndroidUE</span><br><span style="color: hsl(120, 100%, 40%);">+ ms_class = AndroidUE</span><br><span> elif ms_type == 'amarisoftue':</span><br><span> from .ms_amarisoft import AmarisoftUE</span><br><span> ms_class = AmarisoftUE</span><br><span>diff --git a/src/osmo_gsm_tester/obj/ms_android.py b/src/osmo_gsm_tester/obj/ms_android.py</span><br><span>new file mode 100644</span><br><span>index 0000000..2ddc415</span><br><span>--- /dev/null</span><br><span>+++ b/src/osmo_gsm_tester/obj/ms_android.py</span><br><span>@@ -0,0 +1,433 @@</span><br><span style="color: hsl(120, 100%, 40%);">+# osmo_gsm_tester: specifics for running an AndroidUE modem</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright (C) 2020 by sysmocom - s.f.m.c. GmbH</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com></span><br><span style="color: hsl(120, 100%, 40%);">+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com></span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is free software: you can redistribute it and/or modify</span><br><span style="color: hsl(120, 100%, 40%);">+# it under the terms of the GNU General Public License as</span><br><span style="color: hsl(120, 100%, 40%);">+# published by the Free Software Foundation, either version 3 of the</span><br><span style="color: hsl(120, 100%, 40%);">+# License, or (at your option) any later version.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is distributed in the hope that it will be useful,</span><br><span style="color: hsl(120, 100%, 40%);">+# but WITHOUT ANY WARRANTY; without even the implied warranty of</span><br><span style="color: hsl(120, 100%, 40%);">+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span><br><span style="color: hsl(120, 100%, 40%);">+# GNU General Public License for more details.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# You should have received a copy of the GNU General Public License</span><br><span style="color: hsl(120, 100%, 40%);">+# along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import pprint</span><br><span style="color: hsl(120, 100%, 40%);">+import re</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from ..core import log, util, config, remote, process</span><br><span style="color: hsl(120, 100%, 40%);">+from ..core import schema</span><br><span style="color: hsl(120, 100%, 40%);">+from .run_node import RunNode</span><br><span style="color: hsl(120, 100%, 40%);">+from .ms import MS</span><br><span style="color: hsl(120, 100%, 40%);">+from .srslte_common import srslte_common</span><br><span style="color: hsl(120, 100%, 40%);">+from ..core.event_loop import MainLoop</span><br><span style="color: hsl(120, 100%, 40%);">+from .ms_srs import srsUEMetrics</span><br><span style="color: hsl(120, 100%, 40%);">+from .bitrate_monitor import BitRateMonitor</span><br><span style="color: hsl(120, 100%, 40%);">+from .qc_diag import QcDiag</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def on_register_schemas():</span><br><span style="color: hsl(120, 100%, 40%);">+ resource_schema = {</span><br><span style="color: hsl(120, 100%, 40%);">+ 'additional_args[]': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'apn': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'apn_name': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'apn_mcc': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'apn_mnc': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'sel_apn': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'ue_serial': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'enable_pcap': schema.BOOL_STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ for key, val in RunNode.schema().items():</span><br><span style="color: hsl(120, 100%, 40%);">+ resource_schema['run_node.%s' % key] = val</span><br><span style="color: hsl(120, 100%, 40%);">+ schema.register_resource_schema('modem', resource_schema)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ config_schema = {</span><br><span style="color: hsl(120, 100%, 40%);">+ 'enable_pcap': schema.BOOL_STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'log_all_level': schema.STR,</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ schema.register_config_schema('modem', config_schema)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class AndroidUE(MS, srslte_common):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ REMOTEDIR = '/osmo-gsm-tester-androidue'</span><br><span style="color: hsl(120, 100%, 40%);">+ METRICSFILE = 'android_ue_metrics.csv'</span><br><span style="color: hsl(120, 100%, 40%);">+ PCAPFILE = 'android_ue.pcap'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+##############</span><br><span style="color: hsl(120, 100%, 40%);">+# PROTECTED</span><br><span style="color: hsl(120, 100%, 40%);">+##############</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, testenv, conf):</span><br><span style="color: hsl(120, 100%, 40%);">+ self._run_node = RunNode.from_conf(conf.get('run_node', {}))</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__('androidue_%s' % self.addr(), testenv, conf)</span><br><span style="color: hsl(120, 100%, 40%);">+ srslte_common.__init__(self)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv = testenv</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_dir = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_run_dir = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.emm_connected = False</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rrc_connected = False</span><br><span style="color: hsl(120, 100%, 40%);">+ self.conn_reset_intvl = 20 # sec</span><br><span style="color: hsl(120, 100%, 40%);">+ self.connect_timeout = 300 # sec</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_serial = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.enable_pcap = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_pcap_file = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.pcap_file = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_data_intf = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_metrics_file = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.metrics_file = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.brate_mon = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.qc_diag_mon = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _clear_work_dirs(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # clear remote_run_dir</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_clear_run_dir = \</span><br><span style="color: hsl(120, 100%, 40%);">+ ['sudo', 'find', str(self.remote_run_dir), '-type', 'f',</span><br><span style="color: hsl(120, 100%, 40%);">+ '\\(', '-iname', '\\*.qdb', '-o', '-iname', '\\*.qmdl', '-o', '-iname', '\\*.xml', '\\)',</span><br><span style="color: hsl(120, 100%, 40%);">+ '-delete']</span><br><span style="color: hsl(120, 100%, 40%);">+ clear_run_dir_proc = self.run_androidue_host_cmd('clear-remote_run_dir', popen_args_clear_run_dir)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg("Deleted the following files: %s" % clear_run_dir_proc.get_stdout())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # clear diag_logs</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_clear_diag_logs = \</span><br><span style="color: hsl(120, 100%, 40%);">+ ['su', '-c', '\"', 'find', '/data/local/tmp/diag_logs/', '-type', 'f',</span><br><span style="color: hsl(120, 100%, 40%);">+ '\\(', '-iname', '\\*.qdb', '-o', '-iname', '\\*.qmdl', '-o', '-iname', '\\*.xml', '\\)',</span><br><span style="color: hsl(120, 100%, 40%);">+ '-delete', '\"']</span><br><span style="color: hsl(120, 100%, 40%);">+ clear_diag_logs_proc = self.run_androidue_cmd('clear-diag_logs', popen_args_clear_diag_logs)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg("Deleted the following files: %s" % clear_diag_logs_proc.get_stdout())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _restart_adb_inst(self, as_root=False):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_host_cmd("kill-adb", ['sudo', 'adb', 'kill-server'])</span><br><span style="color: hsl(120, 100%, 40%);">+ if as_root:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_host_cmd("start-adb", ['adb', 'start-server'])</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_host_cmd("start-adb", ['sudo', 'adb', 'start-server'])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+########################</span><br><span style="color: hsl(120, 100%, 40%);">+# PUBLIC - INTERNAL API</span><br><span style="color: hsl(120, 100%, 40%);">+########################</span><br><span style="color: hsl(120, 100%, 40%);">+ def cleanup(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.stop()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def check_device_availability(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ serials_cmd = ['adb', 'devices']</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.run_androidue_host_cmd("devices", serials_cmd)</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.ue_serial in proc.get_stdout()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_assigned_addr(self, ipv6=False):</span><br><span style="color: hsl(120, 100%, 40%);">+ ip_prefix = "172.16.0"</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.run_androidue_cmd("get-ip", ['ip', 'addr', 'show'])</span><br><span style="color: hsl(120, 100%, 40%);">+ out_l = proc.get_stdout().split('\n')</span><br><span style="color: hsl(120, 100%, 40%);">+ ip = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for line in out_l:</span><br><span style="color: hsl(120, 100%, 40%);">+ if ip_prefix in line:</span><br><span style="color: hsl(120, 100%, 40%);">+ ip = line.split(' ')[5][:-3] # UE's IP</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_data_intf = line.split(' ')[-1] # UE's data interface</span><br><span style="color: hsl(120, 100%, 40%);">+ return ip</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_carrier_id(self, carrier_name):</span><br><span style="color: hsl(120, 100%, 40%);">+ qry_carrier_cmd = "content query --uri \"content://telephony/carriers\" "</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.run_androidue_cmd("set-apn", [qry_carrier_cmd])</span><br><span style="color: hsl(120, 100%, 40%);">+ available_carriers = proc.get_stdout().split("\n")</span><br><span style="color: hsl(120, 100%, 40%);">+ carr_id = -1</span><br><span style="color: hsl(120, 100%, 40%);">+ for carr in available_carriers:</span><br><span style="color: hsl(120, 100%, 40%);">+ if 'name=' + carrier_name in carr: # found carrier</span><br><span style="color: hsl(120, 100%, 40%);">+ carr_id = re.findall(r'_id=(\S+),', carr)[0]</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+ return carr_id</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def set_new_carrier(self, apn_parameter, carr_id):</span><br><span style="color: hsl(120, 100%, 40%);">+ # check if carrier was found, delete it if exists</span><br><span style="color: hsl(120, 100%, 40%);">+ if carr_id != -1:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.delete_apn(apn_parameter["carrier"])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ set_carrier_cmd = "content insert --uri content://telephony/carriers" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind name:s:\"" + apn_parameter["carrier"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind numeric:s:\"" + apn_parameter["mcc"] + apn_parameter["mnc"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind mcc:s:\"" + apn_parameter["mcc"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind mnc:s:\"" + apn_parameter["mnc"] + "\""\</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind apn:s:\"" + apn_parameter["apn"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind user:s:\"" + apn_parameter["user"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind password:s:\"" + apn_parameter["password"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind mmsc:s:\"" + apn_parameter["mmsc"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind mmsport:s:\"" + apn_parameter["mmsport"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind mmsproxy:s:\"" + apn_parameter["mmsproxy"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind authtype:s:\"" + apn_parameter["auth"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind type:s:\"" + apn_parameter["type"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind protocol:s:\"" + apn_parameter["protocol"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind mvno_type:s:\"" + apn_parameter["mvnotype"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind mvno_match_data:s:\"" + apn_parameter["mvnoval"] + "\"" \</span><br><span style="color: hsl(120, 100%, 40%);">+ + " --bind sub_id:s:\"" + apn_parameter["groupid"] + "\""</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_cmd("set-apn", [set_carrier_cmd])</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.get_carrier_id(apn_parameter["carrier"])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def set_preferred_apn(self, carr_id):</span><br><span style="color: hsl(120, 100%, 40%);">+ if carr_id != -1:</span><br><span style="color: hsl(120, 100%, 40%);">+ set_apn_cmd = "content insert --uri content://telephony/carriers/preferapn --bind apn_id:s:\"" + str(carr_id) + "\""</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_cmd("set-apn", [set_apn_cmd])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def select_apn(self, carr_name):</span><br><span style="color: hsl(120, 100%, 40%);">+ carr_id = self.get_carrier_id(carr_name)</span><br><span style="color: hsl(120, 100%, 40%);">+ if carr_id == 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # select carrier by ID</span><br><span style="color: hsl(120, 100%, 40%);">+ sel_apn_cmd = "content update --uri content://telephony/carriers/preferapn --bind apn_id:s:\"" + str(carr_id) + "\""</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_cmd("set-apn", [sel_apn_cmd])</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def delete_apn(self, carr_name):</span><br><span style="color: hsl(120, 100%, 40%);">+ set_apn_cmd = "content delete --uri content://telephony/carriers --where \'name=\"" + str(carr_name) + "\" \'"</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_cmd("set-apn", [set_apn_cmd])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+###################</span><br><span style="color: hsl(120, 100%, 40%);">+# PUBLIC (test API included)</span><br><span style="color: hsl(120, 100%, 40%);">+###################</span><br><span style="color: hsl(120, 100%, 40%);">+ def run_androidue_cmd(self, name, popen_args, sync=True):</span><br><span style="color: hsl(120, 100%, 40%);">+ # This function executes the given command directly on the Android UE. Therefore,</span><br><span style="color: hsl(120, 100%, 40%);">+ # ADB is used to execute commands locally and ssh for remote execution. Make sure</span><br><span style="color: hsl(120, 100%, 40%);">+ # ADB is installed</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ # use adb locally instead of ssh</span><br><span style="color: hsl(120, 100%, 40%);">+ adb_cmd_pref = ['adb', '-s', self.ue_serial, 'exec-out', 'su', '-c']</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args = adb_cmd_pref + list(popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local_sync(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ old_user = self.rem_host.remote_user</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.remote_user = 'root'</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.rem_host.RemoteProcess(name, popen_args, remote_env={}, remote_port=self._run_node.remote_port())</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch_sync()</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.remote_user = old_user</span><br><span style="color: hsl(120, 100%, 40%);">+ return proc</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def run_androidue_host_cmd(self, name, popen_args, sync=True):</span><br><span style="color: hsl(120, 100%, 40%);">+ # This function executes commands on the host connected to the Android UE.</span><br><span style="color: hsl(120, 100%, 40%);">+ # Make sure ADB is installed</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local_sync(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.rem_host.RemoteProcess(name, popen_args, env={})</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch_sync()</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch()</span><br><span style="color: hsl(120, 100%, 40%);">+ return proc</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def run_netns_wait(self, name, popen_args):</span><br><span style="color: hsl(120, 100%, 40%);">+ # This function guarantees the compatibility with the current ping test. Please</span><br><span style="color: hsl(120, 100%, 40%);">+ # note that this function cannot execute commands on the machine hosting the Android</span><br><span style="color: hsl(120, 100%, 40%);">+ # UE. Use run_androidue_host_cmd() for this purpose.</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.run_androidue_cmd(name, popen_args, sync=True)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def start(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # Sometimes we need to restart adb in case a UE was not recognized</span><br><span style="color: hsl(120, 100%, 40%);">+ if not self.check_device_availability():</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log("Can't find requested device. Restarting ADB and check again")</span><br><span style="color: hsl(120, 100%, 40%);">+ self._restart_adb_inst(True)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not self.check_device_availability():</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error("Device with serial %s is not available" % self.ue_serial)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.set_airplane_mode(True)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def stop(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.brate_mon:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.brate_mon.stop()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.qc_diag_mon:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.qc_diag_mon.stop()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.set_airplane_mode(True)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.enable_pcap and self.qc_diag_mon:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.qc_diag_mon.write_pcap()</span><br><span style="color: hsl(120, 100%, 40%);">+ if not self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ self.scp_back_pcap()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def configure(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ values = dict(ue=config.get_defaults('androidue'))</span><br><span style="color: hsl(120, 100%, 40%);">+ config.overlay(values, dict(ue=self.testenv.suite().config().get('modem', {})))</span><br><span style="color: hsl(120, 100%, 40%);">+ config.overlay(values, dict(ue=self._conf))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg('AndroidUE CONFIG:\n' + pprint.pformat(values))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_serial = values['ue']['ue_serial']</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if 'qc_diag' in self.features():</span><br><span style="color: hsl(120, 100%, 40%);">+ self.enable_pcap = util.str2bool(values['ue'].get('enable_pcap', 'false'))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_dir = util.Dir(self.testenv.test().get_run_dir().new_dir(self.name()))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.metrics_file = self.run_dir.child(AndroidUE.METRICSFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host = remote.RemoteHost(self.run_dir, self._run_node.ssh_user(), self._run_node.ssh_addr())</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_run_dir = util.Dir(AndroidUE.REMOTEDIR)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_metrics_file = self.remote_run_dir.child(AndroidUE.METRICSFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.pcap_file = self.run_dir.child(AndroidUE.PCAPFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_pcap_file = self.remote_run_dir.child(AndroidUE.PCAPFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ apn_params = {</span><br><span style="color: hsl(120, 100%, 40%);">+ "carrier": str(values['ue']['apn_name'] or 'default'),</span><br><span style="color: hsl(120, 100%, 40%);">+ "apn": str(values['ue']['apn']), # mandatory</span><br><span style="color: hsl(120, 100%, 40%);">+ "proxy": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "port": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "user": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "password": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "server": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "mmsc": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "mmsport": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "mmsproxy": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "mcc": str(values['ue']['apn_mcc']), # mandatory</span><br><span style="color: hsl(120, 100%, 40%);">+ "mnc": str(values['ue']['apn_mnc']), # mandatory</span><br><span style="color: hsl(120, 100%, 40%);">+ "auth": str('-1'),</span><br><span style="color: hsl(120, 100%, 40%);">+ "type": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "protocol": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "mvnotype": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "mvnoval": str(''),</span><br><span style="color: hsl(120, 100%, 40%);">+ "groupid": str('-1')</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # On some UEs it is only possible to set a new APN if airplane mode is turned off</span><br><span style="color: hsl(120, 100%, 40%);">+ self.set_airplane_mode(False)</span><br><span style="color: hsl(120, 100%, 40%);">+ MainLoop.sleep(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg("APN parameters: " + str(apn_params))</span><br><span style="color: hsl(120, 100%, 40%);">+ # self.set_apn(apn_params, True)</span><br><span style="color: hsl(120, 100%, 40%);">+ MainLoop.sleep(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.set_airplane_mode(False)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # clear working directories</span><br><span style="color: hsl(120, 100%, 40%);">+ self._clear_work_dirs()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def connect(self, enb):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log('Starting AndroidUE')</span><br><span style="color: hsl(120, 100%, 40%);">+ self.configure()</span><br><span style="color: hsl(120, 100%, 40%);">+ CONN_CHK = 'osmo-gsm-tester_androidue_conn_chk.sh'</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_emm_conn_chk = [CONN_CHK, self.ue_serial]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if 'qc_diag' in self.features():</span><br><span style="color: hsl(120, 100%, 40%);">+ self.qc_diag_mon = QcDiag(self.testenv, self.run_dir, self.remote_run_dir, self._run_node,</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host, self.ue_serial, self._run_node.remote_port())</span><br><span style="color: hsl(120, 100%, 40%);">+ self.qc_diag_mon.start()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ emm_conn_chk_proc = self.run_androidue_host_cmd('emm-conn-chk', popen_args_emm_conn_chk, sync=False)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # TODO</span><br><span style="color: hsl(120, 100%, 40%);">+ # Adjust query 'LTE' to all other network types to make the AndroidUE class compatible with 2G network tests</span><br><span style="color: hsl(120, 100%, 40%);">+ # maybe derive the network type from the enb-parameter</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # check connection status</span><br><span style="color: hsl(120, 100%, 40%);">+ timer = self.connect_timeout</span><br><span style="color: hsl(120, 100%, 40%);">+ while timer > 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ if 'LTE' in emm_conn_chk_proc.get_stdout():</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.get_assigned_addr():</span><br><span style="color: hsl(120, 100%, 40%);">+ self.emm_connected = True</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rrc_connected = True</span><br><span style="color: hsl(120, 100%, 40%);">+ emm_conn_chk_proc.terminate()</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # reset connection</span><br><span style="color: hsl(120, 100%, 40%);">+ if timer % self.conn_reset_intvl == 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.set_airplane_mode(True)</span><br><span style="color: hsl(120, 100%, 40%);">+ MainLoop.sleep(2)</span><br><span style="color: hsl(120, 100%, 40%);">+ timer -= 2</span><br><span style="color: hsl(120, 100%, 40%);">+ self.set_airplane_mode(False)</span><br><span style="color: hsl(120, 100%, 40%);">+ MainLoop.sleep(2)</span><br><span style="color: hsl(120, 100%, 40%);">+ timer -= 1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if timer == 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error("Android UE %s connection timer expired" % self.ue_serial)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.brate_mon = BitRateMonitor(self.testenv, self.run_dir, self._run_node, self.rem_host,</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_serial, self.ue_data_intf, self._run_node.remote_port())</span><br><span style="color: hsl(120, 100%, 40%);">+ self.brate_mon.start()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def is_rrc_connected(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not ('qc_diag' in self.features()):</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error("Monitoring RRC states not supported (missing qc_diag feature?)")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if not self.qc_diag_mon.running():</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error("Diag monitoring crashed or was not started")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ rrc_state = self.qc_diag_mon.get_rrc_state()</span><br><span style="color: hsl(120, 100%, 40%);">+ if 'RRC_IDLE_CAMPED' in rrc_state:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rrc_connected = False</span><br><span style="color: hsl(120, 100%, 40%);">+ elif 'RRC_CONNECTED' in rrc_state:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rrc_connected = True</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.rrc_connected</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def is_registered(self, mcc_mnc=None):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.emm_connected</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_counter(self, counter_name):</span><br><span style="color: hsl(120, 100%, 40%);">+ if counter_name == 'prach_sent':</span><br><span style="color: hsl(120, 100%, 40%);">+ # not implemented so far, return 2 to pass tests</span><br><span style="color: hsl(120, 100%, 40%);">+ return 2</span><br><span style="color: hsl(120, 100%, 40%);">+ elif counter_name == 'paging_received':</span><br><span style="color: hsl(120, 100%, 40%);">+ self.qc_diag_mon.get_paging_counter()</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error("Counter %s not implemented" % counter_name)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def running(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # check if Android UE is available via ADB</span><br><span style="color: hsl(120, 100%, 40%);">+ if not (self.ue_serial is None):</span><br><span style="color: hsl(120, 100%, 40%);">+ serials_cmd = ['adb', 'devices']</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.run_androidue_cmd("devices", serials_cmd)</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.ue_serial in proc.get_stdout():</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg("Device with serial %s is currently not available" % self.ue_serial)</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def addr(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._run_node.run_addr()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def run_node(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._run_node</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def features(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._conf.get('features', [])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_remote_port(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._run_node.remote_port()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def set_airplane_mode(self, apm_state):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log("Setting airplane mode: " + str(apm_state))</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args = ['settings', 'put', 'global', 'airplane_mode_on', str(int(apm_state)), ';',</span><br><span style="color: hsl(120, 100%, 40%);">+ 'wait $!;',</span><br><span style="color: hsl(120, 100%, 40%);">+ 'su', '-c', '\"', 'am', 'broadcast', '-a', 'android.intent.action.AIRPLANE_MODE', '\"', ';']</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_androidue_cmd("set-airplane-mode", popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def set_apn(self, apn_param, sel_apn):</span><br><span style="color: hsl(120, 100%, 40%);">+ # search for carrier in APN database</span><br><span style="color: hsl(120, 100%, 40%);">+ carrier_id = self.get_carrier_id(apn_param["carrier"])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # add/update carrier</span><br><span style="color: hsl(120, 100%, 40%);">+ carrier_id = self.set_new_carrier(apn_param, carrier_id)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # select as preferred APN</span><br><span style="color: hsl(120, 100%, 40%);">+ if sel_apn:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.set_preferred_apn(carrier_id)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def scp_back_pcap(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.scpfrom('scp-back-pcap', self.remote_pcap_file, self.pcap_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ except Exception as e:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log(repr(e))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def verify_metric(self, value, operation='avg', metric='dl_brate', criterion='gt', window=1):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.brate_mon.save_metrics(self.metrics_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ metrics = srsUEMetrics(self.metrics_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ return metrics.verify(value, operation, metric, criterion, window)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def netns(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return None</span><br><span>diff --git a/src/osmo_gsm_tester/obj/qc_diag.py b/src/osmo_gsm_tester/obj/qc_diag.py</span><br><span>new file mode 100644</span><br><span>index 0000000..24486b7</span><br><span>--- /dev/null</span><br><span>+++ b/src/osmo_gsm_tester/obj/qc_diag.py</span><br><span>@@ -0,0 +1,172 @@</span><br><span style="color: hsl(120, 100%, 40%);">+# osmo_gsm_tester: specifics for running an AndroidUE modem</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Copyright (C) 2020 by sysmocom - s.f.m.c. GmbH</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com></span><br><span style="color: hsl(120, 100%, 40%);">+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com></span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is free software: you can redistribute it and/or modify</span><br><span style="color: hsl(120, 100%, 40%);">+# it under the terms of the GNU General Public License as</span><br><span style="color: hsl(120, 100%, 40%);">+# published by the Free Software Foundation, either version 3 of the</span><br><span style="color: hsl(120, 100%, 40%);">+# License, or (at your option) any later version.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is distributed in the hope that it will be useful,</span><br><span style="color: hsl(120, 100%, 40%);">+# but WITHOUT ANY WARRANTY; without even the implied warranty of</span><br><span style="color: hsl(120, 100%, 40%);">+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span><br><span style="color: hsl(120, 100%, 40%);">+# GNU General Public License for more details.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# You should have received a copy of the GNU General Public License</span><br><span style="color: hsl(120, 100%, 40%);">+# along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from ..core import log, process</span><br><span style="color: hsl(120, 100%, 40%);">+from ..core.event_loop import MainLoop</span><br><span style="color: hsl(120, 100%, 40%);">+from .ms_android import AndroidUE</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class QcDiag(log.Origin):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ DIAG_PARSER = 'osmo-gsm-tester_androidue_diag_parser.sh'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+##############</span><br><span style="color: hsl(120, 100%, 40%);">+# PROTECTED</span><br><span style="color: hsl(120, 100%, 40%);">+##############</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, testenv, run_dir, remote_run_dir, run_node, rem_host, ue_serial, remote_port):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__(log.C_TST, 'QcDiag_%s' % ue_serial)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv = testenv</span><br><span style="color: hsl(120, 100%, 40%);">+ self.run_dir = run_dir</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_run_dir = remote_run_dir</span><br><span style="color: hsl(120, 100%, 40%);">+ self._run_node = run_node</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host = rem_host</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ue_serial = ue_serial</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_port = remote_port</span><br><span style="color: hsl(120, 100%, 40%);">+ self.pcap_file = self.run_dir.child(AndroidUE.PCAPFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_pcap_file = self.remote_run_dir.child(AndroidUE.PCAPFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diag_parser_proc = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diag_monitor_proc = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _clear_work_dirs(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # clear remote_run_dir</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_clear_run_dir = \</span><br><span style="color: hsl(120, 100%, 40%);">+ ['sudo', 'find', str(self.remote_run_dir), '-type', 'f',</span><br><span style="color: hsl(120, 100%, 40%);">+ '\\(', '-iname', '\\*.qdb', '-o', '-iname', '\\*.qmdl', '-o', '-iname', '\\*.xml', '\\)',</span><br><span style="color: hsl(120, 100%, 40%);">+ '-delete']</span><br><span style="color: hsl(120, 100%, 40%);">+ clear_run_dir_proc = self.run_androidue_host_cmd('clear-remote_run_dir', popen_args_clear_run_dir)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg("Deleted the following files: %s" % clear_run_dir_proc.get_stdout())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # clear diag_logs</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_clear_diag_logs = \</span><br><span style="color: hsl(120, 100%, 40%);">+ ['su', '-c', '\"', 'find', '/data/local/tmp/diag_logs/', '-type', 'f',</span><br><span style="color: hsl(120, 100%, 40%);">+ '\\(', '-iname', '\\*.qdb', '-o', '-iname', '\\*.qmdl', '-o', '-iname', '\\*.xml', '\\)',</span><br><span style="color: hsl(120, 100%, 40%);">+ '-delete', '\"']</span><br><span style="color: hsl(120, 100%, 40%);">+ clear_diag_logs_proc = self.run_androidue_cmd('clear-diag_logs', popen_args_clear_diag_logs)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.dbg("Deleted the following files: %s" % clear_diag_logs_proc.get_stdout())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+########################</span><br><span style="color: hsl(120, 100%, 40%);">+# PUBLIC - INTERNAL API</span><br><span style="color: hsl(120, 100%, 40%);">+########################</span><br><span style="color: hsl(120, 100%, 40%);">+ def run_androidue_cmd(self, name, popen_args, sync=True):</span><br><span style="color: hsl(120, 100%, 40%);">+ # This function executes the given command directly on the Android UE. Therefore,</span><br><span style="color: hsl(120, 100%, 40%);">+ # ADB is used to execute commands locally and ssh for remote execution. Make sure</span><br><span style="color: hsl(120, 100%, 40%);">+ # ADB is installed</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ # use adb locally instead of ssh</span><br><span style="color: hsl(120, 100%, 40%);">+ adb_cmd_pref = ['adb', '-s', self.ue_serial, 'exec-out', 'su', '-c']</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args = adb_cmd_pref + list(popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local_sync(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ old_user = self.rem_host.remote_user</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.remote_user = 'root'</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.rem_host.RemoteProcess(name, popen_args, remote_env={}, remote_port=self._run_node.remote_port())</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch_sync()</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.remote_user = old_user</span><br><span style="color: hsl(120, 100%, 40%);">+ return proc</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def run_androidue_host_cmd(self, name, popen_args, sync=True):</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local_sync(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = process.run_local(self.run_dir, name, popen_args)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc = self.rem_host.RemoteProcess(name, popen_args, env={})</span><br><span style="color: hsl(120, 100%, 40%);">+ if sync:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch_sync()</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.launch()</span><br><span style="color: hsl(120, 100%, 40%);">+ return proc</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+###################</span><br><span style="color: hsl(120, 100%, 40%);">+# PUBLIC (test API included)</span><br><span style="color: hsl(120, 100%, 40%);">+###################</span><br><span style="color: hsl(120, 100%, 40%);">+ def start(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_diag = ['/vendor/bin/diag_mdlog', '-s', '90000', '-f', '/data/local/tmp/ogt_diag.cfg',</span><br><span style="color: hsl(120, 100%, 40%);">+ '-o', '/data/local/tmp/diag_logs']</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diag_monitor_proc = self.run_androidue_cmd("diag-monitor", popen_args_diag, sync=False)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.remember_to_stop(self.diag_monitor_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_diag_parser = [QcDiag.DIAG_PARSER, str(self.ue_serial), str(self.run_dir), str(self.pcap_file)]</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_diag_parser = [QcDiag.DIAG_PARSER, str(self.ue_serial), str(self.remote_run_dir), str(self.remote_pcap_file)]</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diag_parser_proc = self.run_androidue_host_cmd("diag-parser", popen_args_diag_parser, sync=False)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.remember_to_stop(self.diag_parser_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def stop(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.stop_process(self.diag_parser_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.testenv.stop_process(self.diag_monitor_proc)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def write_pcap(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # We need to stop the diag_parser to avoid pulling a new .qmdl during</span><br><span style="color: hsl(120, 100%, 40%);">+ # the parsing process. The process will be restarted if it was still running.</span><br><span style="color: hsl(120, 100%, 40%);">+ # The diag_monitor process can continue, as it does not hinder this process.</span><br><span style="color: hsl(120, 100%, 40%);">+ restart_diag_parser = False</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.diag_parser_proc.is_running():</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diag_parser_proc.terminate()</span><br><span style="color: hsl(120, 100%, 40%);">+ restart_diag_parser = True</span><br><span style="color: hsl(120, 100%, 40%);">+ self._clear_work_dirs()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._run_node.is_local():</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_diag_parser = [QcDiag.DIAG_PARSER, str(self.ue_serial), str(self.run_dir), str(self.pcap_file)]</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ popen_args_diag_parser = [QcDiag.DIAG_PARSER, str(self.ue_serial), str(self.remote_run_dir), str(self.remote_pcap_file)]</span><br><span style="color: hsl(120, 100%, 40%);">+ diag_parser_proc = self.run_androidue_host_cmd("diag-parser", popen_args_diag_parser, sync=False)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ timer = 300 # sec</span><br><span style="color: hsl(120, 100%, 40%);">+ while timer > 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ diag_parser_stdout = diag_parser_proc.get_stdout()</span><br><span style="color: hsl(120, 100%, 40%);">+ if diag_parser_stdout.count('Pulling new .qmdl file...') > 1:</span><br><span style="color: hsl(120, 100%, 40%);">+ # If the parsers pulls the .qmdl file for the second time we know that</span><br><span style="color: hsl(120, 100%, 40%);">+ # the parsing of the first one is done</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+ MainLoop.sleep(2)</span><br><span style="color: hsl(120, 100%, 40%);">+ timer -= 2</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if timer <= 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error("Timed out writing PCAP file")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if restart_diag_parser:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diag_parser_proc = diag_parser_proc</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ diag_parser_proc.terminate()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_rrc_state(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ diag_parser_stdout_l = self.diag_parser_proc.get_stdout().split('\n')</span><br><span style="color: hsl(120, 100%, 40%);">+ for line in reversed(diag_parser_stdout_l):</span><br><span style="color: hsl(120, 100%, 40%);">+ if 'LTE_RRC_STATE_CHANGE' in line:</span><br><span style="color: hsl(120, 100%, 40%);">+ rrc_state = line.split(' ')[-1].replace('rrc_state=', '')</span><br><span style="color: hsl(120, 100%, 40%);">+ rrc_state.replace('\'', '')</span><br><span style="color: hsl(120, 100%, 40%);">+ return rrc_state</span><br><span style="color: hsl(120, 100%, 40%);">+ return ''</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_paging_counter(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ diag_parser_stdout_l = self.diag_parser_proc.get_stdout().split("\n")</span><br><span style="color: hsl(120, 100%, 40%);">+ return diag_parser_stdout_l.count('Paging received')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def running(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.diag_parser_proc.is_running() and self.diag_monitor_proc.is_running()</span><br><span>diff --git a/sysmocom/scenarios/ms-label.conf b/sysmocom/scenarios/ms-label.conf</span><br><span>new file mode 100644</span><br><span>index 0000000..a129c0e</span><br><span>--- /dev/null</span><br><span>+++ b/sysmocom/scenarios/ms-label.conf</span><br><span>@@ -0,0 +1,3 @@</span><br><span style="color: hsl(120, 100%, 40%);">+resources:</span><br><span style="color: hsl(120, 100%, 40%);">+ modem:</span><br><span style="color: hsl(120, 100%, 40%);">+ - label: ${param1}</span><br><span>diff --git a/sysmocom/suites/4g/iperf3_dl.py b/sysmocom/suites/4g/iperf3_dl.py</span><br><span>index bf5b1f0..ed137bd 100755</span><br><span>--- a/sysmocom/suites/4g/iperf3_dl.py</span><br><span>+++ b/sysmocom/suites/4g/iperf3_dl.py</span><br><span>@@ -23,7 +23,7 @@</span><br><span> max_rate = enb.ue_max_rate(downlink=True, num_carriers=ue.num_carriers)</span><br><span> </span><br><span> iperf3srv.start()</span><br><span style="color: hsl(0, 100%, 40%);">-proc = iperf3cli.prepare_test_proc(iperf3cli.DIR_DL, ue.netns(), bitrate=max_rate)</span><br><span style="color: hsl(120, 100%, 40%);">+proc = iperf3cli.prepare_test_proc(iperf3cli.DIR_DL, ue.netns(), bitrate=max_rate, ue=ue)</span><br><span> </span><br><span> print('waiting for UE to attach...')</span><br><span> wait(ue.is_registered)</span><br><span>diff --git a/sysmocom/suites/4g/iperf3_ul.py b/sysmocom/suites/4g/iperf3_ul.py</span><br><span>index 6c0d25d..6cc9f21 100755</span><br><span>--- a/sysmocom/suites/4g/iperf3_ul.py</span><br><span>+++ b/sysmocom/suites/4g/iperf3_ul.py</span><br><span>@@ -23,7 +23,7 @@</span><br><span> max_rate = enb.ue_max_rate(downlink=False, num_carriers=ue.num_carriers)</span><br><span> </span><br><span> iperf3srv.start()</span><br><span style="color: hsl(0, 100%, 40%);">-proc = iperf3cli.prepare_test_proc(iperf3cli.DIR_UL, ue.netns(), bitrate=max_rate)</span><br><span style="color: hsl(120, 100%, 40%);">+proc = iperf3cli.prepare_test_proc(iperf3cli.DIR_UL, ue.netns(), bitrate=max_rate, ue=ue)</span><br><span> </span><br><span> print('waiting for UE to attach...')</span><br><span> wait(ue.is_registered)</span><br><span>diff --git a/utils/bin/osmo-gsm-tester_androidue_conn_chk.sh b/utils/bin/osmo-gsm-tester_androidue_conn_chk.sh</span><br><span>new file mode 100644</span><br><span>index 0000000..7f0f79d</span><br><span>--- /dev/null</span><br><span>+++ b/utils/bin/osmo-gsm-tester_androidue_conn_chk.sh</span><br><span>@@ -0,0 +1,8 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/bin/bash</span><br><span style="color: hsl(120, 100%, 40%);">+# This script reads the network type of an Android phone via ADB</span><br><span style="color: hsl(120, 100%, 40%);">+# usage: osmo-gsm-tester_androidue_conn_chk.sh $serial</span><br><span style="color: hsl(120, 100%, 40%);">+serial=$1</span><br><span style="color: hsl(120, 100%, 40%);">+while true; do</span><br><span style="color: hsl(120, 100%, 40%);">+ sudo adb -s "${serial}" shell getprop "gsm.network.type";</span><br><span style="color: hsl(120, 100%, 40%);">+ sleep 1;</span><br><span style="color: hsl(120, 100%, 40%);">+done</span><br><span>diff --git a/utils/bin/osmo-gsm-tester_androidue_diag_parser.sh b/utils/bin/osmo-gsm-tester_androidue_diag_parser.sh</span><br><span>new file mode 100644</span><br><span>index 0000000..7d1ea3f</span><br><span>--- /dev/null</span><br><span>+++ b/utils/bin/osmo-gsm-tester_androidue_diag_parser.sh</span><br><span>@@ -0,0 +1,16 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/bin/bash</span><br><span style="color: hsl(120, 100%, 40%);">+# This script pulls the diag folder created by diag_mdlog and parses the</span><br><span style="color: hsl(120, 100%, 40%);">+# .qmdl file. Further, it writes all packets to a pcap file.</span><br><span style="color: hsl(120, 100%, 40%);">+# usage: osmo-gsm-tester_androidue_diag_parser.sh $serial $run_dir $pcap_path</span><br><span style="color: hsl(120, 100%, 40%);">+serial=$1</span><br><span style="color: hsl(120, 100%, 40%);">+run_dir=$2</span><br><span style="color: hsl(120, 100%, 40%);">+pcap_path=$3</span><br><span style="color: hsl(120, 100%, 40%);">+while true; do</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "Pulling new .qmdl file...";</span><br><span style="color: hsl(120, 100%, 40%);">+ sudo adb -s "${serial}" pull /data/local/tmp/diag_logs "${run_dir}" > /dev/null;</span><br><span style="color: hsl(120, 100%, 40%);">+ wait $!;</span><br><span style="color: hsl(120, 100%, 40%);">+ QMDL_FN=$(find "${run_dir}" -maxdepth 2 -type f -name "*.qmdl");</span><br><span style="color: hsl(120, 100%, 40%);">+ wait $!;</span><br><span style="color: hsl(120, 100%, 40%);">+ sudo scat -t qc --event -d "$QMDL_FN" -F "${pcap_path}";</span><br><span style="color: hsl(120, 100%, 40%);">+ wait $!;</span><br><span style="color: hsl(120, 100%, 40%);">+done</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21302">change 21302</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21302"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I79a5d803e869a868d4dac5e0d4c2feb38038dc5c </div>
<div style="display:none"> Gerrit-Change-Number: 21302 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: ninjab3s <nils.fuerste@softwareradiosystems.com> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>