<p>pespin <strong>submitted</strong> this change.</p><p><a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/17756">View Change</a></p><div style="white-space:pre-wrap">Approvals:
Jenkins Builder: Verified
srs_andre: Looks good to me, approved
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">amarisoft_ue: Implement data plane<br><br>Use the ifup script to set up the netns + configure the tun device<br>created by lteue. Use it also as a hook to know when the UE is attached.<br>Since tun setup is done by arch-optimized lte-avx(2) binaries, we also<br>need to give capabilitites to them (instead of allowing inheritance of<br>caps in general in the setcap script).<br><br>Change-Id: I1e228b1296eac8e4cb183c2faa735b0468abe124<br>---<br>M src/osmo_gsm_tester/amarisoft_ue.py<br>M src/osmo_gsm_tester/templates/amarisoft_lteue.cfg.tmpl<br>M utils/bin/osmo-gsm-tester_netns_setup.sh<br>3 files changed, 54 insertions(+), 18 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/amarisoft_ue.py b/src/osmo_gsm_tester/amarisoft_ue.py</span><br><span>index 9bfc182..422f28d 100644</span><br><span>--- a/src/osmo_gsm_tester/amarisoft_ue.py</span><br><span>+++ b/src/osmo_gsm_tester/amarisoft_ue.py</span><br><span>@@ -69,6 +69,7 @@</span><br><span> CFGFILE = 'amarisoft_lteue.cfg'</span><br><span> CFGFILE_RF = 'amarisoft_rf_driver.cfg'</span><br><span> LOGFILE = 'lteue.log'</span><br><span style="color: hsl(120, 100%, 40%);">+ IFUPFILE = 'ue-ifup'</span><br><span> </span><br><span> def __init__(self, suite_run, conf):</span><br><span> self._addr = conf.get('addr', None)</span><br><span>@@ -81,6 +82,7 @@</span><br><span> self._bin_prefix = None</span><br><span> self.config_file = None</span><br><span> self.config_rf_file = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ifup_file = None</span><br><span> self.log_file = None</span><br><span> self.process = None</span><br><span> self.rem_host = None</span><br><span>@@ -88,6 +90,7 @@</span><br><span> self.remote_config_file = None</span><br><span> self.remote_config_rf_file = None</span><br><span> self.remote_log_file = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_ifup_file = None</span><br><span> self.suite_run = suite_run</span><br><span> self.remote_user = conf.get('remote_user', None)</span><br><span> if not rf_type_valid(conf.get('rf_dev_type', None)):</span><br><span>@@ -138,17 +141,16 @@</span><br><span> remote_binary = self.remote_inst.child('', AmarisoftUE.BINFILE)</span><br><span> # setting capabilities will later disable use of LD_LIBRARY_PATH from ELF loader -> modify RPATH instead.</span><br><span> self.log('Setting RPATH for ltetue')</span><br><span style="color: hsl(0, 100%, 40%);">- # amarisoftue binary needs patchelf >= 0.9+52 to avoid failing during patch. OS#4389, patchelf-GH#192.</span><br><span style="color: hsl(120, 100%, 40%);">+ # patchelf >= 0.10 is required to support passing several files at once:</span><br><span> self.rem_host.set_remote_env({'PATCHELF_BIN': '/opt/bin/patchelf-v0.10' })</span><br><span> self.rem_host.change_elf_rpath(remote_binary, str(self.remote_inst))</span><br><span style="color: hsl(120, 100%, 40%);">+ # We also need to patch the arch-optimized binaries that lteue() will exec() into:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.change_elf_rpath(self.remote_inst.child('', 'lteue-*'), str(self.remote_inst))</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- # amarisoftue requires CAP_SYS_ADMIN to cjump to net network namespace: netns(CLONE_NEWNET):</span><br><span style="color: hsl(0, 100%, 40%);">- # amarisoftue requires CAP_NET_ADMIN to create tunnel devices: ioctl(TUNSETIFF):</span><br><span style="color: hsl(0, 100%, 40%);">- self.log('Applying CAP_SYS_ADMIN+CAP_NET_ADMIN capability to ltetue')</span><br><span style="color: hsl(0, 100%, 40%);">- self.rem_host.setcap_netsys_admin(remote_binary)</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- self.log('Creating netns %s' % self.netns())</span><br><span style="color: hsl(0, 100%, 40%);">- self.rem_host.create_netns(self.netns())</span><br><span style="color: hsl(120, 100%, 40%);">+ # lteue requires CAP_NET_ADMIN to create tunnel devices: ioctl(TUNSETIFF):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log('Applying CAP_NET_ADMIN capability to ltetue')</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.setcap_net_admin(remote_binary)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.setcap_net_admin(self.remote_inst.child('', 'lteue-*'))</span><br><span> </span><br><span> args = (remote_binary, self.remote_config_file)</span><br><span> self.process = self.rem_host.RemoteProcess(AmarisoftUE.BINFILE, args)</span><br><span>@@ -162,14 +164,13 @@</span><br><span> # setting capabilities will later disable use of LD_LIBRARY_PATH from ELF loader -> modify RPATH instead.</span><br><span> self.log('Setting RPATH for lteue')</span><br><span> util.change_elf_rpath(binary, util.prepend_library_path(self.inst), self.run_dir.new_dir('patchelf'))</span><br><span style="color: hsl(120, 100%, 40%);">+ # We also need to patch the arch-optimized binaries that lteue() will exec() into:</span><br><span style="color: hsl(120, 100%, 40%);">+ util.change_elf_rpath(self.inst.child('', 'lteue-*'), util.prepend_library_path(self.inst), self.run_dir.new_dir('patchelf2'))</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- # amarisoftue requires CAP_SYS_ADMIN to cjump to net network namespace: netns(CLONE_NEWNET):</span><br><span style="color: hsl(0, 100%, 40%);">- # amarisoftue requires CAP_NET_ADMIN to create tunnel devices: ioctl(TUNSETIFF):</span><br><span style="color: hsl(0, 100%, 40%);">- self.log('Applying CAP_SYS_ADMIN+CAP_NET_ADMIN capability to lteue')</span><br><span style="color: hsl(0, 100%, 40%);">- util.setcap_netsys_admin(binary, self.run_dir.new_dir('setcap_netsys_admin'))</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- self.log('Creating netns %s' % self.netns())</span><br><span style="color: hsl(0, 100%, 40%);">- util.create_netns(self.netns(), self.run_dir.new_dir('create_netns'))</span><br><span style="color: hsl(120, 100%, 40%);">+ # lteue requires CAP_NET_ADMIN to create tunnel devices: ioctl(TUNSETIFF):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.log('Applying CAP_NET_ADMIN capability to lteue')</span><br><span style="color: hsl(120, 100%, 40%);">+ util.setcap_net_admin(binary, self.run_dir.new_dir('setcap_net_admin'))</span><br><span style="color: hsl(120, 100%, 40%);">+ util.setcap_net_admin(self.inst.child('', 'lteue-*'), self.run_dir.new_dir('setcap_net_admin2'))</span><br><span> </span><br><span> args = (binary, os.path.abspath(self.config_file))</span><br><span> self.dbg(run_dir=self.run_dir, binary=binary, env=env)</span><br><span>@@ -192,6 +193,24 @@</span><br><span> self.config_file = self.run_dir.child(AmarisoftUE.CFGFILE)</span><br><span> self.config_rf_file = self.run_dir.child(AmarisoftUE.CFGFILE_RF)</span><br><span> self.log_file = self.run_dir.child(AmarisoftUE.LOGFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ifup_file = self.run_dir.new_file(AmarisoftUE.IFUPFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+ os.chmod(self.ifup_file, 0o744) # add execution permission</span><br><span style="color: hsl(120, 100%, 40%);">+ with open(self.ifup_file, 'w') as f:</span><br><span style="color: hsl(120, 100%, 40%);">+ r = '''#!/bin/sh</span><br><span style="color: hsl(120, 100%, 40%);">+ set -x -e</span><br><span style="color: hsl(120, 100%, 40%);">+ ue_id="$1" # UE ID</span><br><span style="color: hsl(120, 100%, 40%);">+ pdn_id="$2" # PDN unique id (start from 0)</span><br><span style="color: hsl(120, 100%, 40%);">+ ifname="$3" # Interface name</span><br><span style="color: hsl(120, 100%, 40%);">+ ipv4_addr="$4" # IPv4 address</span><br><span style="color: hsl(120, 100%, 40%);">+ ipv4_dns="$5" # IPv4 DNS</span><br><span style="color: hsl(120, 100%, 40%);">+ ipv6_local_addr="$6" # IPv6 local address</span><br><span style="color: hsl(120, 100%, 40%);">+ ipv6_dns="$7" # IPv6 DNS</span><br><span style="color: hsl(120, 100%, 40%);">+ old_link_local=""</span><br><span style="color: hsl(120, 100%, 40%);">+ # script + sudoers file available in osmo-gsm-tester.git/utils/{bin,sudoers.d}</span><br><span style="color: hsl(120, 100%, 40%);">+ sudo /usr/local/bin/osmo-gsm-tester_netns_setup.sh "%s" "$ifname" "$ipv4_addr"</span><br><span style="color: hsl(120, 100%, 40%);">+ echo "${ue_id}: netns %s configured"</span><br><span style="color: hsl(120, 100%, 40%);">+ ''' % (self.netns(), self.netns())</span><br><span style="color: hsl(120, 100%, 40%);">+ f.write(r)</span><br><span> </span><br><span> if not self.setup_runs_locally():</span><br><span> self.rem_host = remote.RemoteHost(self.run_dir, self.remote_user, self._addr)</span><br><span>@@ -202,6 +221,7 @@</span><br><span> self.remote_config_file = remote_run_dir.child(AmarisoftUE.CFGFILE)</span><br><span> self.remote_config_rf_file = remote_run_dir.child(AmarisoftUE.CFGFILE_RF)</span><br><span> self.remote_log_file = remote_run_dir.child(AmarisoftUE.LOGFILE)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.remote_ifup_file = remote_run_dir.child(AmarisoftUE.IFUPFILE)</span><br><span> </span><br><span> values = dict(ue=config.get_defaults('amarisoft'))</span><br><span> config.overlay(values, dict(ue=config.get_defaults('amarisoftue')))</span><br><span>@@ -211,7 +231,9 @@</span><br><span> config.overlay(values, dict(ue=dict(num_antennas = self.enb.num_ports())))</span><br><span> </span><br><span> logfile = self.log_file if self.setup_runs_locally() else self.remote_log_file</span><br><span style="color: hsl(0, 100%, 40%);">- config.overlay(values, dict(ue=dict(log_filename=logfile)))</span><br><span style="color: hsl(120, 100%, 40%);">+ ifupfile = self.ifup_file if self.setup_runs_locally() else self.remote_ifup_file</span><br><span style="color: hsl(120, 100%, 40%);">+ config.overlay(values, dict(ue=dict(log_filename=logfile,</span><br><span style="color: hsl(120, 100%, 40%);">+ ifup_filename=ifupfile)))</span><br><span> </span><br><span> # We need to set some specific variables programatically here to match IP addresses:</span><br><span> if self._conf.get('rf_dev_type') == 'zmq':</span><br><span>@@ -263,9 +285,12 @@</span><br><span> self.rem_host.recreate_remote_dir(remote_run_dir)</span><br><span> self.rem_host.scp('scp-cfg-to-remote', self.config_file, self.remote_config_file)</span><br><span> self.rem_host.scp('scp-cfg-rf-to-remote', self.config_rf_file, self.remote_config_rf_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rem_host.scp('scp-ifup-to-remote', self.ifup_file, self.remote_ifup_file)</span><br><span> </span><br><span> def is_connected(self, mcc_mnc=None):</span><br><span style="color: hsl(0, 100%, 40%);">- return 'Network attach successful.' in (self.process.get_stdout() or '')</span><br><span style="color: hsl(120, 100%, 40%);">+ # lteue doesn't call the ifup script until after it becomes attached, so</span><br><span style="color: hsl(120, 100%, 40%);">+ # simply look for our ifup script output at the end of it:</span><br><span style="color: hsl(120, 100%, 40%);">+ return 'netns %s configured' % (self.netns()) in (self.process.get_stdout() or '')</span><br><span> </span><br><span> def is_attached(self):</span><br><span> return self.is_connected()</span><br><span>diff --git a/src/osmo_gsm_tester/templates/amarisoft_lteue.cfg.tmpl b/src/osmo_gsm_tester/templates/amarisoft_lteue.cfg.tmpl</span><br><span>index 59ab8b2..720c0ba 100644</span><br><span>--- a/src/osmo_gsm_tester/templates/amarisoft_lteue.cfg.tmpl</span><br><span>+++ b/src/osmo_gsm_tester/templates/amarisoft_lteue.cfg.tmpl</span><br><span>@@ -55,7 +55,7 @@</span><br><span> //external_sim: true,</span><br><span> </span><br><span> /* Enable it to create a TUN interface for each UE PDN */</span><br><span style="color: hsl(0, 100%, 40%);">- //tun_setup_script: "ue-ifup",</span><br><span style="color: hsl(120, 100%, 40%);">+ tun_setup_script: "${ue.ifup_filename}",</span><br><span> </span><br><span> sim_events: [{</span><br><span> event: "power_on",</span><br><span>diff --git a/utils/bin/osmo-gsm-tester_netns_setup.sh b/utils/bin/osmo-gsm-tester_netns_setup.sh</span><br><span>index e3d09ea..c53e41a 100755</span><br><span>--- a/utils/bin/osmo-gsm-tester_netns_setup.sh</span><br><span>+++ b/utils/bin/osmo-gsm-tester_netns_setup.sh</span><br><span>@@ -2,6 +2,7 @@</span><br><span> </span><br><span> netns="$1"</span><br><span> ifname="$2" # optional</span><br><span style="color: hsl(120, 100%, 40%);">+ip_addr="$3" # optional</span><br><span> </span><br><span> if [ -f "/var/run/netns/${netns}" ]; then</span><br><span> echo "netns $netns already exists"</span><br><span>@@ -20,3 +21,13 @@</span><br><span> else</span><br><span> ip netns exec $netns ls "/sys/class/net/${ifname}" >/dev/null && echo "iface $ifname already in netns $netns"</span><br><span> fi</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+if [ "x$ip_addr" = "x" ]; then</span><br><span style="color: hsl(120, 100%, 40%);">+ exit 0</span><br><span style="color: hsl(120, 100%, 40%);">+fi</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+echo "Setting up iface $ifname with ${ip_addr}"</span><br><span style="color: hsl(120, 100%, 40%);">+ip netns exec $netns ip link set dev $ifname up</span><br><span style="color: hsl(120, 100%, 40%);">+ip netns exec $netns ip addr add ${ip_addr}/24 dev $ifname</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+#ip netns exec $netns ip route add default via ${ip_addr}</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/17756">change 17756</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/17756"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I1e228b1296eac8e4cb183c2faa735b0468abe124 </div>
<div style="display:none"> Gerrit-Change-Number: 17756 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins Builder </div>
<div style="display:none"> Gerrit-Reviewer: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: srs_andre <andre@softwareradiosystems.com> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>