pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/upf-benchmark/+/39890?usp=email )

Change subject: testsuites: upf: Support setting net iface irqs in c240 host ......................................................................

testsuites: upf: Support setting net iface irqs in c240 host

The Cisco C240 machines consist of 2 CPU packages, each with 20 cores (hyperthreading disabled). Those 2 CPUs packages are hence placed in 2 different NUMA zones.

By default, mlx5 driver creates one rx-queue per core, in this case it creates 40 rx-queues with 40 irqs (one for each rx-queue/core). This means it doesn't take into account the fact that the network card is plugged into a given PCIe bus belonging to one of the 2 available NUMA zones. As a result, when a packet is received and put into the rx-queue+irq belonging to the other NUMA zone, a penalty in memory-bandwitch is incurred, which ends up in a performance throughput penalty.

This commit is similar to 224e2dc3ab67311b4bad94bf891d1b5c5ca6128f applied to the tunmap scenario.

Change-Id: Ia4c006910e946e22ab375e9349940f1f756e7800 --- M testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh M testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh 2 files changed, 62 insertions(+), 10 deletions(-)

git pull ssh://gerrit.osmocom.org:29418/upf-benchmark refs/changes/90/39890/1

diff --git a/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh b/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh index c4a7ae6..319485d 100755 --- a/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh +++ b/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh @@ -1,12 +1,38 @@ #!/bin/bash -xe

+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +NUM_ENB=100 # Interface towards UPF: ADDR_GW_CN="172.16.31.200" -NUM_ENB=100 -IFACE_RAN="enp2s0f1np1" -IFACE_CN="enp2s0f0np0" -MAC_RAN_PEER="ec:0d:9a:8a:27:52" -MAC_CN_PEER="ec:0d:9a:8a:27:53" + +case $(hostname) in +gtplab1) + IFACE_RAN="enp2s0f1np1" + IFACE_CN="enp2s0f0np0" + MAC_RAN_PEER="ec:0d:9a:8a:27:52" + MAC_CN_PEER="ec:0d:9a:8a:27:53" + ;; +c240-2) + IFACE_RAN="ens2f0np0" + IFACE_CN="ens2f1np1" + MAC_RAN_PEER="b8:ce:f6:fe:62:e0" + MAC_CN_PEER="b8:ce:f6:fe:62:e1" + # Adapt Rx/Tx Queues and IRQs to Numa Node of device: + # cat /sys/class/net/$IFACE_RAN/device/numa_node + # 0 + # cat /sys/class/net/$IFACE_RAN/device/local_cpulist + # 0-19 + sudo ethtool -L $IFACE_RAN combined 20 + sudo ethtool -L $IFACE_CN combined 20 + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_RAN + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_CN + ;; +*) + echo "UNEXPECTED HOSTNAME: $(hostname)" + exit 1 + ;; +esac

# Disable ethernet flow control: sudo ethtool -A $IFACE_RAN autoneg off rx off tx off diff --git a/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh b/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh index 613caf2..1a12b80 100755 --- a/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh +++ b/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh @@ -1,13 +1,39 @@ #!/bin/bash -xe

+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +NUM_ENB=100 + # Interface towards UPF: IFACE_APN="apn-flood" ADDR_GW_CN="172.16.31.200" -NUM_ENB=100 -IFACE_RAN="enp2s0f1np1" -IFACE_CN="enp2s0f0np0" -MAC_RAN_PEER="88:e9:a4:3b:f0:48" -MAC_CN_PEER="88:e9:a4:3b:f0:49" +case $(hostname) in +gtplab1) + IFACE_RAN="enp2s0f1np1" + IFACE_CN="enp2s0f0np0" + MAC_RAN_PEER="ec:0d:9a:8a:27:52" + MAC_CN_PEER="ec:0d:9a:8a:27:53" + ;; +c240-2) + IFACE_RAN="ens2f0np0" + IFACE_CN="ens2f1np1" + MAC_RAN_PEER="b8:ce:f6:fe:62:e0" + MAC_CN_PEER="b8:ce:f6:fe:62:e1" + # Adapt Rx/Tx Queues and IRQs to Numa Node of device: + # cat /sys/class/net/$IFACE_RAN/device/numa_node + # 0 + # cat /sys/class/net/$IFACE_RAN/device/local_cpulist + # 0-19 + sudo ethtool -L $IFACE_RAN combined 20 + sudo ethtool -L $IFACE_CN combined 20 + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_RAN + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_CN + ;; +*) + echo "UNEXPECTED HOSTNAME: $(hostname)" + exit 1 + ;; +esac

# Disable ethernet flow control: sudo ethtool -A $IFACE_RAN autoneg off rx off tx off