pespin has submitted this change. ( https://gerrit.osmocom.org/c/upf-benchmark/+/39890?usp=email ) Change subject: testsuites: upf: Support setting net iface irqs in c240 host ...................................................................... testsuites: upf: Support setting net iface irqs in c240 host The Cisco C240 machines consist of 2 CPU packages, each with 20 cores (hyperthreading disabled). Those 2 CPUs packages are hence placed in 2 different NUMA zones. By default, mlx5 driver creates one rx-queue per core, in this case it creates 40 rx-queues with 40 irqs (one for each rx-queue/core). This means it doesn't take into account the fact that the network card is plugged into a given PCIe bus belonging to one of the 2 available NUMA zones. As a result, when a packet is received and put into the rx-queue+irq belonging to the other NUMA zone, a penalty in memory-bandwitch is incurred, which ends up in a performance throughput penalty. This commit is similar to 224e2dc3ab67311b4bad94bf891d1b5c5ca6128f applied to the tunmap scenario. Change-Id: Ia4c006910e946e22ab375e9349940f1f756e7800 --- M testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh M testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh 2 files changed, 62 insertions(+), 10 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, but someone else must approve pespin: Looks good to me, approved diff --git a/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh b/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh index c4a7ae6..eede062 100755 --- a/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh +++ b/testsuites/gtplab-sysmo2025/tunend/upf/eupf/configure.sh @@ -1,12 +1,38 @@ #!/bin/bash -xe +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +NUM_ENB=100 # Interface towards UPF: ADDR_GW_CN="172.16.31.200" -NUM_ENB=100 -IFACE_RAN="enp2s0f1np1" -IFACE_CN="enp2s0f0np0" -MAC_RAN_PEER="ec:0d:9a:8a:27:52" -MAC_CN_PEER="ec:0d:9a:8a:27:53" + +case $(hostname) in +gtplab1) + IFACE_RAN="enp2s0f1np1" + IFACE_CN="enp2s0f0np0" + MAC_RAN_PEER="ec:0d:9a:8a:27:52" + MAC_CN_PEER="ec:0d:9a:8a:27:53" + ;; +c240-2) + IFACE_RAN="ens2f0np0" + IFACE_CN="ens2f1np1" + MAC_RAN_PEER="88:e9:a4:3b:f0:48" + MAC_CN_PEER="88:e9:a4:3b:f0:49" + # Adapt Rx/Tx Queues and IRQs to Numa Node of device: + # cat /sys/class/net/$IFACE_RAN/device/numa_node + # 0 + # cat /sys/class/net/$IFACE_RAN/device/local_cpulist + # 0-19 + sudo ethtool -L $IFACE_RAN combined 20 + sudo ethtool -L $IFACE_CN combined 20 + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_RAN + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_CN + ;; +*) + echo "UNEXPECTED HOSTNAME: $(hostname)" + exit 1 + ;; +esac # Disable ethernet flow control: sudo ethtool -A $IFACE_RAN autoneg off rx off tx off diff --git a/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh b/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh index 613caf2..b574738 100755 --- a/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh +++ b/testsuites/gtplab-sysmo2025/tunend/upf/osmo-upf/configure.sh @@ -1,13 +1,39 @@ #!/bin/bash -xe +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +NUM_ENB=100 + # Interface towards UPF: IFACE_APN="apn-flood" ADDR_GW_CN="172.16.31.200" -NUM_ENB=100 -IFACE_RAN="enp2s0f1np1" -IFACE_CN="enp2s0f0np0" -MAC_RAN_PEER="88:e9:a4:3b:f0:48" -MAC_CN_PEER="88:e9:a4:3b:f0:49" +case $(hostname) in +gtplab1) + IFACE_RAN="enp2s0f1np1" + IFACE_CN="enp2s0f0np0" + MAC_RAN_PEER="ec:0d:9a:8a:27:52" + MAC_CN_PEER="ec:0d:9a:8a:27:53" + ;; +c240-2) + IFACE_RAN="ens2f0np0" + IFACE_CN="ens2f1np1" + MAC_RAN_PEER="88:e9:a4:3b:f0:48" + MAC_CN_PEER="88:e9:a4:3b:f0:49" + # Adapt Rx/Tx Queues and IRQs to Numa Node of device: + # cat /sys/class/net/$IFACE_RAN/device/numa_node + # 0 + # cat /sys/class/net/$IFACE_RAN/device/local_cpulist + # 0-19 + sudo ethtool -L $IFACE_RAN combined 20 + sudo ethtool -L $IFACE_CN combined 20 + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_RAN + sudo "$SCRIPT_DIR/../../../../../scripts/set_irq_affinity_cpulist.sh" 0-19 $IFACE_CN + ;; +*) + echo "UNEXPECTED HOSTNAME: $(hostname)" + exit 1 + ;; +esac # Disable ethernet flow control: sudo ethtool -A $IFACE_RAN autoneg off rx off tx off -- To view, visit https://gerrit.osmocom.org/c/upf-benchmark/+/39890?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: upf-benchmark Gerrit-Branch: master Gerrit-Change-Id: Ia4c006910e946e22ab375e9349940f1f756e7800 Gerrit-Change-Number: 39890 Gerrit-PatchSet: 2 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>