pespin has uploaded this change for review.
WIP: doc: Document MTU considerations
Related: OS#6298
Related: SYS#7122
Change-Id: Ib6e974b38107fe48072380f768e1881f0fc95e80
---
M doc/examples/osmo-ggsn-kernel-gtp.cfg
M doc/examples/osmo-ggsn.cfg
M doc/manuals/chapters/configuration.adoc
M doc/manuals/chapters/running.adoc
4 files changed, 89 insertions(+), 12 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-ggsn refs/changes/00/38500/1
diff --git a/doc/examples/osmo-ggsn-kernel-gtp.cfg b/doc/examples/osmo-ggsn-kernel-gtp.cfg
index bed52b9..69ab51b 100644
--- a/doc/examples/osmo-ggsn-kernel-gtp.cfg
+++ b/doc/examples/osmo-ggsn-kernel-gtp.cfg
@@ -44,6 +44,7 @@
gtpu-mode kernel-gtp
tun-device tun4
type-support v4
+ mtu default apply
ip prefix dynamic 172.16.222.0/24
ip dns 0 8.8.8.8
ip dns 1 8.8.4.4
diff --git a/doc/examples/osmo-ggsn.cfg b/doc/examples/osmo-ggsn.cfg
index 53cebc1..52b1034 100644
--- a/doc/examples/osmo-ggsn.cfg
+++ b/doc/examples/osmo-ggsn.cfg
@@ -44,6 +44,7 @@
gtpu-mode tun
tun-device tun4
type-support v4
+ mtu default apply
ip prefix dynamic 172.16.222.0/24
ip dns 0 8.8.8.8
ip dns 1 8.8.4.4
@@ -53,6 +54,7 @@
gtpu-mode tun
tun-device tun6
type-support v6
+ mtu default apply
ipv6 prefix dynamic 2001:780:44:2000:0:0:0:0/56
ipv6 dns 0 2001:4860:4860::8888
ipv6 dns 1 2001:4860:4860::8844
@@ -62,6 +64,7 @@
gtpu-mode tun
tun-device tun46
type-support v4v6
+ mtu default apply
ip prefix dynamic 172.16.46.0/24
ip dns 0 8.8.8.8
ip dns 1 8.8.4.4
diff --git a/doc/manuals/chapters/configuration.adoc b/doc/manuals/chapters/configuration.adoc
index f0889d3..2dfe0d3 100644
--- a/doc/manuals/chapters/configuration.adoc
+++ b/doc/manuals/chapters/configuration.adoc
@@ -1,3 +1,4 @@
+[[osmoggsn_configuring]]
== Configuring OsmoGGSN
All configuration of OsmoGGSN is performed using the VTY. For more
@@ -22,6 +23,7 @@
gtpu-mode tun
tun-device tun4
type-support v4
+ mtu default apply
ip prefix dynamic 176.16.222.0/24
ip dns 0 192.168.100.1
ip dns 1 8.8.8.8
@@ -162,18 +164,20 @@
----
OsmoGGSN(config-ggsn-apn)# gtpu-mode tun <1>
OsmoGGSN(config-ggsn-apn)# type-support v4 <2>
-OsmoGGSN(config-ggsn-apn)# ip prefix dynamic 176.16.222.0/24 <3>
-OsmoGGSN(config-ggsn-apn)# ip dns 0 192.168.100.1 <4>
-OsmoGGSN(config-ggsn-apn)# ip dns 1 8.8.8.8 <5>
-OsmoGGSN(config-ggsn-apn)# ip ifconfig 176.16.222.0/24 <6>
+OsmoGGSN(config-ggsn-apn)# mtu 1420 apply <3>
+OsmoGGSN(config-ggsn-apn)# ip prefix dynamic 176.16.222.0/24 <4>
+OsmoGGSN(config-ggsn-apn)# ip dns 0 192.168.100.1 <5>
+OsmoGGSN(config-ggsn-apn)# ip dns 1 8.8.8.8 <6>
+OsmoGGSN(config-ggsn-apn)# ip ifconfig 176.16.222.0/24 <7>
----
<1> Use the userspace GTP-U handling using a TUN device
<2> Support (only) IPv4 Addresses
-<3> Specify the pool of dynamic IPv4 addresses to be allocated to PDP
+<2> Specify MTU to announce to MS. Apply the MTU on the tunnel interface.
+<4> Specify the pool of dynamic IPv4 addresses to be allocated to PDP
contexts
-<4> Specify the primary DNS server to be provided using IPCP/PCO
-<5> Specify the secondary DNS server to be provided using IPCP/PCO
-<6> Request OsmoGGSN to configure the `tun4` device network/netmask
+<5> Specify the primary DNS server to be provided using IPCP/PCO
+<6> Specify the secondary DNS server to be provided using IPCP/PCO
+<7> Request OsmoGGSN to configure the `tun4` device network/netmask
NOTE:: If you use the optional `ip ifconfig` command to set the network
device address/mask, OsmoGGSN must run with root or `CAP_NET_ADMIN`
@@ -271,6 +275,7 @@
----
# ip addr add 192.168.7.1/24 dev apn0
+# ip link set mtu 1420 dev apn0
# ip link set apn0 up
----
@@ -298,13 +303,18 @@
[Match]
Name=apn0 <1>
+[Link]
+MTUBytes=1420 <2>
+
[Network]
-Address=192.168.7.1/24 <2>
-IPMasquerade=yes <3>
+Address=192.168.7.1/24 <3>
+IPMasquerade=yes <4>
----
<1> The network device name, which must match the one in the apn0.netdev unit file above
-<2> The local IP address configured on the device
-<3> Requesting systemd to configure IP masquerading for this interface. Depending on your needs,
+<2> Requesting systemd to set the MTU for this interface. The MTU of the tun
+ interface should be lower than regular, since it must accommodate the extra IP/UDP/GTPv1U headers.
+<3> The local IP address configured on the device
+<4> Requesting systemd to configure IP masquerading for this interface. Depending on your needs,
You may not want this if you have proper end-to-end routing set up, and want to have transparent
inbound IP access to your GPRS-attached devices.
@@ -326,6 +336,7 @@
gtpu-mode tun
tun-device apn0
type-support v4
+ mtu 1420
ip prefix dynamic 192.168.7.0/24
ip dns 0 192.168.100.1
ip dns 1 8.8.8.8
diff --git a/doc/manuals/chapters/running.adoc b/doc/manuals/chapters/running.adoc
index 04c5b06..2371ec3 100644
--- a/doc/manuals/chapters/running.adoc
+++ b/doc/manuals/chapters/running.adoc
@@ -52,6 +52,68 @@
your distribution's documentation -- e.g. look for @net.ipv4.ip_forward=1@ in
@/etc/sysctl.d/@, and https://wiki.debian.org/iptables for masquerading.
+=== MTU considerations
+
+When running osmo-ggsn, the user must take network Maximum Transmission Unit
+(MTU) into consideration, and configure it based on network specific setup.
+
+In osmo-ggsn, the MTU can be configured per APN through the VTY, see <<osmoggsn_configuring>>.
+If told so by the config, osmo-ggsn will apply the MTU on the APN network interface.
+
+==== MTU announced to MS
+
+The configured MTU is also announced to the MS through:
+
+* IPv4 APN: GTPv1C Create PDP Context Response, PCO IE "IPv4 Link MTU", 3GPP TS 24.008 Table 10.5.154.
+* IPv6 APN: ICMPv6 Routing Advertisement during IPv6 SLAAC procedure, RFC 4861.
+
+==== Figuring out maximum MTU value
+
+There is no golden MTU value, since it really depends on the local (and remote) networks where traffic is routed.
+The idea is finding out a value which:
+* Is as big as possible, to avoid need to split big chunks of data into lots of small packets, hence affecting performance due to processing overhead: extra headers being trnasmitted, plus processing of extra packets.
+* Is small enough so that it can be transported over the lower layers of the links involving the communication, avoiding IP fragmentation, which again hurts performance.
+
+OsmoGGSN, by default, announces an MTU of `1420` bytes. This is calculated on the assumption that traffic is transported over Ethernet, which has a usual MTU of 1500 bytes. On top of that, it must account for the IP/UDP/GTPv1U headers used to encapsulate the traffic sent towards the RAN. Hence the following bytes are subtracted from the Ethernet base MTU of 1500 bytes:
+* IP: IPv4 headers can take up to 60 bytes (due to IPv4 options). IPv6 headers can take up to 40 bytes (assuming no extension headers for IPv6 in general, since they are uncommon). Hence, the 60 bytes of IPv4 are picked since that's greater than the IPv4.
+* UDP: The UDP header takes 8 bytes.
+* GTPv1U: The GTPv1U header takes 12 bytes, assuming here no extensions headers are used (OsmoGGSN doesn't use them).
+Hence, `TUNNEL_MTU = ETH_MTU - IP_HDR - UDP_HDR - GTPv1U_HDR = 1500 - 60 - 8 - 12 = 1420`.
+
+Under certain networks, the base MTU may already be smaller than Ethernet's MTU (1500 bytes), due to, for instance, existence of some sort of extra tunneling protocol in the path, such as a VPN, ipsec, extra VLAN levels, etc. Under this scenario, the user must take care of figuring out the new base MTU value to use for the calculations presented above. This can be accomplished by packet inspection (eg. `wireshark`) or with tools such as `ping`, running it with a certain packet size and the IPv4 DF bit set, and see up to which packet size the networks is able to forward the message. For instance:
+
+.Example: Test if packets of 1420 bytes can reach peer host 176.16.222.4
+----
+ping -M probe 176.16.222.4 -s 1420
+----
+
+==== TCP MSS Clamping
+
+Usually endpoints use Path MTU Discovery (PMTUD) to determine the maximum MTU to reach the peer.
+However, this technique may sometimes not be successful for all users of OsmOGGSN:
+* MS may not support requesting and/or configuring the MTU OsmoGGSN announced.
+* MS may not support PMTUD on its network stack, or may not have it enabled or may be buggy.
+* Network may be misconfigured or some middlebox may be buggy (eg. not forwarding ICMP `Packet Too Big` packets).
+
+Furthermore, PMTUD takes time to figure out the maximum MTU to use, since it relies on sending data and checking if it got lost, and adapting to the fact, reducing efficiency (throughput) of connections or even stalling them completely when big packets are generated.
+
+TODO:
+- Explain what's TCP MSS
+- Explain what's TCP MSS clamping and why is it useful in this case (we are reducing MTU from usual routes due to encapsulating traffic)
+- Show an example with iptables on how to configure TCP MSS.
+
+==== Further Reading
+
+For more related specs, check the following specs:
+
+* 3GPP TS 29.061 section 11.2.1.5
+* 3GPP TS 290.060 section 13.2 IP Fragmentation
+* 3GPP TS 25.414 section 6.1.3.3
+* 3GPP TS 23.060 section 9.3, Annex C
+* 3GPP TS 24.008 (PCO IPv4 MTU)
+* RFC 4861 (IPv6 Router Advertisement)
+
+
=== Multiple instances
Running multiple instances of `osmo-ggsn` is possible if all GGSN instances
To view, visit change 38500. To unsubscribe, or for help writing mail filters, visit settings.