On Wed, 2019-04-10 at 13:35 +0700, Vadim Yanitskiy wrote: 2012! But, umm, I don't really have time for a whole video right now - anyone have the slides? :-) But yeah, the first slides look sensible :-) Not sure I get this, but I also don't really care all that much. It's just a pretty strange design if the kernel were to output this, I'm not even sure how I'd do that properly. I don't want to be generating UDP packets there... Perhaps we can define something (GSMTAPv3) to not really care how it's encapsulated, and for 'native' packet captures like what I want on Linux when integrated with the driver, actually use an ARPHDR_GSMTAP, and encapsulate in UDP when you create it in an application and want to send it elsewhere, rather than just writing it to a pcap file? johannes

Hi Marcel, Yes, that's true, though the control path is actually going through one of the data pipes as well. So the control path *itself* would be there, I guess, but ... there's definitely room for more information than _just_ the control path "chat", also application state etc. would be useful, and logging etc. Typically on wifi we feed all of this together into kernel tracing (to record with trace-cmd) rather than trying to encapsulate it some other way. Right. This is something we'd typically use tracing for in wifi. I don't really know what the right model for WWAN would be, I guess. Right now - and I really should've said this before - really the only problem I was thinking of was how we can mux multiple "chat" sessions with a device into a single data stream. Currently, this is all vendor-specific. If you have a Qualcomm modem, you'd be able to see all the open sessions on the underlying netdev, and the QMI header tells you what session a given 'packet' belongs to, and if you follow along maybe you can figure out if this is a control or IP 'packet' (could be an AT command). cdc_mbim uses VLAN tags instead to achieve this, and decapsulates the VLAN tags to send them down to the hardware in a different way. There are a few reasons why I think that this model of having a single underlying netdev controlled by the modem driver, with additional netdevs layered on top in software will not work right in the future. I think drivers should and will need to migrate to creating "real" netdevs for the sessions instead of pure software ones. But if you do this, you lose the ability to listen to all the session streams at the same time, you can only do it for each netdev. Adding this ability back seems worthwhile, but then we probably shouldn't do it in a vendor-specific way, but rather in a generic way. So basically right now that's all I'm trying to solve. In WiFi we don't have the problem of "sessions" because we just use the addresses in the frames to disambiguate - on such 'monitor' netdevs we see the frames including full 802.11 headers. Now, here's maybe where I'm getting off the right path - in wifi we mostly couple that with PHY information as well, and so we have PHY information + full 802.11 headers + data for capturing what's going on. I figured that theoretically at least that would be possible/useful for the modem as well (obviously control packets have no PHY data), but there doesn't seem to be any hardware that would actually expose data in this way. Everyone I've spoken to says these things are only available as modem trace data. What I haven't figured out though is if that's by some other design trade-off, or just because no such infrastructure is/was available. It may well be that doing kernel-tracing instead of doing it via some kind of monitor netdev is perfectly sufficient, but I have a feeling that the relative simplicity of starting tcpdump/wireshark might be preferable. johannes

On Fri, 2019-04-12 at 21:49 +0200, Marcel Holtmann wrote: That's true, but as far as I can tell in a lot of cases the pipe is actually still a virtual netdev (session, encoded in whatever way) using some sort of serial line format on top? But I agree that typically this will not be sufficient anyway. Sure. Looking forward though, perhaps tracing *would* indeed be the simplest thing to do. you just have a single netdev from the driver, it gets harder to do multiple TX/RX queues properly. Sure. Though some drivers/devices fake them anyway. Right. But you do - afaict there are no ways other than the netdevs to communicate with the device, and people do things like "socat" to set up PTYs and pretend to have serial channels there, on top of the netdevs? johannes

Hi Johannes, On Wed, Apr 10, 2019 at 09:23:13AM +0200, Johannes Berg wrote: Well, Osmocom is a very small community, with probably somewhere less than 25 active developers over the last few years (less than 15 full-time), with an *incredibly* large scope: Implement virtually any protocol layer of any protocol stack on any of the 3GPP interfaces and all their related network elements for 2G/3G as well as even other technologies like TETRA, GMR-1, ... And all that in a field of technology that has less free software than the Operating Systems world had in the mid-1990ies. It really feels a bit like the Linux community 20 years ago. So resources are always *extremely* tight, and given those limited resources, I'm actually very happy with the results by now, having automatied CI, build verifications, unit tests, functional test suites, end-to-end testing, and all the code we implemented on git.osmocom.org :) While current GSMTAPv2 is ugly, it works rather solid for all known existing use cases, so there was no urgency to introduce a new version of it. Well, with all respect, GSMTAP was created for a variety of use cases, see my other lengthy mail. It's fine if you don't care, but unless you could explain your use cases with a few paragraphs, neither you nor us are able to determine if there is common functionality and if it makes sense to use GSMTAP or not :) So far I have not seen any explanation about what kind of data you want to encapsulate at all. There are well-established APIs for having sockets in the kernel and for generating + receiving UDP packets from it. NFS has been doing this for decades, as do various kernel-side tunneling helpers including the GTP kernel module. I'm not saying it's the right approach for your problem, I'm just saying kernel-side code can for sure use UDP sockets. Sure, that works. But the real question is, to me: Are there common GSMTAP payload types that both the existing GSMTAP users carry, as well as what you would want to carry? If yes, then it makes sense to think about a common encapsulation like GSMTAP. If the payload types differ, then it seems rather like there are two distinct use cases that wouldn't benefit from standardizing on one format. Regards, Harald -- - Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)

On Apr 12, 2019, at 10:15 AM, Johannes Berg <johannes(a)sipsolutions.net> wrote: I see nothing about a struct gsmtap_hdr: http://osmocom.org/projects/baseband/wiki/GSMTAP that 1) requires that it plus the payload be encapsulated in a UDP datagram or 2) would prevent it from being at the beginning of a LINKTYPE_GSMTAP/DLT_GSMTAP packet in a pcap or pcapng file.

Dear Guy and others, On Fri, Apr 12, 2019 at 03:47:26PM -0700, Guy Harris wrote: I completely agree that there is no technical reason why one couldn't put GSMTAP headers in current (v2) or future formats inside an ETHERTYPE / DLT and have them natively appearing in a pcap file of the given DLT. There would be no objections from my side to do so. One could then simply call the same dissector in wireshark from that DLT or from the existing UDP port number based dispatch via the IANA-registered GSMTAP port number. However, among the existing users of GSMTAP in the last decade or so, there would be no advantage, as the related sources of GSMTAP frames all exist in userspace and are feeding data, particularly from potentially multiple sources, which can very well run on different hosts. But that of course doesn't prevent new users from using different transport mechanisms of getting GSMTAP from e.g. the kernel into userspace. Regards, Harald -- - Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)

Hi Johannes, On Fri, Apr 12, 2019 at 07:15:56PM +0200, Johannes Berg wrote: Sure, like with most protocols you can wrap them in anything you want. Let me put it like this: You don't have to run RTP inside UDP, you could equally put the RTP frames in to SCTP or DCTP. It's just not what the original users of the protocol/spec had envisioned, but it can for sure be done, and has no side-effect other than not being interoperable with existing implementations. So it's basically the information whch PDP/PDN context your user IP packet belongs to. A single integer "tag". that resembles the identifier that was used by the control plane when activating that context (which is basically a tunnel). I don't think there's much commonality here. A modem will "never" give you access to the actual cellular protocol layers that we care about in the Osmocom/srsLTE/YateBTS/OpenBTS/airprobe/gr-gsm/... world, for which GSMTAP was designed (see more below). I'm not saying it cannot be done. If you want to use GSMTAP, I'm happy to help. But at least so far, I don't see why it would make any sense. There are a couple of files attached at https://osmocom.org/projects/baseband/wiki/WiresharkIntegration GSMTAP was generated initially for GSM. GSM is a circuit-switched digital wireless telephony system that has no inherent/native way of transporting IP payload. As such, the GSMTAP Um captures linked above will not contain IP user data but will contain the signaling plane data of the Um air interface protocol stack, consisting of the LAPDm potocol on Layer2 and the TS 04.08 RR (Radio Resource), MM (Mobility Management) and CC (Call Control) data. Of course one can also use GSMTAP for GPRS, which is packet oriented. In this case, you have the following layering stack inside a GSMTAP Um frame: TCP IP SNDCP (TS 04.65) LLC (TS 04.64) RLC/MAC PHY There is segmentation/reassembly potentially at least at the LLC and at the RLC/MAC layer. There can be encryption at the LLC layer so the IP payload is already invisible at that point. You cab also have both header and payload compression at the SNDCP layer. In the end, you can have start/middle/end segments of LLC frames inside a single RLC/MAC block, belonging to either one or multiple LLC frames, and then the LLC frames contain [segments of] IP packets. GSMTAP can be used on various other interfaces of the cellular network, which are *not* the radio interface between modem/phone and base station (such as e.g. between the Phone and the SIM card), so they're not of interest to your use case and I'll not cover them here. GSMTAP can also be used to encapsulate later cellular technologies such as UMTS aka WCDMA aka 3G, or LTE aka 4G. In all those cases you always have a different (sometimes deeper and sometimes not so deep) protocol stack between the user IP payload and the actual radio interface (PHY). You can think of it a bit if you are not thinking about a User IP packet but you assume for the sake of an argument that you want to see "HTML". Then underneath HTML you either have, depending on the technoolgy, HTTP 1.0, 1.1 or 2.0. And under that you can have any number of additional protocol stacking whether it's TCP based, with or without SSL or TLS, with IPv3 or IPv6, with or without IPsec, inside a VLAN or not, ... - and most importantly, next to all of that you have important "control information" like let's say DHCP or neighbor discovery. So, in other words, the user IP plan is *very* far away from the PHY on the radio interface, and the really interesting things are those that are happening beneath or next to the user IP. However, commercially available cellular modems will go to the utmost extent to make sure you never have access to any of this, and they invent various different ways to make the user (whether that's ofono, the ModemManager, ...) as far away from that as possible. The interfaces between the cellular protocol stacks and the user (whether QMI, MBIM, AT commands, ...) are so abstract that virtually any useful resemblance to what happens on the actual radio interface is lost. You can get a glimpse to some of what's happening with Qualcomm DIAG protocol, but even that doesn't really [always] expose full protocol traces to you, particularly not on the user plane. Not really. The modem implements the entire protocol stacks for the various cellular technologies and in the end provides you as the user something like a tunnel endpoint. The most widespread technology to obtain tracing/debugging information is the Qualcomm DIAG protocol, which is a collection of dozens of sub-systems each of whcih has up to hundreds of individual flags/settings of categories that can be enabled and/or disabled. We once did a bit of development in that area, see osmo-qcdiag to enable this (http://git.osmocom.org/osmo-qcdiag/) as well as *extremely minimal* wireshark dissectors at http://git.osmocom.org/wireshark/log/?h=laforge/qcdiag However, this is a proprietary mechanism available only to Qualcomm stack based devices, will only work on those devices where one has figured out how to enable an access it. And the information it provides is completely asynchronous to the user IP plane. There is a similar project for (now very old) Infineon XGold based phones, see https://github.com/2b-as/xgoldmon. It also generates GSMTAP. But AFAIR only for the control plane data and not the user plane data. To be honest, I don't even think there's a lot of context that one could theoretically provide "attached" to the user IP packet, as the interesting bits are happening at lower protocol layers, and due to segmentation/reassembly/compression/... etc. there is no 1:1 mapping between the user IP packet and what's happening on the radio interface. Regards, Harald -- - Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)

On Apr 12, 2019, at 5:24 AM, Johannes Berg <johannes(a)sipsolutions.net> wrote: That's just because the rpcap protocol hasn't been revved yet to handle the new create/set options/activate mechanism in libpcap to allow monitor-mode capturing, so you only get to see fake Ethernet frames. :-) I.e., there's a split there between "capture" and "getting the packets from a capture delivered to you over an IP network". Perhaps there should be a GSMTAP link-layer header type, so you can have a GSMTAP pcap file or GSMTAP interface in a pcapng file, combined with a more general "remote capture" mechanism in libpcap so that you could capture on gsmtap://host:port and capture from a host using the GSMTAP-over-UDP encapsulation - or capture using rpcap. 1) You could pipe into libpcap or otherwise have a way for a libpcap module to connect to a user space source and get packets from it. 2) You could pipe a pcap file into tcpdump/Wireshark/etc.. 3) You could have an extcap program: https://www.wireshark.org/docs/wsdg_html_chunked/ChCaptureExtcap.html provide packets to Wireshark. Depending on how your system is set up: $ ls -l /dev/bpf* crw-rw---- 1 root access_bpf 23, 0 Apr 10 22:57 /dev/bpf0 crw-rw---- 1 root access_bpf 23, 1 Apr 10 22:56 /dev/bpf1 ... and it could just be rw-rw-rw-. Perhaps other systems make it harder to grant capture privileges. If there's a need for that, whatever provides the packets could impose that (by finding out the UID/GID of the other process if this is going over a UNIX-domain socket, at least on some UN*Xes, or by requiring a login if this is going over a network).

Hi Johannes, I'll only adress some of the points that haven't been clarified in other mails in this thread alreasdy: On Fri, Apr 12, 2019 at 02:24:25PM +0200, Johannes Berg wrote: understood. Agreed. However, the protocol stacks on WiFi and cellular technologies are very different, to say the least. There is no easy mapping of PHY related parameters to a given IP packet, and the related quality metrics of the radio channels don't work that way. But yes, I agree, that whatever transport mechanism you wan to use between the modem and the user/Linux side should allow for additional, extensible metadata beyond the identifier of the PDP/PDN context. This is because your 802.11 implementations are in hardware, while the open source GSM/UMTS/LTE implementations using GSMTAP are implemented in userspace - or at the very least the higher-level parts of the protocol stacks, or the tracing frameworks that are used to generate the GSMTAP data from some hardware. But why? I'm in an userspace program, and I want to send data to one or more other userspace programs. Why would I not simply open a UDP socket to do so? I would have to have CAP_NET_RAW and open a packet socket, and then generate ethernet frames from that? I admit that the use case with wireshark is a bit odd, but there are other receivers out there. Also, for debugging cellular network elements, it's very useful if you can look at the RAN or core network interfaces (Abis, Gb, Gp, A, Iu, S1, ...) in the same protocol trace in which you also get traces from the radio interface in GSMTAP. Then you can see what happens at each and every interface / network element in one time-line. And as the Osmcoom programs also allow generating log output wrapped in GSMTAP, you get not only protocol traces of all the interfaces, but even textual log information (with machine-readable log level + sub-system) information all in one capture/timeline. The use cases differ, but having UDP encapsulation enables a lot of flexibility and has been proven very useful so far. Yes, you're looking only at a single interface (the radio interface between one BSS and one STA). You're not looking at five different interfaces at five different levels of network hierarchy/topology in the "wifi controller" and want to mix in a radio interface trace in the same timeline. sorry. '... UDP based transport is ahat enables theis use case' As stated elsewhere, there's an M:N mapping between user (IP) payload and actual radio interface "MAC blocks", so I'm not aware of anyone mapping radio interface performance to user plane IP. the "physical link info" is present in GSMTAP, but the granularity of GSMTAP frames is not user-IP frames, but "MAC blocks". So your user IP frame might not be visible as it's still compressed, encrypted, fragmented, etc. I admit it's a bit odd. But has been very useful, particularly in more distributed setups with creating a shared timeline of various GSMTAP sources that may or may not run on the same machine. -- - Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)

Hi Harald, Sure. I'm (vaguely) aware of that. Really the question it goes down to is whether there's possible overlap here with GSMTAP or not. If yes - we can define a GSMTAPv3, with TLVs or so, and add a "PDP/PDN context" or "session" field or something like that, and maybe a tag that says "IP packet is here", or some kind of "content type" field. I expect there would be some such "sessions" that actually just transport the local AT command chat. If no, I guess we'll just define something else for this use case? Or, perhaps, even get rid of it entirely and just rely on trace-cmd recording or so, though then I guess we'd really want to teach libpcap and/or wireshark to understand this in some way. Though it almost sounds like in GSMTAP you're never actually transporting IP data, but other types of packets that in turn transport the (fragmented/compressed/encrypted) IP data. That would kinda mean there's very little potential overlap. Yeah, ok. I was thinking kinda the other way around - if all you need is network transparency for wireshark it shouldn't matter, but that's really discussed over in the other subthread with Guy Harris better. Anyway it doesn't matter, and I'm beginning to understand that a (maybe even the primary) use case is getting multiple parts of your stack to talk to each other (over UDP). Indeed. Well, actually, we often do, but use different mechanics for that (trace-cmd record comes to mind, it records all our driver/fw chat as well as possibly adding in logging from wpa_supplicant etc.) Right, OK. Sure. As Guy Harris pointed out, this is also true for wifi though - we don't have compression, but certainly encryption & fragmentation, wireshark was taught to handle that and recreate the original MSDU (mac service data unit). johannes