<p>lynxis lazus has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/14196">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">utils: add gsmtap_logread.py a gsmtap log reader<br><br>Receive gsmtap logs and feeds it into the python logging<br>framework. Allows to use generic logging features and<br>further utilities.<br><br>Change-Id: I24478d8e16066c6118e867bdba54c6418c15e170<br>---<br>A utils/gsmtap.py<br>A utils/gsmtap_logread.py<br>2 files changed, 163 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/96/14196/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/utils/gsmtap.py b/utils/gsmtap.py</span><br><span>new file mode 100644</span><br><span>index 0000000..68658c5</span><br><span>--- /dev/null</span><br><span>+++ b/utils/gsmtap.py</span><br><span>@@ -0,0 +1,75 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# license: MIT</span><br><span style="color: hsl(120, 100%, 40%);">+# 2019 Alexander Couzens <lynxis@fe80.eu></span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import struct</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+GSMTAP_VERSION = 0x02</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+GSMTAP_TYPE_OSMOCORE_LOG = 0x10</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class TooSmall(RuntimeError):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# struct gsmtap_hdr {</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t version; /*!< version, set to 0x01 currently */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t hdr_len; /*!< length in number of 32bit words */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t type; /*!< see GSMTAP_TYPE_* */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t timeslot; /*!< timeslot (0..7 on Um) */</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# uint16_t arfcn; /*!< ARFCN (frequency) */</span><br><span style="color: hsl(120, 100%, 40%);">+# int8_t signal_dbm; /*!< signal level in dBm */</span><br><span style="color: hsl(120, 100%, 40%);">+# int8_t snr_db; /*!< signal/noise ratio in dB */</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# uint32_t frame_number; /*!< GSM Frame Number (FN) */</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t sub_type; /*!< Type of burst/channel, see above */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t antenna_nr; /*!< Antenna Number */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t sub_slot; /*!< sub-slot within timeslot */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t res; /*!< reserved for future use (RFU) */</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class gsmtap_hdr():</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, data):</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(data) < 2:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise TooSmall()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.version, self.hdr_len = struct.unpack('!BB', data[0:2])</span><br><span style="color: hsl(120, 100%, 40%);">+ self.hdr_len *= 4</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.hdr_len >= 3:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.type = struct.unpack('!B', data[2:3])[0]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# /*! Structure of the GSMTAP libosmocore logging header */</span><br><span style="color: hsl(120, 100%, 40%);">+# struct gsmtap_osmocore_log_hdr {</span><br><span style="color: hsl(120, 100%, 40%);">+# struct {</span><br><span style="color: hsl(120, 100%, 40%);">+# uint32_t sec;</span><br><span style="color: hsl(120, 100%, 40%);">+# uint32_t usec;</span><br><span style="color: hsl(120, 100%, 40%);">+# } ts;</span><br><span style="color: hsl(120, 100%, 40%);">+# char proc_name[16]; /*!< name of process */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint32_t pid; /*!< process ID */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t level; /*!< logging level */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint8_t _pad[3];</span><br><span style="color: hsl(120, 100%, 40%);">+# /* TODO: color */</span><br><span style="color: hsl(120, 100%, 40%);">+# char subsys[16]; /*!< logging sub-system */</span><br><span style="color: hsl(120, 100%, 40%);">+# struct {</span><br><span style="color: hsl(120, 100%, 40%);">+# char name[32]; /*!< source file name */</span><br><span style="color: hsl(120, 100%, 40%);">+# uint32_t line_nr;/*!< line number */</span><br><span style="color: hsl(120, 100%, 40%);">+# } src_file;</span><br><span style="color: hsl(120, 100%, 40%);">+# } __attribute__((packed));</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class gsmtap_log():</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, data):</span><br><span style="color: hsl(120, 100%, 40%);">+ packformat = '!II16sIB3xxx16s32sI'</span><br><span style="color: hsl(120, 100%, 40%);">+ packlen = struct.calcsize(packformat)</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(data) < packlen:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise TooSmall()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.sec, self.usec, \</span><br><span style="color: hsl(120, 100%, 40%);">+ self.proc_name, self.pid, \</span><br><span style="color: hsl(120, 100%, 40%);">+ self.level, self.subsys, \</span><br><span style="color: hsl(120, 100%, 40%);">+ self.filename, self.fileline_nr = struct.unpack(packformat, data[:packlen])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ message_len = len(data) - packlen</span><br><span style="color: hsl(120, 100%, 40%);">+ if message_len > 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.message = data[packlen:].decode('utf-8')</span><br><span>diff --git a/utils/gsmtap_logread.py b/utils/gsmtap_logread.py</span><br><span>new file mode 100644</span><br><span>index 0000000..77bf47d</span><br><span>--- /dev/null</span><br><span>+++ b/utils/gsmtap_logread.py</span><br><span>@@ -0,0 +1,88 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# license: MIT</span><br><span style="color: hsl(120, 100%, 40%);">+# 2019 Alexander Couzens <lynxis@fe80.eu></span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import logging</span><br><span style="color: hsl(120, 100%, 40%);">+import socket</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from gsmtap import GSMTAP_TYPE_OSMOCORE_LOG, gsmtap_hdr, gsmtap_log, TooSmall</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+LOG = logging.getLogger("gsmlogreader")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def parse_gsm(packet):</span><br><span style="color: hsl(120, 100%, 40%);">+ hdr = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ hdr = gsmtap_hdr(packet)</span><br><span style="color: hsl(120, 100%, 40%);">+ except TooSmall:</span><br><span style="color: hsl(120, 100%, 40%);">+ return None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if hdr.type != GSMTAP_TYPE_OSMOCORE_LOG:</span><br><span style="color: hsl(120, 100%, 40%);">+ return None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(packet) <= hdr.hdr_len:</span><br><span style="color: hsl(120, 100%, 40%);">+ return None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ return gsmtap_log(packet[hdr.hdr_len:])</span><br><span style="color: hsl(120, 100%, 40%);">+ except TooSmall:</span><br><span style="color: hsl(120, 100%, 40%);">+ return None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def gsmtaplevel_to_loglevel(level):</span><br><span style="color: hsl(120, 100%, 40%);">+ """ convert a gsmtap log level into a python log level """</span><br><span style="color: hsl(120, 100%, 40%);">+ if level <= 1:</span><br><span style="color: hsl(120, 100%, 40%);">+ return logging.DEBUG</span><br><span style="color: hsl(120, 100%, 40%);">+ if level <= 3:</span><br><span style="color: hsl(120, 100%, 40%);">+ return logging.INFO</span><br><span style="color: hsl(120, 100%, 40%);">+ if level <= 5:</span><br><span style="color: hsl(120, 100%, 40%);">+ return logging.WARNING</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ return logging.ERROR</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def convert_gsmtap_log(gsmtap):</span><br><span style="color: hsl(120, 100%, 40%);">+ level = gsmtaplevel_to_loglevel(gsmtap.level)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # TODO: convert sec, usec into logging timestamp</span><br><span style="color: hsl(120, 100%, 40%);">+ attr = {</span><br><span style="color: hsl(120, 100%, 40%);">+ "name": "gsmtap",</span><br><span style="color: hsl(120, 100%, 40%);">+ "levelno": level,</span><br><span style="color: hsl(120, 100%, 40%);">+ "levelname": gsmtap_get_logname(gsmtap.level),</span><br><span style="color: hsl(120, 100%, 40%);">+ "pathname": gsmtap.filename,</span><br><span style="color: hsl(120, 100%, 40%);">+ "lineno": gsmtap.fileline_nr,</span><br><span style="color: hsl(120, 100%, 40%);">+ "processName": gsmtap.proc_name,</span><br><span style="color: hsl(120, 100%, 40%);">+ "process": gsmtap.pid,</span><br><span style="color: hsl(120, 100%, 40%);">+ "module": gsmtap.subsys,</span><br><span style="color: hsl(120, 100%, 40%);">+ "msg": gsmtap.message.replace('\n', ' '),</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ return attr</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def gsmtap_get_logname(level):</span><br><span style="color: hsl(120, 100%, 40%);">+ names = {</span><br><span style="color: hsl(120, 100%, 40%);">+ 1: "DEBUG",</span><br><span style="color: hsl(120, 100%, 40%);">+ 3: "INFO",</span><br><span style="color: hsl(120, 100%, 40%);">+ 5: "NOTICE",</span><br><span style="color: hsl(120, 100%, 40%);">+ 7: "ERROR",</span><br><span style="color: hsl(120, 100%, 40%);">+ 8: "FATAL",</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ if level in names:</span><br><span style="color: hsl(120, 100%, 40%);">+ return names[level]</span><br><span style="color: hsl(120, 100%, 40%);">+ return "UNKNOWN"</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+if __name__ == "__main__":</span><br><span style="color: hsl(120, 100%, 40%);">+ # Create a UDP socket</span><br><span style="color: hsl(120, 100%, 40%);">+ sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)</span><br><span style="color: hsl(120, 100%, 40%);">+ server_address = ('0.0.0.0', 4729)</span><br><span style="color: hsl(120, 100%, 40%);">+ sock.bind(server_address)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ logger = logging.getLogger("gsmtap")</span><br><span style="color: hsl(120, 100%, 40%);">+ logging.basicConfig(level=logging.DEBUG)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ while True:</span><br><span style="color: hsl(120, 100%, 40%);">+ data, address = sock.recvfrom(4096)</span><br><span style="color: hsl(120, 100%, 40%);">+ log = parse_gsm(data)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not log:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ record = logging.makeLogRecord(convert_gsmtap_log(log))</span><br><span style="color: hsl(120, 100%, 40%);">+ logger.handle(record)</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/14196">change 14196</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/14196"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: libosmocore </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: I24478d8e16066c6118e867bdba54c6418c15e170 </div>
<div style="display:none"> Gerrit-Change-Number: 14196 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: lynxis lazus <lynxis@fe80.eu> </div>