<p>laforge <strong>submitted</strong> this change.</p><p><a href="https://gerrit.osmocom.org/c/osmo-msc/+/15944">View Change</a></p><div style="white-space:pre-wrap">Approvals:
Jenkins Builder: Verified
pespin: Looks good to me, but someone else must approve
laforge: Looks good to me, approved
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">add msc_log_to_ladder.py<br><br>Add script that reads in an osmo-msc log output and extracts the interesting<br>information for displaying a sequence chart of voice call log, in mscgen<br>format.<br><br>I want to visualize how the sequence of messages changes across patches. It is<br>error prone to do it manually, and re-doing the sequence chart for every patch<br>(and patch rework) would be prohibitively time consuming.<br><br>Change-Id: I2e4d8778f7b83dee558517a9b23450b817ee325d<br>---<br>A doc/sequence_charts/msc_log_to_ladder.py<br>1 file changed, 724 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/doc/sequence_charts/msc_log_to_ladder.py b/doc/sequence_charts/msc_log_to_ladder.py</span><br><span>new file mode 100755</span><br><span>index 0000000..1ceaeb6</span><br><span>--- /dev/null</span><br><span>+++ b/doc/sequence_charts/msc_log_to_ladder.py</span><br><span>@@ -0,0 +1,724 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span style="color: hsl(120, 100%, 40%);">+doc=r'''</span><br><span style="color: hsl(120, 100%, 40%);">+Reading a log, it can be hard to figure out what is the important bit going on.</span><br><span style="color: hsl(120, 100%, 40%);">+This is a tool that reads an osmo-msc log and tries to compose a ladder diagram from it automatically.</span><br><span style="color: hsl(120, 100%, 40%);">+'''</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import argparse</span><br><span style="color: hsl(120, 100%, 40%);">+import sys</span><br><span style="color: hsl(120, 100%, 40%);">+import re</span><br><span style="color: hsl(120, 100%, 40%);">+import tempfile</span><br><span style="color: hsl(120, 100%, 40%);">+import os</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def error(*msg):</span><br><span style="color: hsl(120, 100%, 40%);">+ sys.stderr.write('%s\n' % (''.join(msg)))</span><br><span style="color: hsl(120, 100%, 40%);">+ exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def quote(msg, quote='"'):</span><br><span style="color: hsl(120, 100%, 40%);">+ return '"%s"' % (msg.replace('"', r'\"'))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Entity:</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, name, descr=None, attrs={}):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.name = name</span><br><span style="color: hsl(120, 100%, 40%);">+ self.descr = descr</span><br><span style="color: hsl(120, 100%, 40%);">+ self.attrs = attrs</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Arrow:</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, mo_mt, left, arrow, right, descr=None, attrs={}, ran_conn=None, imsi=None, tmsi=None):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.mo_mt = mo_mt</span><br><span style="color: hsl(120, 100%, 40%);">+ self.left = left</span><br><span style="color: hsl(120, 100%, 40%);">+ self.arrow = arrow</span><br><span style="color: hsl(120, 100%, 40%);">+ self.right = right</span><br><span style="color: hsl(120, 100%, 40%);">+ self.descr = descr</span><br><span style="color: hsl(120, 100%, 40%);">+ self.attrs = attrs</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ran_conn = ran_conn</span><br><span style="color: hsl(120, 100%, 40%);">+ self.imsi = imsi</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tmsi = tmsi</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __repr__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return 'Arrow(%s %s %s %s: %s IMSI=%s)' % (self.mo_mt, self.left, self.arrow, self.right, self.descr, self.imsi)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Separator:</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.separator = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.descr = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.attrs = {}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class EmptyLine:</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.count = 1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+MS = 'ms'</span><br><span style="color: hsl(120, 100%, 40%);">+UE = 'ms' #'ue'</span><br><span style="color: hsl(120, 100%, 40%);">+MS_UE_UNKNOWN = 'ms' #None</span><br><span style="color: hsl(120, 100%, 40%);">+MSC = 'msc'</span><br><span style="color: hsl(120, 100%, 40%);">+MGW = 'mgw'</span><br><span style="color: hsl(120, 100%, 40%);">+SIP = 'sip'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+MO = 'mo'</span><br><span style="color: hsl(120, 100%, 40%);">+MT = 'mt'</span><br><span style="color: hsl(120, 100%, 40%);">+MO_MT_UNKNOWN = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class OutputBase:</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, write_to, start_with_re=None):</span><br><span style="color: hsl(120, 100%, 40%);">+ self._write_to = write_to</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.start_with_re = None</span><br><span style="color: hsl(120, 100%, 40%);">+ if start_with_re is not None:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.start_with_re = re.compile(start_with_re)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def head(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('# Generated by msc_log_to_ladder.py')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def tail(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def render(self, diagram):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.head()</span><br><span style="color: hsl(120, 100%, 40%);">+ if diagram.root_attrs:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.root_attrs(diagram.root_attrs)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.entities(diagram.entities)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ started = (self.start_with_re is None)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for line in diagram.lines:</span><br><span style="color: hsl(120, 100%, 40%);">+ if not started:</span><br><span style="color: hsl(120, 100%, 40%);">+ if hasattr(line, 'descr') and self.start_with_re.match(line.descr):</span><br><span style="color: hsl(120, 100%, 40%);">+ started = True</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ self.add(line)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tail()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def entities(self, entities):</span><br><span style="color: hsl(120, 100%, 40%);">+ for entity in entities:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.entity(entity)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def write(self, line):</span><br><span style="color: hsl(120, 100%, 40%);">+ self._write_to.write(line)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def writeln(self, line):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.write('%s\n' % line)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def emptyline(self, emptyline):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.write('\n' * emptyline.count);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def add(self, thing):</span><br><span style="color: hsl(120, 100%, 40%);">+ func = getattr(self, thing.__class__.__name__.lower())</span><br><span style="color: hsl(120, 100%, 40%);">+ func(thing)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class OutputLadder(OutputBase):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def indent_multiline(self, s):</span><br><span style="color: hsl(120, 100%, 40%);">+ return s.replace('\n', '\n\t')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def attrs_str(self, attrs, prefix=' '):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not attrs:</span><br><span style="color: hsl(120, 100%, 40%);">+ return ''</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s{%s}' % (prefix or '', ','.join('%s=%s' % (k,v) for k,v in attrs.items()))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def root_attrs(self, attrs):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln(self.attrs_str(attrs, prefix=None))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def entity(self, entity):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('%s = %s%s' % (entity.name, self.indent_multiline(entity.descr), self.attrs_str(entity.attrs)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def arrow(self, arrow):</span><br><span style="color: hsl(120, 100%, 40%);">+ mo_mt = arrow.mo_mt or MO</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def prepend_mo_mt(name):</span><br><span style="color: hsl(120, 100%, 40%);">+ if name in ('.', SIP):</span><br><span style="color: hsl(120, 100%, 40%);">+ return name</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s%s' % (mo_mt, name)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('%s %s %s%s%s%s'</span><br><span style="color: hsl(120, 100%, 40%);">+ % (prepend_mo_mt(arrow.left),</span><br><span style="color: hsl(120, 100%, 40%);">+ arrow.arrow,</span><br><span style="color: hsl(120, 100%, 40%);">+ prepend_mo_mt(arrow.right),</span><br><span style="color: hsl(120, 100%, 40%);">+ '\t' if arrow.descr else '',</span><br><span style="color: hsl(120, 100%, 40%);">+ self.indent_multiline(arrow.descr or ''),</span><br><span style="color: hsl(120, 100%, 40%);">+ self.attrs_str(arrow.attrs)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def separator(self, sep_str, descr, attrs):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('%s%s%s%s'</span><br><span style="color: hsl(120, 100%, 40%);">+ % (separator.separator,</span><br><span style="color: hsl(120, 100%, 40%);">+ ' ' if separator.descr else '',</span><br><span style="color: hsl(120, 100%, 40%);">+ self.indent_multiline(separator.descr or ''),</span><br><span style="color: hsl(120, 100%, 40%);">+ self.attrs_str(separator.attrs)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class OutputMscgen(OutputBase):</span><br><span style="color: hsl(120, 100%, 40%);">+ ARROWS = {</span><br><span style="color: hsl(120, 100%, 40%);">+ '>' : '=>>',</span><br><span style="color: hsl(120, 100%, 40%);">+ '->' : '=>',</span><br><span style="color: hsl(120, 100%, 40%);">+ '-->' : '>>',</span><br><span style="color: hsl(120, 100%, 40%);">+ '~>' : '->',</span><br><span style="color: hsl(120, 100%, 40%);">+ '=>' : ':>',</span><br><span style="color: hsl(120, 100%, 40%);">+ '-><' : '-x',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ '<' : '<<=',</span><br><span style="color: hsl(120, 100%, 40%);">+ '<-' : '<=',</span><br><span style="color: hsl(120, 100%, 40%);">+ '<--' : '<<',</span><br><span style="color: hsl(120, 100%, 40%);">+ '<~' : '<-',</span><br><span style="color: hsl(120, 100%, 40%);">+ '<=' : '<:',</span><br><span style="color: hsl(120, 100%, 40%);">+ '><-' : 'x-',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ '<>' : 'abox',</span><br><span style="color: hsl(120, 100%, 40%);">+ '()' : 'rbox',</span><br><span style="color: hsl(120, 100%, 40%);">+ '[]' : 'note',</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def head(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().head()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('msc {')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def tail(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('}')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def entities(self, entities):</span><br><span style="color: hsl(120, 100%, 40%);">+ estr = []</span><br><span style="color: hsl(120, 100%, 40%);">+ for entity in entities:</span><br><span style="color: hsl(120, 100%, 40%);">+ estr.append('%s%s' % (entity.name, self.attrs_str(self.all_attrs(entity.descr, entity.attrs), prefix='')))</span><br><span style="color: hsl(120, 100%, 40%);">+ if not estr:</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('%s;' % (','.join(estr)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def attrs_list_str(self, attrs):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not attrs:</span><br><span style="color: hsl(120, 100%, 40%);">+ return ''</span><br><span style="color: hsl(120, 100%, 40%);">+ def escape(s):</span><br><span style="color: hsl(120, 100%, 40%);">+ return s.replace('\n', r'\n').replace('\r', '').replace('\t', r'\t')</span><br><span style="color: hsl(120, 100%, 40%);">+ return ','.join('%s="%s"' % (k,escape(v)) for k,v in attrs.items())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def attrs_str(self, attrs, prefix=' '):</span><br><span style="color: hsl(120, 100%, 40%);">+ attrs_list_str = self.attrs_list_str(attrs)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not attrs_list_str:</span><br><span style="color: hsl(120, 100%, 40%);">+ return ''</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s[%s]' % (prefix or '', attrs_list_str)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def root_attrs(self, attrs):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not attrs:</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('%s;' % self.attrs_list_str(attrs))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def all_attrs(self, descr, attrs):</span><br><span style="color: hsl(120, 100%, 40%);">+ a = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ if descr:</span><br><span style="color: hsl(120, 100%, 40%);">+ a['label'] = descr</span><br><span style="color: hsl(120, 100%, 40%);">+ a.update(attrs)</span><br><span style="color: hsl(120, 100%, 40%);">+ return a</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def entity(self, entity):</span><br><span style="color: hsl(120, 100%, 40%);">+ error('OutputMscgen.entity() should not be called')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def arrow_txlate(self, arrow):</span><br><span style="color: hsl(120, 100%, 40%);">+ txlate = OutputMscgen.ARROWS.get(arrow)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not txlate:</span><br><span style="color: hsl(120, 100%, 40%);">+ error('Unknown arrow: %r' % arrow)</span><br><span style="color: hsl(120, 100%, 40%);">+ return txlate</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def arrow(self, arrow):</span><br><span style="color: hsl(120, 100%, 40%);">+ mo_mt = arrow.mo_mt or MO</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def prepend_mo_mt(name):</span><br><span style="color: hsl(120, 100%, 40%);">+ if name in ('.', SIP):</span><br><span style="color: hsl(120, 100%, 40%);">+ return name</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s%s' % (mo_mt, name)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ l = prepend_mo_mt(arrow.left)</span><br><span style="color: hsl(120, 100%, 40%);">+ r = arrow.right</span><br><span style="color: hsl(120, 100%, 40%);">+ if r == '.':</span><br><span style="color: hsl(120, 100%, 40%);">+ r = l</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ r = prepend_mo_mt(r)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ a = {'label': arrow.descr}</span><br><span style="color: hsl(120, 100%, 40%);">+ a.update(arrow.attrs)</span><br><span style="color: hsl(120, 100%, 40%);">+ attrs = self.attrs_str(a)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('%s %s %s%s;'</span><br><span style="color: hsl(120, 100%, 40%);">+ % (l, self.arrow_txlate(arrow.arrow), r,</span><br><span style="color: hsl(120, 100%, 40%);">+ self.attrs_str(self.all_attrs(arrow.descr, arrow.attrs), prefix='\t')))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def separator(self, sep_str, descr, attrs):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.writeln('%s%s%s%s;'</span><br><span style="color: hsl(120, 100%, 40%);">+ % (separator.separator,</span><br><span style="color: hsl(120, 100%, 40%);">+ self.attrs_str(self.all_attrs(descr, attrs), prefix='\t')))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def emptyline(self, emptyline):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.write('\n' * emptyline.count);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def ms_from_ran(ran_type_or_conn):</span><br><span style="color: hsl(120, 100%, 40%);">+ if ran_type_or_conn.startswith('UTRAN-Iu'):</span><br><span style="color: hsl(120, 100%, 40%);">+ return UE</span><br><span style="color: hsl(120, 100%, 40%);">+ if ran_type_or_conn.startswith('RANAP'):</span><br><span style="color: hsl(120, 100%, 40%);">+ return UE</span><br><span style="color: hsl(120, 100%, 40%);">+ if ran_type_or_conn.startswith('GERAN-A'):</span><br><span style="color: hsl(120, 100%, 40%);">+ return MS</span><br><span style="color: hsl(120, 100%, 40%);">+ if ran_type_or_conn.startswith('BSS'):</span><br><span style="color: hsl(120, 100%, 40%);">+ return MS</span><br><span style="color: hsl(120, 100%, 40%);">+ return MS_UE_UNKNOWN</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Diagram:</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.root_attrs = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ self.entities = []</span><br><span style="color: hsl(120, 100%, 40%);">+ self.lines = []</span><br><span style="color: hsl(120, 100%, 40%);">+ self.mo_mt_unknown_lines = []</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def add_line(self, line):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.lines.append(line)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def resolve_unknown_mo_mt(self):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def try_match(a, b):</span><br><span style="color: hsl(120, 100%, 40%);">+ if a < 0 or a >= len(self.lines):</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+ if b < 0 or b >= len(self.lines):</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+ la = self.lines[a]</span><br><span style="color: hsl(120, 100%, 40%);">+ lb = self.lines[b]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if not hasattr(lb, 'mo_mt'):</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+ if lb.mo_mt == MO_MT_UNKNOWN:</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for match_attr in ('imsi', 'tmsi', 'ran_conn'):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not hasattr(la, match_attr):</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ if not hasattr(lb, match_attr):</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ la_attr = getattr(la, match_attr)</span><br><span style="color: hsl(120, 100%, 40%);">+ if la_attr is None:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ lb_attr = getattr(lb, match_attr)</span><br><span style="color: hsl(120, 100%, 40%);">+ if la_attr == lb_attr:</span><br><span style="color: hsl(120, 100%, 40%);">+ la.mo_mt = lb.mo_mt</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ while True:</span><br><span style="color: hsl(120, 100%, 40%);">+ changes = 0</span><br><span style="color: hsl(120, 100%, 40%);">+ for i in range(len(self.lines)):</span><br><span style="color: hsl(120, 100%, 40%);">+ line = self.lines[i]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if not hasattr(line, 'mo_mt'):</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ if line.mo_mt is not MO_MT_UNKNOWN:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # don't know MO/MT, try to resolve from near messages</span><br><span style="color: hsl(120, 100%, 40%);">+ for j in range(1,100):</span><br><span style="color: hsl(120, 100%, 40%);">+ if try_match(i, i-j):</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+ if try_match(i, i+j):</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+ if line.mo_mt is not MO_MT_UNKNOWN:</span><br><span style="color: hsl(120, 100%, 40%);">+ changes += 1</span><br><span style="color: hsl(120, 100%, 40%);">+ if not changes:</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+re_source_file_last = re.compile(r'(.*) \(([^):]+:[0-9]+)\)$')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Rule:</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, name, re_str, handler):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.name = name</span><br><span style="color: hsl(120, 100%, 40%);">+ self.re = re.compile(re_str)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.handler = handler</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def match(self, line):</span><br><span style="color: hsl(120, 100%, 40%);">+ m = self.re.match(line)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not m:</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.handler(m)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def mo_mt_from_l3type(l3type):</span><br><span style="color: hsl(120, 100%, 40%);">+ if l3type == 'PAGING_RESP':</span><br><span style="color: hsl(120, 100%, 40%);">+ return MT</span><br><span style="color: hsl(120, 100%, 40%);">+ elif l3type == 'CM_SERVICE_REQ':</span><br><span style="color: hsl(120, 100%, 40%);">+ return MO</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ return MO_MT_UNKNOWN</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def int_to_hex(val, bits):</span><br><span style="color: hsl(120, 100%, 40%);">+ return hex((int(val) + (1 << bits)) % (1 << bits))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Callref:</span><br><span style="color: hsl(120, 100%, 40%);">+ MAX = 0x7fffffff</span><br><span style="color: hsl(120, 100%, 40%);">+ MIN = -0x80000000</span><br><span style="color: hsl(120, 100%, 40%);">+ BITS = 32</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def int_to_hex(val):</span><br><span style="color: hsl(120, 100%, 40%);">+ return int_to_hex(val, Callref.BITS)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def hex_to_int(hexstr):</span><br><span style="color: hsl(120, 100%, 40%);">+ val = int(hexstr, 16)</span><br><span style="color: hsl(120, 100%, 40%);">+ if val > Callref.MAX:</span><br><span style="color: hsl(120, 100%, 40%);">+ val = Callref.MIN + (val & Callref.MAX)</span><br><span style="color: hsl(120, 100%, 40%);">+ return val</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Parse:</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, output, mask_values=False):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram = Diagram()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.output = output</span><br><span style="color: hsl(120, 100%, 40%);">+ self.linenr = 0</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rules = []</span><br><span style="color: hsl(120, 100%, 40%);">+ self.seen_udtrace_mncc = False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.callrefs_mo_mt = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ self.mask_values = mask_values</span><br><span style="color: hsl(120, 100%, 40%);">+ self.masked_values = {}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for member in dir(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not member.startswith('rule_'):</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ func = getattr(self, member)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not callable(func):</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ docstr = func.__doc__</span><br><span style="color: hsl(120, 100%, 40%);">+ if not docstr:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ re_str = docstr.splitlines()[0]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.rules.append(Rule(name=member, re_str=re_str, handler=func))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def error(self, *msg):</span><br><span style="color: hsl(120, 100%, 40%);">+ error('line %d: ' % self.linenr, *msg)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def start(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.root_attrs = {'hscale': '3'}</span><br><span style="color: hsl(120, 100%, 40%);">+ for name, descr in (</span><br><span style="color: hsl(120, 100%, 40%);">+ ('moms', 'MS,BSS (MO)\\nUE,hNodeB (MO)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ #('moue', 'UE,hNodeB (MO)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ ('momgw', 'MGW for MSC (MO)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ ('momsc', 'MSC (MO)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ ('sip', 'MNCC to PBX via\nosmo-sip-connector'),</span><br><span style="color: hsl(120, 100%, 40%);">+ ('mtmsc', 'MSC (MT)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ ('mtmgw', 'MGW for MSC (MT)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ ('mtms', 'BSS,MS (MT)\\nhNodeB,UE (MT)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ #('mtue', 'hNodeB,UE (MT)'),</span><br><span style="color: hsl(120, 100%, 40%);">+ ):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.entities.append(Entity(name, descr))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def end(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.resolve_unknown_mo_mt()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.output.render(self.diagram)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def mask_value(self, name, val):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not self.mask_values:</span><br><span style="color: hsl(120, 100%, 40%);">+ return val</span><br><span style="color: hsl(120, 100%, 40%);">+ if not val:</span><br><span style="color: hsl(120, 100%, 40%);">+ return val</span><br><span style="color: hsl(120, 100%, 40%);">+ name_dict = self.masked_values.get(name)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not name_dict:</span><br><span style="color: hsl(120, 100%, 40%);">+ name_dict = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ self.masked_values[name] = name_dict</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ masked_val = name_dict.get(val)</span><br><span style="color: hsl(120, 100%, 40%);">+ if masked_val is None:</span><br><span style="color: hsl(120, 100%, 40%);">+ masked_val = '%s-%d' % (name, len(name_dict) + 1)</span><br><span style="color: hsl(120, 100%, 40%);">+ name_dict[val] = masked_val</span><br><span style="color: hsl(120, 100%, 40%);">+ return masked_val</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def add_line(self, line):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.linenr += 1</span><br><span style="color: hsl(120, 100%, 40%);">+ if line.endswith('\n'):</span><br><span style="color: hsl(120, 100%, 40%);">+ line = line[:-1]</span><br><span style="color: hsl(120, 100%, 40%);">+ if line.endswith('\r'):</span><br><span style="color: hsl(120, 100%, 40%);">+ line = line[:-1]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.interpret(line)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def interpret(self, line):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ m = re_source_file_last.match(line)</span><br><span style="color: hsl(120, 100%, 40%);">+ if m:</span><br><span style="color: hsl(120, 100%, 40%);">+ line = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for rule in self.rules:</span><br><span style="color: hsl(120, 100%, 40%);">+ if rule.match(line):</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ RE_DTAP_NAME = re.compile('.*GSM48_MT_([^_]+)_(.+)')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_paging(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*ran_peer\(([^:]+):.* Paging for ([^ ]+) on ([^ ]+)'</span><br><span style="color: hsl(120, 100%, 40%);">+ ran_type, subscr, cell = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(MT, ms_from_ran(ran_type), '<', MSC, 'Paging'))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ RE_IMSI = re.compile('IMSI-([0-9]+)')</span><br><span style="color: hsl(120, 100%, 40%);">+ RE_TMSI = re.compile('TMSI-0x([0-9a-fA-F]+)')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_dtap(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*msc_a\(([^)]*):([^:]+):([^:]+)\).* (Dispatching 04.08 message|Sending DTAP): (.+)$'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ subscr, ran_conn, l3type, rx_tx, dtap = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+ tx = (rx_tx == 'Sending DTAP')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ m = self.RE_DTAP_NAME.match(dtap)</span><br><span style="color: hsl(120, 100%, 40%);">+ if m:</span><br><span style="color: hsl(120, 100%, 40%);">+ dtap = '%s %s' % m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if 'IMSI_DETACH_IND' in dtap:</span><br><span style="color: hsl(120, 100%, 40%);">+ # detecting IMSI Detach separately</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if l3type == 'NONE' and not tx and dtap.endswith('PAG_RESP'):</span><br><span style="color: hsl(120, 100%, 40%);">+ e = MT</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ e = mo_mt_from_l3type(l3type)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_IMSI.finditer(subscr):</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ tmsi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_TMSI.finditer(subscr):</span><br><span style="color: hsl(120, 100%, 40%);">+ tmsi = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, ms_from_ran(ran_conn), '<' if tx else '>', MSC, dtap,</span><br><span style="color: hsl(120, 100%, 40%);">+ ran_conn=ran_conn, imsi=imsi, tmsi=tmsi))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_imsi_detach(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*IMSI DETACH for IMSI-([0-9]+):.*'</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ e = MO_MT_UNKNOWN</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MS_UE_UNKNOWN, '>', MSC, 'IMSI Detach', imsi=imsi))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_mgcp_tx(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*mgw-endp\([^)]*:([^:]+):([^:]+)\).* (rtpbridge[^ ]+) .* RTP_TO_(RAN|CN)( CI=([^:]+)|): ([^ :]+).*: Sending'</span><br><span style="color: hsl(120, 100%, 40%);">+ ran, l3type, endp, rtp_to, cond_ci, ci, verb = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+ e = mo_mt_from_l3type(l3type)</span><br><span style="color: hsl(120, 100%, 40%);">+ if '*' not in endp:</span><br><span style="color: hsl(120, 100%, 40%);">+ endp = self.mask_value('EP', endp)</span><br><span style="color: hsl(120, 100%, 40%);">+ ci = self.mask_value('CI', ci)</span><br><span style="color: hsl(120, 100%, 40%);">+ ci_str = ''</span><br><span style="color: hsl(120, 100%, 40%);">+ if ci:</span><br><span style="color: hsl(120, 100%, 40%);">+ ci_str = ' %s' % ci</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MGW, '<', MSC, 'for %s: %s\\n%s%s' % (rtp_to, verb, endp, ci_str)))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_mgcp_rx(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*mgw-endp\([^)]*:([^:]+):([^:]+)\).* (rtpbridge[^ ]+) .* RTP_TO_(RAN|CN)( CI=([^:]+)|).*: received successful response to ([^:]+): (.*)'</span><br><span style="color: hsl(120, 100%, 40%);">+ ran, l3type, endp, rtp_to, cond_ci, ci, verb, details = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+ e = mo_mt_from_l3type(l3type)</span><br><span style="color: hsl(120, 100%, 40%);">+ endp = self.mask_value('EP', endp)</span><br><span style="color: hsl(120, 100%, 40%);">+ ci = self.mask_value('CI', ci)</span><br><span style="color: hsl(120, 100%, 40%);">+ ci_str = ''</span><br><span style="color: hsl(120, 100%, 40%);">+ if ci:</span><br><span style="color: hsl(120, 100%, 40%);">+ ci_str = ' %s' % ci</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MGW, '>', MSC, 'for %s: %s OK\\n%s%s' % (rtp_to, verb, endp, ci_str)))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_ran_tx(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*msc_a\(([^)]*):([^:]+):([^:]+)\).* RAN encode: ([^: ]+): (.+)$'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ subscr, ran_conn, l3type, ran_type, msg_type = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if msg_type in ('DTAP', 'DirectTransfer'):</span><br><span style="color: hsl(120, 100%, 40%);">+ # will get DTAP details from rule_dtap() instead, not from BSSMAP logging</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+ if msg_type.startswith('Tx'):</span><br><span style="color: hsl(120, 100%, 40%);">+ # skip 'Tx RANAP SECURITY MODE COMMAND to RNC, ik 47...'</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+ if '=' in msg_type:</span><br><span style="color: hsl(120, 100%, 40%);">+ # skip 'RAB Assignment: rab_id=1, rtp=192.168.178.66:50008, use_x213_nsap=1'</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if l3type == 'NONE':</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ e = mo_mt_from_l3type(l3type)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_IMSI.finditer(subscr):</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ tmsi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_TMSI.finditer(subscr):</span><br><span style="color: hsl(120, 100%, 40%);">+ tmsi = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, ms_from_ran(ran_conn), '<', MSC, '(%s) %s' % (ran_type, msg_type),</span><br><span style="color: hsl(120, 100%, 40%);">+ ran_conn=ran_conn, imsi=imsi, tmsi=tmsi))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_ran_rx(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*msc_a\(([^)]*):([^:]+):([^:]+)\).* RAN decode: ([^: ]+) (.+)$'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ subscr, ran_conn, l3type, ran_type, msg_type = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if msg_type in ('DTAP', 'DirectTransfer', 'DirectTransfer RAN PDU'):</span><br><span style="color: hsl(120, 100%, 40%);">+ # will get DTAP details from rule_dtap() instead, not from BSSMAP logging</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if l3type == 'NONE':</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ e = mo_mt_from_l3type(l3type)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_IMSI.finditer(subscr):</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ tmsi = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_TMSI.finditer(subscr):</span><br><span style="color: hsl(120, 100%, 40%);">+ tmsi = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, ms_from_ran(ran_type), '>', MSC, '(%s) %s' % (ran_type, msg_type),</span><br><span style="color: hsl(120, 100%, 40%);">+ ran_conn=ran_conn, imsi=imsi, tmsi=tmsi))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_cc_state(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*trans\(CC[^) ]* [^ )]+:([^:)]+) callref-([^ ]+) [^)]+\) new state ([^ ]+) -> ([^ ]+)'</span><br><span style="color: hsl(120, 100%, 40%);">+ l3type, callref_hex, from_state, to_state = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ e = mo_mt_from_l3type(l3type)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.callrefs_mo_mt[callref_hex] = e</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MSC, '<>', '.', 'CC state:\\n%s' % to_state))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_log_mncc(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*trans\(CC[^) ]* [^ )]+:([^:)]+) callref-([^ ]+) [^)]+\) (tx|rx) (MNCC_[^ ]*)$'</span><br><span style="color: hsl(120, 100%, 40%);">+ l3type, callref_hex, tx_rx, mncc_msg = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.seen_udtrace_mncc:</span><br><span style="color: hsl(120, 100%, 40%);">+ # If no udtrace is present, take the MNCC logging.</span><br><span style="color: hsl(120, 100%, 40%);">+ # But if there is udtrace logging available, we should not duplicate those MNCC lines.</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ tx = (tx_rx == 'tx')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ e = self.callrefs_mo_mt.get(callref_hex, MT)</span><br><span style="color: hsl(120, 100%, 40%);">+ except:</span><br><span style="color: hsl(120, 100%, 40%);">+ e = MT</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MSC, '>' if tx else '<', 'sip', mncc_msg))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ RE_MNCC_RTP = re.compile(' ip := ([^, ]+), rtp_port := ([0-9]+),')</span><br><span style="color: hsl(120, 100%, 40%);">+ RE_MNCC_CALLREF = re.compile(' callref := ([^ ,]+), ')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_udtrace_mncc(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*(write|recv).* (Tx|Rx): \{ msg_type := ([^ ]+) .* u := \{ (.*) \} \}$'</span><br><span style="color: hsl(120, 100%, 40%);">+ write_recv, tx_rx, msg_type, u = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.seen_udtrace_mncc = True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ tx = (tx_rx == 'Tx')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ callref_intstr = None</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_MNCC_CALLREF.finditer(u):</span><br><span style="color: hsl(120, 100%, 40%);">+ callref_intstr = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not callref_intstr:</span><br><span style="color: hsl(120, 100%, 40%);">+ # log only MNCC that has a callref</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ e = self.callrefs_mo_mt.get(Callref.int_to_hex(callref_intstr), MT)</span><br><span style="color: hsl(120, 100%, 40%);">+ except:</span><br><span style="color: hsl(120, 100%, 40%);">+ e = MT</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ descr = msg_type</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in Parse.RE_MNCC_RTP.finditer(u):</span><br><span style="color: hsl(120, 100%, 40%);">+ ip_str, port_str = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ if int(ip_str) == 0 or int(port_str) == 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+ except:</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+ ip_hex = int_to_hex(ip_str, 32)</span><br><span style="color: hsl(120, 100%, 40%);">+ ip = []</span><br><span style="color: hsl(120, 100%, 40%);">+ ip_val = int(ip_hex, 16)</span><br><span style="color: hsl(120, 100%, 40%);">+ for byte in range(4):</span><br><span style="color: hsl(120, 100%, 40%);">+ ip.insert(0, (ip_val & (0xff << (8*byte))) >> (8*byte))</span><br><span style="color: hsl(120, 100%, 40%);">+ rtp_info = '%s:%s' % ('.'.join(str(b) for b in ip), port_str)</span><br><span style="color: hsl(120, 100%, 40%);">+ rtp_info = self.mask_value('IP:port', rtp_info)</span><br><span style="color: hsl(120, 100%, 40%);">+ descr = '%s\\n%s' % (descr, rtp_info)</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MSC, '>' if tx else '<', 'sip', descr))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def rule_cc_timer(self, m):</span><br><span style="color: hsl(120, 100%, 40%);">+ r'.*trans\(CC.*IMSI-([0-9]+):.*\) (starting|stopping pending) (guard timer|timer T[0-9]+)( with ([0-9]+) seconds|)'</span><br><span style="color: hsl(120, 100%, 40%);">+ imsi, start_stop, t, with_cond, s = m.groups()</span><br><span style="color: hsl(120, 100%, 40%);">+ start = (start_stop == 'starting')</span><br><span style="color: hsl(120, 100%, 40%);">+ e = MO_MT_UNKNOWN</span><br><span style="color: hsl(120, 100%, 40%);">+ if start:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MSC, '[]', '.', 'CC starts %s (%ss)' % (t, s), imsi=imsi))</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.diagram.add_line(Arrow(e, MSC, '[]', '.', 'CC stops %s' % (t), imsi=imsi))</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def translate(inf, outf, cmdline):</span><br><span style="color: hsl(120, 100%, 40%);">+ if cmdline.output_format == 'mscgen':</span><br><span style="color: hsl(120, 100%, 40%);">+ output = OutputMscgen(outf, cmdline.start_with_re)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ output = OutputLadder(outf, cmdline.start_with_re)</span><br><span style="color: hsl(120, 100%, 40%);">+ parse = Parse(output, cmdline.mask_values)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ parse.start()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ while inf.readable():</span><br><span style="color: hsl(120, 100%, 40%);">+ line = inf.readline()</span><br><span style="color: hsl(120, 100%, 40%);">+ if not line:</span><br><span style="color: hsl(120, 100%, 40%);">+ break;</span><br><span style="color: hsl(120, 100%, 40%);">+ parse.add_line(line)</span><br><span style="color: hsl(120, 100%, 40%);">+ parse.end()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def open_output(inf, cmdline):</span><br><span style="color: hsl(120, 100%, 40%);">+ if cmdline.output_file == '-':</span><br><span style="color: hsl(120, 100%, 40%);">+ translate(inf, sys.stdout, cmdline)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ with tempfile.NamedTemporaryFile(dir=os.path.dirname(cmdline.output_file), mode='w', encoding='utf-8') as tmp_out:</span><br><span style="color: hsl(120, 100%, 40%);">+ translate(inf, tmp_out, cmdline)</span><br><span style="color: hsl(120, 100%, 40%);">+ if os.path.exists(cmdline.output_file):</span><br><span style="color: hsl(120, 100%, 40%);">+ os.unlink(cmdline.output_file)</span><br><span style="color: hsl(120, 100%, 40%);">+ os.link(tmp_out.name, cmdline.output_file)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def open_input(cmdline):</span><br><span style="color: hsl(120, 100%, 40%);">+ if cmdline.input_file == '-':</span><br><span style="color: hsl(120, 100%, 40%);">+ open_output(sys.stdin, cmdline)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ with open(cmdline.input_file, 'r') as f:</span><br><span style="color: hsl(120, 100%, 40%);">+ open_output(f, cmdline)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def main(cmdline):</span><br><span style="color: hsl(120, 100%, 40%);">+ open_input(cmdline)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+if __name__ == '__main__':</span><br><span style="color: hsl(120, 100%, 40%);">+ parser = argparse.ArgumentParser(description=doc)</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument('-i', '--input-file', dest='input_file', default="-",</span><br><span style="color: hsl(120, 100%, 40%);">+ help='Read from this file, or stdin if "-"')</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument('-o', '--output-file', dest='output_file', default="-",</span><br><span style="color: hsl(120, 100%, 40%);">+ help='Write to this file, or stdout if "-"')</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument('-t', '--output-format', dest='output_format', default="mscgen",</span><br><span style="color: hsl(120, 100%, 40%);">+ choices=('mscgen','ladder'),</span><br><span style="color: hsl(120, 100%, 40%);">+ help='Pick output format: mscgen format or libosmocore/contrib/ladder_to_msc.py format')</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument('-m', '--mask-values', dest='mask_values', action='store_true',</span><br><span style="color: hsl(120, 100%, 40%);">+ help='Do not output specific values like IP address, port, endpoint CI, instead just indicate that a value is'</span><br><span style="color: hsl(120, 100%, 40%);">+ ' present. This makes the output reproducable across various logs.')</span><br><span style="color: hsl(120, 100%, 40%);">+ parser.add_argument('-s', '--start-with', dest='start_with_re', default=None,</span><br><span style="color: hsl(120, 100%, 40%);">+ help='Skip until the first message with this label (regex), e.g. -s "CC SETUP"')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ cmdline = parser.parse_args()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ main(cmdline)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# vim: shiftwidth=8 noexpandtab tabstop=8 autoindent nocindent</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-msc/+/15944">change 15944</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-msc/+/15944"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-msc </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I2e4d8778f7b83dee558517a9b23450b817ee325d </div>
<div style="display:none"> Gerrit-Change-Number: 15944 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: neels <nhofmeyr@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins Builder </div>
<div style="display:none"> Gerrit-Reviewer: laforge <laforge@osmocom.org> </div>
<div style="display:none"> Gerrit-Reviewer: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>