<p>neels <strong>submitted</strong> this change.</p><p><a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21512">View Change</a></p><div style="white-space:pre-wrap"></div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">process: add get_output_mark() and grep_output()<br><br>Allow showing log lines matching specific regexes, from a specific start<br>point of a log.<br><br>My use case is to echo the handover related logging after an expected<br>handover failed, so that the reason is visible already in the console<br>output of a jenkins run. So far I would need to open the endless bsc log<br>and look up the matching place in it to get a conclusion about why a<br>handover failed.<br><br>Change-Id: Ib6569f7486e9d961bd79a5f24232e58d053667a1<br>---<br>M src/osmo_gsm_tester/core/process.py<br>1 file changed, 53 insertions(+), 8 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/core/process.py b/src/osmo_gsm_tester/core/process.py</span><br><span>index 0b96f2e..bff8dba 100644</span><br><span>--- a/src/osmo_gsm_tester/core/process.py</span><br><span>+++ b/src/osmo_gsm_tester/core/process.py</span><br><span>@@ -23,6 +23,7 @@</span><br><span> import signal</span><br><span> from abc import ABCMeta, abstractmethod</span><br><span> from datetime import datetime</span><br><span style="color: hsl(120, 100%, 40%);">+import re</span><br><span> </span><br><span> from . import log</span><br><span> from .event_loop import MainLoop</span><br><span>@@ -320,13 +321,24 @@</span><br><span> self.poll()</span><br><span> return self.process_obj is not None and self.result is None</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def get_output(self, which):</span><br><span style="color: hsl(0, 100%, 40%);">- ''' Read process output '''</span><br><span style="color: hsl(120, 100%, 40%);">+ @staticmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def end_ansi_colors(txt):</span><br><span style="color: hsl(120, 100%, 40%);">+ '''Make sure no ANSI colors leak out of logging output'''</span><br><span style="color: hsl(120, 100%, 40%);">+ color_off = '\033[0;m'</span><br><span style="color: hsl(120, 100%, 40%);">+ color_any = '\033['</span><br><span style="color: hsl(120, 100%, 40%);">+ if txt.rfind(color_any) > txt.rfind(color_off):</span><br><span style="color: hsl(120, 100%, 40%);">+ return txt + color_off</span><br><span style="color: hsl(120, 100%, 40%);">+ return txt</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_output(self, which, since_mark=0):</span><br><span style="color: hsl(120, 100%, 40%);">+ ''' Read process output. For since_mark, see get_output_mark(). '''</span><br><span> path = self.get_output_file(which)</span><br><span> if path is None:</span><br><span> return None</span><br><span style="color: hsl(0, 100%, 40%);">- with open(path, 'r') as f2:</span><br><span style="color: hsl(0, 100%, 40%);">- return f2.read()</span><br><span style="color: hsl(120, 100%, 40%);">+ with open(path, 'r') as f:</span><br><span style="color: hsl(120, 100%, 40%);">+ if since_mark > 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ f.seek(since_mark)</span><br><span style="color: hsl(120, 100%, 40%);">+ return f.read()</span><br><span> </span><br><span> def get_output_file(self, which):</span><br><span> ''' Return filename for given output '''</span><br><span>@@ -344,11 +356,44 @@</span><br><span> tail = min(len(out), tail)</span><br><span> return prefix + ('\n' + prefix).join(out[-tail:])</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def get_stdout(self):</span><br><span style="color: hsl(0, 100%, 40%);">- return self.get_output('stdout')</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_output_mark(self, which):</span><br><span style="color: hsl(120, 100%, 40%);">+ '''Usage:</span><br><span style="color: hsl(120, 100%, 40%);">+ # remember a start marker</span><br><span style="color: hsl(120, 100%, 40%);">+ my_mark = my_process.get_output_mark('stderr')</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def get_stderr(self):</span><br><span style="color: hsl(0, 100%, 40%);">- return self.get_output('stderr')</span><br><span style="color: hsl(120, 100%, 40%);">+ do_actions_that_produce_log_output()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ my_log = my_process.get_output('stderr', since_mark=my_mark)</span><br><span style="color: hsl(120, 100%, 40%);">+ # my_log contains the stderr of that process since the start marker.</span><br><span style="color: hsl(120, 100%, 40%);">+ '''</span><br><span style="color: hsl(120, 100%, 40%);">+ path = self.get_output_file(which)</span><br><span style="color: hsl(120, 100%, 40%);">+ if path is None:</span><br><span style="color: hsl(120, 100%, 40%);">+ return None</span><br><span style="color: hsl(120, 100%, 40%);">+ with open(path, 'r') as f:</span><br><span style="color: hsl(120, 100%, 40%);">+ return f.seek(0, 2)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def grep_output(self, which, regex, since_mark=0, line_nrs=False):</span><br><span style="color: hsl(120, 100%, 40%);">+ lines = self.get_output(which, since_mark=since_mark).splitlines()</span><br><span style="color: hsl(120, 100%, 40%);">+ if not lines:</span><br><span style="color: hsl(120, 100%, 40%);">+ return None</span><br><span style="color: hsl(120, 100%, 40%);">+ matches = []</span><br><span style="color: hsl(120, 100%, 40%);">+ r = re.compile(regex)</span><br><span style="color: hsl(120, 100%, 40%);">+ line_nr = since_mark</span><br><span style="color: hsl(120, 100%, 40%);">+ for line in lines:</span><br><span style="color: hsl(120, 100%, 40%);">+ line_nr += 1</span><br><span style="color: hsl(120, 100%, 40%);">+ if r.search(line):</span><br><span style="color: hsl(120, 100%, 40%);">+ line = self.end_ansi_colors(line)</span><br><span style="color: hsl(120, 100%, 40%);">+ if line_nrs:</span><br><span style="color: hsl(120, 100%, 40%);">+ matches.append((line_nr, line))</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ matches.append(line)</span><br><span style="color: hsl(120, 100%, 40%);">+ return matches</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_stdout(self, since_mark=0):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.get_output('stdout', since_mark=since_mark)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def get_stderr(self, since_mark=0):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.get_output('stderr', since_mark=since_mark)</span><br><span> </span><br><span> def get_stdout_tail(self, tail=10, prefix=''):</span><br><span> return self.get_output_tail('stdout', tail, prefix)</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21512">change 21512</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21512"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: Ib6569f7486e9d961bd79a5f24232e58d053667a1 </div>
<div style="display:none"> Gerrit-Change-Number: 21512 </div>
<div style="display:none"> Gerrit-PatchSet: 4 </div>
<div style="display:none"> Gerrit-Owner: neels <nhofmeyr@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins Builder </div>
<div style="display:none"> Gerrit-CC: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>