<p>Holger Freyther has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/13085">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">process: Speed-up terminating lots of processes by batching it<br><br>Introduce a strategy to terminate processes and begin with an<br>implementation for parallel (that has no degree of parallelism<br>right now).<br><br>Change-Id: I7dd4a7e26aca758198aa08a434eaf5f3f5af632d<br>---<br>M src/osmo_gsm_tester/process.py<br>M src/osmo_gsm_tester/suite.py<br>2 files changed, 38 insertions(+), 10 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/85/13085/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/src/osmo_gsm_tester/process.py b/src/osmo_gsm_tester/process.py</span><br><span>index b1769f8..1e53aba 100644</span><br><span>--- a/src/osmo_gsm_tester/process.py</span><br><span>+++ b/src/osmo_gsm_tester/process.py</span><br><span>@@ -21,12 +21,38 @@</span><br><span> import time</span><br><span> import subprocess</span><br><span> import signal</span><br><span style="color: hsl(120, 100%, 40%);">+from abc import ABCMeta, abstractmethod</span><br><span> from datetime import datetime</span><br><span> </span><br><span> from . import log</span><br><span> from .event_loop import MainLoop</span><br><span> from .util import Dir</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+class TerminationStrategy(log.Origin, metaclass=ABCMeta):</span><br><span style="color: hsl(120, 100%, 40%);">+ """A baseclass for terminating a collection of processes."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self._processes = []</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def add_process(self, process):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Remembers a process that needs to be terminated."""</span><br><span style="color: hsl(120, 100%, 40%);">+ self._processes.append(process)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def terminate_all(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ "Terminates all scheduled processes and waits for the termination."""</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class ParallelTerminationStrategy(TerminationStrategy):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Processes will be terminated in parallel."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def terminate_all(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # TODO(zecke): Actually make this non-sequential.</span><br><span style="color: hsl(120, 100%, 40%);">+ for process in self._processes:</span><br><span style="color: hsl(120, 100%, 40%);">+ process.terminate()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> class Process(log.Origin):</span><br><span> </span><br><span> def __init__(self, name, run_dir, popen_args, **popen_kwargs):</span><br><span>diff --git a/src/osmo_gsm_tester/suite.py b/src/osmo_gsm_tester/suite.py</span><br><span>index e5ac9a8..39da917 100644</span><br><span>--- a/src/osmo_gsm_tester/suite.py</span><br><span>+++ b/src/osmo_gsm_tester/suite.py</span><br><span>@@ -23,7 +23,7 @@</span><br><span> import pprint</span><br><span> from . import config, log, util, resource, test</span><br><span> from .event_loop import MainLoop</span><br><span style="color: hsl(0, 100%, 40%);">-from . import osmo_nitb, osmo_hlr, osmo_mgcpgw, osmo_mgw, osmo_msc, osmo_bsc, osmo_stp, osmo_ggsn, osmo_sgsn, modem, esme, osmocon, ms_driver, iperf3</span><br><span style="color: hsl(120, 100%, 40%);">+from . import osmo_nitb, osmo_hlr, osmo_mgcpgw, osmo_mgw, osmo_msc, osmo_bsc, osmo_stp, osmo_ggsn, osmo_sgsn, modem, esme, osmocon, ms_driver, iperf3, process</span><br><span> </span><br><span> class Timeout(Exception):</span><br><span> pass</span><br><span>@@ -246,9 +246,11 @@</span><br><span> self._processes.insert(0, (process, respawn))</span><br><span> </span><br><span> def stop_processes(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ strategy = process.ParallelTerminationStrategy()</span><br><span> while self._processes:</span><br><span style="color: hsl(0, 100%, 40%);">- process, respawn = self._processes.pop()</span><br><span style="color: hsl(0, 100%, 40%);">- process.terminate()</span><br><span style="color: hsl(120, 100%, 40%);">+ proc, _ = self._processes.pop()</span><br><span style="color: hsl(120, 100%, 40%);">+ strategy.add_process(proc)</span><br><span style="color: hsl(120, 100%, 40%);">+ strategy.terminate_all()</span><br><span> </span><br><span> def stop_process(self, process):</span><br><span> 'Remove process from monitored list and stop it'</span><br><span>@@ -382,15 +384,15 @@</span><br><span> </span><br><span> def poll(self):</span><br><span> if self._processes:</span><br><span style="color: hsl(0, 100%, 40%);">- for process, respawn in self._processes:</span><br><span style="color: hsl(0, 100%, 40%);">- if process.terminated():</span><br><span style="color: hsl(120, 100%, 40%);">+ for proc, respawn in self._processes:</span><br><span style="color: hsl(120, 100%, 40%);">+ if proc.terminated():</span><br><span> if respawn == True:</span><br><span style="color: hsl(0, 100%, 40%);">- process.respawn()</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.respawn()</span><br><span> else:</span><br><span style="color: hsl(0, 100%, 40%);">- process.log_stdout_tail()</span><br><span style="color: hsl(0, 100%, 40%);">- process.log_stderr_tail()</span><br><span style="color: hsl(0, 100%, 40%);">- log.ctx(process)</span><br><span style="color: hsl(0, 100%, 40%);">- raise log.Error('Process ended prematurely: %s' % process.name())</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.log_stdout_tail()</span><br><span style="color: hsl(120, 100%, 40%);">+ proc.log_stderr_tail()</span><br><span style="color: hsl(120, 100%, 40%);">+ log.ctx(proc)</span><br><span style="color: hsl(120, 100%, 40%);">+ raise log.Error('Process ended prematurely: %s' % proc.name())</span><br><span> </span><br><span> def prompt(self, *msgs, **msg_details):</span><br><span> 'ask for user interaction. Do not use in tests that should run automatically!'</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/13085">change 13085</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/13085"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-gsm-tester </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: I7dd4a7e26aca758198aa08a434eaf5f3f5af632d </div>
<div style="display:none"> Gerrit-Change-Number: 13085 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Holger Freyther <holger@freyther.de> </div>