<p>fixeria has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/18923">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">library/RSL_Emulation: implement waiting queue for DCHAN messages<br><br>Since change [1] has been merged, we see multiple regressions in<br>ttcn3-bsc-test (all LCLS test cases) and ttcn3-bsc-test-sccplite<br>(sporadic failures). In all failed cases, the reason is similar:<br><br> RSL for unknown Dchan<br> BSC_Tests.ttcn:4501 BSC_Tests control part<br> BSC_Tests.ttcn:2176 TC_assignment_codec_fr testcase<br><br>The mentioned change enables TCP_NODELAY option for all IPA based<br>connections, including both OML and RSL. This option disables<br>Nagle's algorithm [2], so we get less delays on IPA based links.<br><br>It took me a lot of time to investigate, and finally, I figured<br>out what is actually causing those regressions. The TCP_NODELAY<br>itself is not a problem, of course. As it turned out, the<br>problem is here, in our TTCN-3 test case framework.<br><br>Each test case involves several components (actors) running in parallel.<br>One of them is RSL_Emulation_CT, which is responsible for handling and<br>routing of RSL messages between the connected components.<br><br>A test case may register dedicated channel handlers by calling<br>f_rslem_register(), so DCHAN/RLL/IPACCESS messages will be matched<br>by RslChannelNr/TrxNr and routed to the corresponding one.<br><br>If no handler is found for a given RSL message, the RSL_Emulation_CT<br>would abort the test case execution. And that's where the problem is.<br><br>Given that all components are running in parallel, it may happen<br>that a received RSL message would be processed by the RSL emulation<br>component faster than the test case would call f_rslem_register().<br>The test case would be aborted due to "RSL for unknown Dchan".<br><br>Speaking in context of the failing BSC test cases, a test case<br>calls f_rslem_register() on receipt of an Assignment Command as<br>it contains all the assignment parameters. After that we expect<br>to receive an RSL ip.access CRCX for that channel.<br><br>The problem is that both Assignment Command and ip.access CRCX<br>messages are sent by the BSC simultaneously, so the later may<br>be handled faster than the first one. Race condition!<br><br>Let's work this around by maintaining a waiting queue, where the<br>messages, for which no handler was found, will be kept until the<br>corresponding dedicated channel is registered.<br><br>This is an optional feature that will be enabled by default, but<br>can be disabled by setting 'mp_rslem_enable_queue' to false.<br><br>Makes all LCLS test cases pass on my machine with TCP_NODELAY enabled.<br><br>[1] Ia3d4c41bf0659e682f0b7ae5f3d58ed0f28edb58<br>[2] https://en.wikipedia.org/wiki/Nagle%27s_algorithm<br><br>Change-Id: I25e10e28de174337233e6a3bb32cc16f2d7d614e<br>Related: OS#4619<br>---<br>M library/RSL_Emulation.ttcn<br>1 file changed, 38 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks refs/changes/23/18923/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/library/RSL_Emulation.ttcn b/library/RSL_Emulation.ttcn</span><br><span>index 618cf11..6108b4f 100644</span><br><span>--- a/library/RSL_Emulation.ttcn</span><br><span>+++ b/library/RSL_Emulation.ttcn</span><br><span>@@ -39,6 +39,10 @@</span><br><span> /* Work around switch for ttcn3-bts-test-latest, enables patching of IPA</span><br><span> * stream ID in the "BSC" mode. See I5927f59a49724170a63e87be604973f7c9d5d8be. */</span><br><span> boolean mp_rslem_patch_ipa_cid := false;</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Whether to keep RSL messages, for which no handler is found in ConnectionTable,</span><br><span style="color: hsl(120, 100%, 40%);">+ * in a queue. These messages will remain in the queue until the appropriate</span><br><span style="color: hsl(120, 100%, 40%);">+ * connection handler is registered. */</span><br><span style="color: hsl(120, 100%, 40%);">+ boolean mp_rslem_enable_queue := true;</span><br><span> };</span><br><span> </span><br><span> /* General "base class" component definition, of which specific implementations</span><br><span>@@ -415,6 +419,8 @@</span><br><span> testcase.stop("Failed to patch IPA stream ID in ASP RSL UD: ", ud);</span><br><span> }</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+private type record of ASP_RSL_Unitdata ASP_RSL_UDList;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> type component RSL_Emulation_CT {</span><br><span> /* port facing down towards IPA emulation */</span><br><span> port IPA_RSL_PT IPA_PT;</span><br><span>@@ -428,6 +434,9 @@</span><br><span> /* state of all concurrent connections / dedicated channels */</span><br><span> var ConnectionData ConnectionTable[64];</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ /* RSL messages for which no handler is currently registered */</span><br><span style="color: hsl(120, 100%, 40%);">+ var ASP_RSL_UDList WaitingQueue := { };</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> /* last RSL CHAN ACT for each chan_nr */</span><br><span> var LastActData LastActTable[64];</span><br><span> </span><br><span>@@ -584,6 +593,9 @@</span><br><span> rx_rsl.rsl.ies[0].body.chan_nr);</span><br><span> if (cid != -1) {</span><br><span> CLIENT_PT.send(rx_rsl.rsl) to ConnectionTable[cid].comp_ref;</span><br><span style="color: hsl(120, 100%, 40%);">+ } else if (mp_rslem_enable_queue) {</span><br><span style="color: hsl(120, 100%, 40%);">+ log("Storing an RSL message in the waiting queue");</span><br><span style="color: hsl(120, 100%, 40%);">+ WaitingQueue := WaitingQueue & { rx_rsl };</span><br><span> } else {</span><br><span> setverdict(fail, "RSL for unknown Dchan");</span><br><span> mtc.stop;</span><br><span>@@ -630,6 +642,10 @@</span><br><span> [] RSL_PROC.getcall(RSLEM_register:{?,?,?}) -> param(trx_nr, chan_nr, vc_conn) {</span><br><span> f_cid_create_cnr(trx_nr, chan_nr, vc_conn);</span><br><span> RSL_PROC.reply(RSLEM_register:{trx_nr, chan_nr, vc_conn}) to vc_conn;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Dispatch pending messages to the new handler (if any) */</span><br><span style="color: hsl(120, 100%, 40%);">+ if (mp_rslem_enable_queue)</span><br><span style="color: hsl(120, 100%, 40%);">+ { f_WaitingQueue_process(trx_nr, chan_nr, vc_conn); }</span><br><span> }</span><br><span> </span><br><span> [] RSL_PROC.getcall(RSLEM_unregister:{?,?,?}) -> param(trx_nr, chan_nr, vc_conn) {</span><br><span>@@ -668,6 +684,28 @@</span><br><span> }</span><br><span> }</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+private function f_WaitingQueue_process(uint8_t trx_nr, RslChannelNr chan_nr,</span><br><span style="color: hsl(120, 100%, 40%);">+ RSL_DchanHdlr comp_ref)</span><br><span style="color: hsl(120, 100%, 40%);">+runs on RSL_Emulation_CT {</span><br><span style="color: hsl(120, 100%, 40%);">+ var ASP_RSL_UDList NewWaitingQueue := { };</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ /* NOTE: we cannot (safely) remove elements in the loop, so we create</span><br><span style="color: hsl(120, 100%, 40%);">+ * a new queue with not matching messages and then replace the old one. */</span><br><span style="color: hsl(120, 100%, 40%);">+ for (var integer i := 0; i < lengthof(WaitingQueue); i := i + 1) {</span><br><span style="color: hsl(120, 100%, 40%);">+ if (f_trx_by_streamId(WaitingQueue[i].streamId) == trx_nr and</span><br><span style="color: hsl(120, 100%, 40%);">+ WaitingQueue[i].rsl.ies[0].body.chan_nr == chan_nr) {</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Matched, send this message to the given RSL_DchanHdlr */</span><br><span style="color: hsl(120, 100%, 40%);">+ log("Dispatching a stalled message to ", comp_ref);</span><br><span style="color: hsl(120, 100%, 40%);">+ CLIENT_PT.send(WaitingQueue[i].rsl) to comp_ref;</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Does not match, put this one into the new queue */</span><br><span style="color: hsl(120, 100%, 40%);">+ NewWaitingQueue := NewWaitingQueue & { WaitingQueue[i] };</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ WaitingQueue := NewWaitingQueue;</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> /* client/conn_hdlr side function to use procedure port to register stream_id/chan_nr */</span><br><span> function f_rslem_register(uint8_t trx_nr, RslChannelNr chan_nr, RSLEM_PROC_PT PT := RSL_PROC)</span><br><span> runs on RSL_DchanHdlr {</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/18923">change 18923</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/18923"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-ttcn3-hacks </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I25e10e28de174337233e6a3bb32cc16f2d7d614e </div>
<div style="display:none"> Gerrit-Change-Number: 18923 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: fixeria <vyanitskiy@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>