February 2025 - gerrit-log - lists.osmocom.org

[M] Change in osmocom-bb[master]: Revert "Revert "trx_toolkit/transceiver.py: implement the transmit bu...

by kirr

Attention is currently required from: fixeria. kirr has posted comments on this change by kirr. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39328?usp=email ) Change subject: Revert "Revert "trx_toolkit/transceiver.py: implement the transmit burst queue"" ...................................................................... Patch Set 5: (1 comment) Patchset: PS3: > Please see ... Sorry, please see https://gerrit.osmocom.org/c/osmocom-bb/+/39532 for why things were not working before. -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39328?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I41291708effdd2c767be680fff22ffbd9a56815e Gerrit-Change-Number: 39328 Gerrit-PatchSet: 5 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-Reviewer: Jenkins Builder Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: laforge <laforge(a)osmocom.org> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Comment-Date: Wed, 12 Feb 2025 15:47:18 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: kirr <kirr(a)nexedi.com> Comment-In-Reply-To: fixeria <vyanitskiy(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

[M] Change in osmocom-bb[master]: Revert "Revert "trx_toolkit/transceiver.py: implement the transmit bu...

by kirr

Attention is currently required from: fixeria. kirr has posted comments on this change by kirr. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39328?usp=email ) Change subject: Revert "Revert "trx_toolkit/transceiver.py: implement the transmit burst queue"" ...................................................................... Patch Set 5: (1 comment) Patchset: PS3: > See https://gerrit.osmocom.org/c/osmocom-bb/+/39392 . I could finally study things around and make fake_trx to work in multitrx setup with tx-queue activated. Therefore this patch is no longer WIP. Please see https://gerrit.osmocom.org/c/osmocom-bb/+/39328 for why things were not working. Please also see the following patches that are needed for fake_trx to be able to keep up with one BTS and several MS'es https://gerrit.osmocom.org/c/osmocom-bb/+/39532 https://gerrit.osmocom.org/c/osmocom-bb/+/39533 https://gerrit.osmocom.org/c/osmocom-bb/+/39534 https://gerrit.osmocom.org/c/osmocom-bb/+/39535 https://gerrit.osmocom.org/c/osmocom-bb/+/39536 https://gerrit.osmocom.org/c/osmocom-bb/+/39537 Kirill -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39328?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I41291708effdd2c767be680fff22ffbd9a56815e Gerrit-Change-Number: 39328 Gerrit-PatchSet: 5 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-Reviewer: Jenkins Builder Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: laforge <laforge(a)osmocom.org> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Comment-Date: Wed, 12 Feb 2025 15:46:02 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: kirr <kirr(a)nexedi.com> Comment-In-Reply-To: fixeria <vyanitskiy(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

[XS] Change in osmo-ccid-firmware[master]: out ep: fix swapped msgb list error handling

by Hoernchen

Attention is currently required from: laforge. Hoernchen has posted comments on this change by Hoernchen. ( https://gerrit.osmocom.org/c/osmo-ccid-firmware/+/39501?usp=email ) Change subject: out ep: fix swapped msgb list error handling ...................................................................... Patch Set 2: (1 comment) Patchset: PS2: > I wonder how this would have ever worked? I guess it never did? this is only called once at the begining when usb is in a known state to submit the first buffer, and only from the usb callback that handles data afterwards, so there was basically no way for it to fail. -- To view, visit https://gerrit.osmocom.org/c/osmo-ccid-firmware/+/39501?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmo-ccid-firmware Gerrit-Branch: master Gerrit-Change-Id: I2833f0aca3ffebfceaaf2a0681a9f8242817e722 Gerrit-Change-Number: 39501 Gerrit-PatchSet: 2 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Comment-Date: Wed, 12 Feb 2025 15:41:19 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: laforge <laforge(a)osmocom.org>

4 months, 3 weeks

1
0
0 0

[M] Change in osmocom-bb[master]: Revert "Revert "trx_toolkit/transceiver.py: implement the transmit bu...

by kirr

Attention is currently required from: fixeria. kirr has posted comments on this change by kirr. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39328?usp=email ) Change subject: Revert "Revert "trx_toolkit/transceiver.py: implement the transmit burst queue"" ...................................................................... Set Ready For Review -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39328?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I41291708effdd2c767be680fff22ffbd9a56815e Gerrit-Change-Number: 39328 Gerrit-PatchSet: 5 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-Reviewer: Jenkins Builder Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: laforge <laforge(a)osmocom.org> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Comment-Date: Wed, 12 Feb 2025 15:41:08 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: No

4 months, 3 weeks

1
0
0 0

[S] Change in osmocom-bb[master]: trx_toolkit/transceiver: Use with tx_queue_lock instead of manual acq...

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39535?usp=email ) Change subject: trx_toolkit/transceiver: Use with tx_queue_lock instead of manual acquire/release ...................................................................... trx_toolkit/transceiver: Use with tx_queue_lock instead of manual acquire/release - it is a bit faster - it is a bit more robust as the lock becomes released in case some exception is raised before reaching release Noticed while moving forwarding out of tx_queue_lock in I7d10c972c45b2b5765e7c3a28f8646508b3c8a82. Change-Id: I74b194120bcc518d44796b57e36368bdc8de4aab --- M src/target/trx_toolkit/transceiver.py 1 file changed, 9 insertions(+), 11 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/35/39535/1 diff --git a/src/target/trx_toolkit/transceiver.py b/src/target/trx_toolkit/transceiver.py index 2a638c5..e6af013 100644 --- a/src/target/trx_toolkit/transceiver.py +++ b/src/target/trx_toolkit/transceiver.py @@ -312,18 +312,16 @@ emit = [] wait = [] - self._tx_queue_lock.acquire() + with self._tx_queue_lock: + for msg in self._tx_queue: + if msg.fn < fn: + drop.append(msg) + elif msg.fn == fn: + emit.append(msg) + else: + wait.append(msg) - for msg in self._tx_queue: - if msg.fn < fn: - drop.append(msg) - elif msg.fn == fn: - emit.append(msg) - else: - wait.append(msg) - - self._tx_queue = wait - self._tx_queue_lock.release() + self._tx_queue = wait for msg in emit: fwd.forward_msg(self, msg) -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39535?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I74b194120bcc518d44796b57e36368bdc8de4aab Gerrit-Change-Number: 39535 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

[M] Change in osmocom-bb[master]: trx_toolkit/*: Represent bursts as arrays instead of lists

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39536?usp=email ) Change subject: trx_toolkit/*: Represent bursts as arrays instead of lists ...................................................................... trx_toolkit/*: Represent bursts as arrays instead of lists Continuing fake_trx profiling story I noticed that on rx path a noticeable time is spent in converting from ubits to sbits via list comprehensions. By changing burst representation from py list, which stores each item as full python object, to an array, which stores each item as just byte, and by leveraging bytearray.translate, we can sped that conversion by ~ 10x: before: In [1]: from data_msg import Msg In [2]: burst = [0, 1] * (142//2) In [3]: burst Out[3]: [0, 1, 0, 1, 0, ... 0, 1] In [4]: Msg.ubit2sbit(burst) Out[4]: [127, -127, 127, -127, ... 127, -127] In [5]: %timeit Msg.ubit2sbit(burst) 3.01 µs ± 43.3 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each) after: In [2]: burst = bytearray([0, 1] * (142//2)) In [3]: burst Out[3]: bytearray(b'\x00\x01\x00\x01...\x00\x01') In [4]: Msg.ubit2sbit(burst) Out[4]: array('b', [127, -127, 127, -127, ... 127, -127]) In [5]: %timeit Msg.ubit2sbit(burst) 325 ns ± 12.1 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) Change-Id: I7314e9e79752e06fa86b9e346a9beacc5e59579e --- M src/target/trx_toolkit/data_msg.py M src/target/trx_toolkit/gsm_shared.py M src/target/trx_toolkit/rand_burst_gen.py M src/target/trx_toolkit/test_data_msg.py 4 files changed, 36 insertions(+), 25 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/36/39536/1 diff --git a/src/target/trx_toolkit/data_msg.py b/src/target/trx_toolkit/data_msg.py index 898a4ae..b5993d2 100644 --- a/src/target/trx_toolkit/data_msg.py +++ b/src/target/trx_toolkit/data_msg.py @@ -23,8 +23,13 @@ from typing import List from enum import Enum +from array import array from gsm_shared import * +# _bu2s converts unsigned byte to signed. +def _bu2s(x): + return struct.unpack('b', struct.pack('B', x))[0] + class Modulation(Enum): """ Modulation types defined in 3GPP TS 45.002 """ ModGMSK = (0b0000, 1 * GMSK_BURST_LEN) @@ -63,7 +68,7 @@ KNOWN_VERSIONS = (0, 1) def __init__(self, fn = None, tn = None, burst = None, ver = 0): - self.burst = burst + self.burst = burst # bytes|bytearray for ubit, array[b] for sbit, array[B] for usbit self.ver = ver self.fn = fn self.tn = tn @@ -126,24 +131,29 @@ return result @staticmethod - def usbit2sbit(bits: List[int]) -> List[int]: + def usbit2sbit(bits): # array[B] -> array[b] ''' Convert unsigned soft-bits {254..0} to soft-bits {-127..127}. ''' - return [-127 if (b == 0xff) else 127 - b for b in bits] + return array('b', bits.tobytes().translate(Msg._tab_usbit2sbit)) + _tab_usbit2sbit = array('b', [-127 if (b == 0xff) else 127 - b for b in range(0x100)]) @staticmethod - def sbit2usbit(bits: List[int]) -> List[int]: + def sbit2usbit(bits): # array[b] -> array[B] ''' Convert soft-bits {-127..127} to unsigned soft-bits {254..0}. ''' - return [127 - b for b in bits] + return array('B', bits.tobytes().translate(Msg._tab_sbit2usbit)) + _tab_sbit2usbit = array('B', [127 - _bu2s(b) for b in range(0x100)]) @staticmethod - def sbit2ubit(bits: List[int]) -> List[int]: + def sbit2ubit(bits): # array[b] -> bytearray ''' Convert soft-bits {-127..127} to bits {1..0}. ''' - return [int(b < 0) for b in bits] + return bytearray(bits).translate(Msg._tab_sbit2ubit) + _tab_sbit2ubit = array('B', [int(_bu2s(b) < 0) for b in range(0x100)]) @staticmethod - def ubit2sbit(bits: List[int]) -> List[int]: + def ubit2sbit(bits: bytearray): # -> array[b] ''' Convert bits {1..0} to soft-bits {-127..127}. ''' - return [-127 if b else 127 for b in bits] + return array('b', bits.translate(Msg._tab_ubit2sbit)) + _tab_ubit2sbit = array('b', [-127 if b else 127 for b in range(0x100)]) + def validate(self): ''' Validate the message fields (throws ValueError). ''' @@ -328,13 +338,13 @@ # Distinguish between GSM and EDGE if length >= EDGE_BURST_LEN: - self.burst = list(burst[:EDGE_BURST_LEN]) + self.burst = bytearray(burst[:EDGE_BURST_LEN]) else: - self.burst = list(burst[:GMSK_BURST_LEN]) + self.burst = bytearray(burst[:GMSK_BURST_LEN]) def rand_burst(self, length = GMSK_BURST_LEN): ''' Generate a random message specific burst. ''' - self.burst = [random.randint(0, 1) for _ in range(length)] + self.burst = bytearray([random.randint(0, 1) for _ in range(length)]) def trans(self, ver = None): ''' Transform this message into RxMsg. ''' @@ -659,7 +669,7 @@ def parse_burst(self, burst): ''' Parse message specific burst. ''' - burst = list(burst) + burst = array('B', burst) if self.ver == 0x00: burst = self._parse_burst_v0(burst) @@ -673,7 +683,7 @@ if length is None: length = self.mod_type.bl - self.burst = [random.randint(-127, 127) for _ in range(length)] + self.burst = array('b', [random.randint(-127, 127) for _ in range(length)]) def trans(self, ver = None): ''' Transform this message to TxMsg. ''' diff --git a/src/target/trx_toolkit/gsm_shared.py b/src/target/trx_toolkit/gsm_shared.py index 5f59ba6..977d2aa 100644 --- a/src/target/trx_toolkit/gsm_shared.py +++ b/src/target/trx_toolkit/gsm_shared.py @@ -81,7 +81,7 @@ self.tsc_set = tsc_set # Generate Training Sequence bits - self.seq = [int(x) for x in seq_str] + self.seq = bytearray(int(x) for x in seq_str) @classmethod def pick(self, burst): diff --git a/src/target/trx_toolkit/rand_burst_gen.py b/src/target/trx_toolkit/rand_burst_gen.py index c474c75..6cdab30 100644 --- a/src/target/trx_toolkit/rand_burst_gen.py +++ b/src/target/trx_toolkit/rand_burst_gen.py @@ -25,7 +25,7 @@ class RandBurstGen: # GSM 05.02 Chapter 5.2.6 Dummy Burst - db_bits = [ + db_bits = bytearray([ 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, @@ -37,7 +37,7 @@ 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, - ] + ]) # Pick a random TSC for a given burst type def get_rand_tsc(self, bt): @@ -72,11 +72,11 @@ # Tailing bits buf += [0] * 3 - return buf + return bytearray(buf) # Generate a frequency correction burst def gen_fb(self): - return [0] * GMSK_BURST_LEN + return bytearray([0] * GMSK_BURST_LEN) # Generate a synchronization burst def gen_sb(self, tsc = None): @@ -99,7 +99,7 @@ # Tailing bits buf += [0] * 3 - return buf + return bytearray(buf) # Generate a dummy burst def gen_db(self): @@ -126,4 +126,4 @@ # Guard period buf += [0] * 60 - return buf + return bytearray(buf) diff --git a/src/target/trx_toolkit/test_data_msg.py b/src/target/trx_toolkit/test_data_msg.py index ec6c2ec..13a64a6 100644 --- a/src/target/trx_toolkit/test_data_msg.py +++ b/src/target/trx_toolkit/test_data_msg.py @@ -19,6 +19,7 @@ # GNU General Public License for more details. import unittest +from array import array from data_msg import Msg, TxMsg, RxMsg @@ -136,8 +137,8 @@ # Validate bit conversations def test_bit_conv(self): - usbits_ref = list(range(0, 256)) - sbits_ref = list(range(-127, 128)) + usbits_ref = array('B', range(0, 256)) + sbits_ref = array('b', range(-127, 128)) # Test both usbit2sbit() and sbit2usbit() sbits = Msg.usbit2sbit(usbits_ref) @@ -147,10 +148,10 @@ # Test both sbit2ubit() and ubit2sbit() ubits = Msg.sbit2ubit(sbits_ref) - self.assertEqual(ubits, ([1] * 127 + [0] * 128)) + self.assertEqual(ubits, bytearray([1] * 127 + [0] * 128)) sbits = Msg.ubit2sbit(ubits) - self.assertEqual(sbits, ([-127] * 127 + [127] * 128)) + self.assertEqual(sbits, array('b', [-127] * 127 + [127] * 128)) def _test_transform(self, msg): # Prepare given messages -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39536?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I7314e9e79752e06fa86b9e346a9beacc5e59579e Gerrit-Change-Number: 39536 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

[M] Change in osmocom-bb[master]: trx_toolkit/*: Try to avoid copying burst data where possible

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39537?usp=email ) Change subject: trx_toolkit/*: Try to avoid copying burst data where possible ...................................................................... trx_toolkit/*: Try to avoid copying burst data where possible Conveying burst data is the primary flow in data place of what fake_trx does, so the less copies we do, the less we make CPU loaded. After this change I can finally run 1 BTS + 2 Mobile + 1 ccch_scan without hitting "Stale message ..." on fake_trx side. However fake_trx cpu load is close to 100% and there are internal clock overruns often: [WARNING] clck_gen.py:97 CLCKGen: time overrun by -1385us; resetting the clock [WARNING] clck_gen.py:97 CLCKGen: time overrun by -2657us; resetting the clock [WARNING] clck_gen.py:97 CLCKGen: time overrun by -1264us; resetting the clock [WARNING] clck_gen.py:97 CLCKGen: time overrun by -2913us; resetting the clock [WARNING] clck_gen.py:97 CLCKGen: time overrun by -1836us; resetting the clock ... This suggests that even though fake_trx.py + tx-queue started to work somehow, the rewrite of fake_trx in C, as explained in OS#6672, is still better to do. Change-Id: I147da2f110dedc863361059c931f609c28a69e9c --- M src/target/trx_toolkit/data_if.py M src/target/trx_toolkit/data_msg.py 2 files changed, 24 insertions(+), 34 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/37/39537/1 diff --git a/src/target/trx_toolkit/data_if.py b/src/target/trx_toolkit/data_if.py index 8e80894..5bc243f 100644 --- a/src/target/trx_toolkit/data_if.py +++ b/src/target/trx_toolkit/data_if.py @@ -67,7 +67,7 @@ # Attempt to parse a TRXD Tx message try: msg = TxMsg() - msg.parse_msg(bytearray(data)) + msg.parse_msg(data) except: log.error("Failed to parse a TRXD Tx message " "from R:%s:%u" % (self.remote_addr, self.remote_port)) diff --git a/src/target/trx_toolkit/data_msg.py b/src/target/trx_toolkit/data_msg.py index b5993d2..21641cb 100644 --- a/src/target/trx_toolkit/data_msg.py +++ b/src/target/trx_toolkit/data_msg.py @@ -79,16 +79,16 @@ return 1 + 4 # (VER + TN) + FN @abc.abstractmethod - def gen_hdr(self): - ''' Generate message specific header. ''' + def append_hdr_to(self, buf): + ''' Generate message specific header by appending it to buf. ''' @abc.abstractmethod def parse_hdr(self, hdr): ''' Parse message specific header. ''' @abc.abstractmethod - def gen_burst(self): - ''' Generate message specific burst. ''' + def append_burst_to(self, buf): + ''' Generate message specific burst by appending it to buf. ''' @abc.abstractmethod def parse_burst(self, burst): @@ -189,12 +189,11 @@ buf += struct.pack(">L", self.fn) # Generate message specific header part - hdr = self.gen_hdr() - buf += hdr + self.append_hdr_to(buf) # Generate burst if self.burst is not None: - buf += self.gen_burst() + self.append_burst_to(buf) # This is a rudiment from (legacy) OpenBTS transceiver, # some L1 implementations still expect two dummy bytes. @@ -228,7 +227,7 @@ self.parse_hdr(msg) # Copy burst, skipping header - msg_burst = msg[self.HDR_LEN:] + msg_burst = memoryview(msg)[self.HDR_LEN:] if len(msg_burst) > 0: self.parse_burst(msg_burst) else: @@ -308,28 +307,23 @@ # Strip useless whitespace and return return result.strip() - def gen_hdr(self): - ''' Generate message specific header part. ''' - - # Allocate an empty byte-array - buf = bytearray() + def append_hdr_to(self, buf): + ''' Generate message specific header by appending it to buf. ''' # Put power buf.append(self.pwr) - return buf - def parse_hdr(self, hdr): ''' Parse message specific header part. ''' # Parse power level self.pwr = hdr[5] - def gen_burst(self): - ''' Generate message specific burst. ''' + def append_burst_to(self, buf): + ''' Generate message specific burst by appending it to buf. ''' # Copy burst 'as is' - return bytearray(self.burst) + return buf.extend(self.burst) def parse_burst(self, burst): ''' Parse message specific burst. ''' @@ -338,9 +332,13 @@ # Distinguish between GSM and EDGE if length >= EDGE_BURST_LEN: - self.burst = bytearray(burst[:EDGE_BURST_LEN]) + if length > EDGE_BURST_LEN: + burst = burst[:EDGE_BURST_LEN] else: - self.burst = bytearray(burst[:GMSK_BURST_LEN]) + if length > GMSK_BURST_LEN: + burst = burst[:GMSK_BURST_LEN] + + self.burst = bytearray(burst) def rand_burst(self, length = GMSK_BURST_LEN): ''' Generate a random message specific burst. ''' @@ -602,11 +600,8 @@ self.mod_type = Modulation.ModGMSK self.tsc_set = mts & 0b11 - def gen_hdr(self): - ''' Generate message specific header part. ''' - - # Allocate an empty byte-array - buf = bytearray() + def append_hdr_to(self, buf): + ''' Generate message specific header by appending it to buf. ''' # Put RSSI buf.append(-self.rssi) @@ -623,8 +618,6 @@ # C/I: Carrier-to-Interference ratio (in centiBels) buf += struct.pack(">h", self.ci) - return buf - def parse_hdr(self, hdr): ''' Parse message specific header part. ''' @@ -641,14 +634,11 @@ # C/I: Carrier-to-Interference ratio (in centiBels) self.ci = struct.unpack(">h", hdr[9:11])[0] - def gen_burst(self): - ''' Generate message specific burst. ''' + def append_burst_to(self, buf): + ''' Generate message specific burst appending it to buf. ''' # Convert soft-bits to unsigned soft-bits - burst_usbits = self.sbit2usbit(self.burst) - - # Encode to bytes - return bytearray(burst_usbits) + buf.extend( self.sbit2usbit(self.burst) ) # XXX copies; can probably remove them with numpy def _parse_burst_v0(self, burst): ''' Parse message specific burst for header version 0. ''' -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39537?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I147da2f110dedc863361059c931f609c28a69e9c Gerrit-Change-Number: 39537 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

[XS] Change in osmocom-bb[master]: trx_toolkit/transceiver: Do not scan tx_queue twice on tx path

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39534?usp=email ) Change subject: trx_toolkit/transceiver: Do not scan tx_queue twice on tx path ...................................................................... trx_toolkit/transceiver: Do not scan tx_queue twice on tx path Noticed while moving forwarding out of tx_queue_lock in the previous patch. Change-Id: I225c44c4cc327b6786efce96d1278c6ec68fbc25 --- M src/target/trx_toolkit/transceiver.py 1 file changed, 4 insertions(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/34/39534/1 diff --git a/src/target/trx_toolkit/transceiver.py b/src/target/trx_toolkit/transceiver.py index 7297abf..2a638c5 100644 --- a/src/target/trx_toolkit/transceiver.py +++ b/src/target/trx_toolkit/transceiver.py @@ -310,6 +310,7 @@ drop = [] emit = [] + wait = [] self._tx_queue_lock.acquire() @@ -318,8 +319,10 @@ drop.append(msg) elif msg.fn == fn: emit.append(msg) + else: + wait.append(msg) - self._tx_queue = [msg for msg in self._tx_queue if msg.fn > fn] + self._tx_queue = wait self._tx_queue_lock.release() for msg in emit: -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39534?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I225c44c4cc327b6786efce96d1278c6ec68fbc25 Gerrit-Change-Number: 39534 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

[XS] Change in osmocom-bb[master]: trxcon: Advance Uplink TDMA Fn by default again

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39532?usp=email ) Change subject: trxcon: Advance Uplink TDMA Fn by default again ...................................................................... trxcon: Advance Uplink TDMA Fn by default again This essentially reverts 923e9b0b (trxcon: do not advance Uplink TDMA Fn by default; I838b1ebc54e4c5d116f8af2155d97215a6133ba4) for the following reason: In trxcon TRX clock is unused, because the signal from BTS is used as the master clock source instead (see 45c821ae/Ic8a5b6277c6b16392026e0557376257d71c9d230 "trxcon: get rid of the timer driven clock module" for details". Before restoration of tx-queue in fake_trx this was working ok even with fn-advance=0 on Ms side, but after I41291708effdd2c767be680fff22ffbd9a56815e (Revert "Revert "trx_toolkit/transceiver.py: implement the transmit burst queue"") fake_trx is sending frames having Fn when exactly same Fn happens corresponding on fake_trx clock. This results in BTS frames (that are sent with fn-advance=2 by default (see I7da3d0948f38e12342fb714b29f8edc5e9d0933d in osmo-bts.git and OS#4487) to be queued, waited to be sent, and then actually sent to Ms on fn=msg.fn . And then even if Ms replies immediately with that same fn, that message will be dropped by fake_trx as stalled, because fake_trx thinks that the message is too late since that fn already happened according to fake_trx clock. Here is a trace of how that looks like with 1 BTS and 1 MS(*): 7.106.927 CLOCK fn=80 # fake_trx running 7.111.592 CLOCK fn=81 7.116.289 CLOCK fn=82 7.120.949 CLOCK fn=83 7.125.523 CLOCK fn=84 7.130.000 CLOCK fn=85 7.134.575 CLOCK fn=86 ... 7.209.222 CLOCK fn=102 7.209.897 BTS -> fn=104 tn=0 # BTS starts to emit RF 7.210.221 BTS -> fn=104 tn=1 7.210.556 BTS -> fn=104 tn=2 7.210.796 BTS -> fn=104 tn=3 7.211.019 BTS -> fn=104 tn=4 7.211.234 BTS -> fn=104 tn=5 7.211.479 BTS -> fn=104 tn=6 7.211.768 BTS -> fn=104 tn=7 7.213.086 CLOCK fn=103 7.214.354 BTS -> fn=105 tn=0 7.214.566 BTS -> fn=105 tn=1 7.214.685 BTS -> fn=105 tn=2 7.214.792 BTS -> fn=105 tn=3 7.214.890 BTS -> fn=105 tn=4 7.214.985 BTS -> fn=105 tn=5 7.215.083 BTS -> fn=105 tn=6 7.215.184 BTS -> fn=105 tn=7 7.217.823 CLOCK fn=104 7.218.869 BTS -> fn=106 tn=0 7.219.092 BTS -> fn=106 tn=1 7.219.224 BTS -> fn=106 tn=2 7.219.330 BTS -> fn=106 tn=3 7.219.431 BTS -> fn=106 tn=4 7.219.527 BTS -> fn=106 tn=5 7.219.621 BTS -> fn=106 tn=6 7.219.718 BTS -> fn=106 tn=7 7.222.535 CLOCK fn=105 ... 9.995.869 CLOCK fn=706 # MS will soon connect. 9.997.138 BTS -> fn=709 tn=0 # Note: BTS is sending fn=709 before CLOCK fn=707 9.997.338 BTS -> fn=709 tn=1 # so this messages become queued before CLOCK fn=709 happens 9.997.444 BTS -> fn=709 tn=2 9.997.535 BTS -> fn=709 tn=3 9.997.620 BTS -> fn=709 tn=4 9.997.708 BTS -> fn=709 tn=5 9.997.789 BTS -> fn=709 tn=6 9.997.874 BTS -> fn=709 tn=7 10.000.583 CLOCK fn=707 10.001.735 BTS -> fn=710 tn=0 10.001.932 BTS -> fn=710 tn=1 10.002.041 BTS -> fn=710 tn=2 10.002.134 BTS -> fn=710 tn=3 10.002.220 BTS -> fn=710 tn=4 10.002.373 BTS -> fn=710 tn=5 10.002.459 BTS -> fn=710 tn=6 10.002.718 BTS -> fn=710 tn=7 [DEBUG] ctrl_if_trx.py:115 (MS) Recv POWEROFF cmd # MS starts to connect [INFO] ctrl_if_trx.py:117 (MS) Stopping transceiver... [DEBUG] ctrl_if_trx.py:229 (MS) Ignore CMD ECHO 10.005.203 CLOCK fn=708 [DEBUG] ctrl_if_trx.py:229 (MS) Ignore CMD SETSLOT 10.006.406 BTS -> fn=711 tn=0 [DEBUG] ctrl_if_trx.py:124 (MS) Recv RXTUNE cmd 10.006.999 BTS -> fn=711 tn=1 10.007.153 BTS -> fn=711 tn=2 [DEBUG] ctrl_if_trx.py:131 (MS) Recv TXTUNE cmd 10.007.590 BTS -> fn=711 tn=3 10.007.728 BTS -> fn=711 tn=4 [DEBUG] ctrl_if_trx.py:97 (MS) Recv POWERON CMD # MS connected and activated RF [INFO] ctrl_if_trx.py:109 (MS) Starting transceiver... 10.008.344 BTS -> fn=711 tn=5 10.008.471 BTS -> fn=711 tn=6 10.008.563 BTS -> fn=711 tn=7 10.009.868 CLOCK fn=709 # CLOCK fn=709 happens 10.009.987 MS <- fn=709 tn=0 # messages of BTS queued previously with that fn=709 are forwarded to Ms 10.010.696 MS <- fn=709 tn=1 10.010.904 MS -> fn=709 tn=0 # <-- MS sends UL message with that same fn=709 _before_ CLOCK fn=710 10.011.397 BTS -> fn=712 tn=0 10.011.507 MS <- fn=709 tn=2 10.011.770 MS <- fn=709 tn=3 10.011.968 MS <- fn=709 tn=4 10.012.156 MS <- fn=709 tn=5 10.012.342 MS <- fn=709 tn=6 10.012.527 MS <- fn=709 tn=7 10.012.914 BTS <- fn=709 tn=0 10.013.166 BTS -> fn=712 tn=1 10.013.524 MS -> fn=709 tn=1 # <-- MS sends UL message with that same fn=709 _before_ CLOCK fn=710 10.013.832 BTS -> fn=712 tn=2 10.013.949 MS -> fn=709 tn=2 # <-- MS sends UL message with that same fn=709 _before_ CLOCK fn=710 10.014.081 BTS -> fn=712 tn=3 10.014.177 MS -> fn=709 tn=3 # <-- MS sends UL message with that same fn=709 _before_ CLOCK fn=710 10.014.361 BTS -> fn=712 tn=4 10.014.475 CLOCK fn=710 # but most of those messages of MS with fn=709 are not picked up 10.014.713 MS -> fn=709 tn=4 # instantly and so become dropped as stale on CLOCK fn=710 10.014.815 MS <- fn=710 tn=0 10.015.032 BTS -> fn=712 tn=5 10.015.687 MS <- fn=710 tn=1 10.016.189 MS <- fn=710 tn=2 10.016.464 MS <- fn=710 tn=3 10.016.648 MS <- fn=710 tn=4 10.016.882 MS <- fn=710 tn=5 10.017.110 MS <- fn=710 tn=6 10.017.336 MS <- fn=710 tn=7 [WARNING] transceiver.py:321 (MS) Stale TRXD message (fn=710): fn=709 tn=1 pwr=0 [WARNING] transceiver.py:321 (MS) Stale TRXD message (fn=710): fn=709 tn=2 pwr=0 [WARNING] transceiver.py:321 (MS) Stale TRXD message (fn=710): fn=709 tn=3 pwr=0 [WARNING] transceiver.py:321 (MS) Stale TRXD message (fn=710): fn=709 tn=4 pwr=0 So without adding some fn-advance it is practically not possible for Ms to be on time with tx-queueing on TRX even if Ms sends its uplink frames right immediately after receiving downlink ones. This way Ms fn-advance has to be 1 at the minimum, so that immediate UL replies can in principle arrive before fn+1 happens on fake_trx side, even for tn=7. And it is also better to increase fn-advance once more by another +1, to compensate for possible jitter due to OS scheduling latencies and similar things. This way default fn-advance=2 on Ms side becomes symmetric to default fn-advance on BTS side and Ms<->BTS exchange starts to work ok even with tx-queueing activated on fake_trx. In theory it should be possible to reduce those fn-advances to 1 on both sides, but that will likely require to switch clock granularity from Fn to Tn increasing precision by an order of magnitude, which will likely also result in the need to make architectural change of moving trx to work inside BTS and MS instead of being separate service processes. That's a big task and I'm not delving into that here. Note: Uplink Fn advance > 0 is needed for Ms when working with regular TRX'es as well. The reason is exactly the same as explained above. In 923e9b0b the reason for setting fn-advance=0 by default was that trxcon is usually being used with fake_trx, and that with fake_trx it is not needed. But after reenabling tx-queueing we have to revisit even fake_trx case again. (*) the trace was captured with the help of the following debugging patch: --- b/src/target/trx_toolkit/burst_fwd.py +++ a/src/target/trx_toolkit/burst_fwd.py @@ -22,6 +22,18 @@ from trx_list import TRXList +import sys, time + +# trace logs msg to stderr with also marking it with high-resolution timestamp. +t0 = time.time() +def trace(msg): + t = time.time() - t0 + t_ms = int(t * 1e3) / 1e3 + us = int((t - t_ms) * 1e6) + print("%7.3f.%03d %s" % (t_ms, us, msg), file=sys.stderr) + class BurstForwarder(TRXList): """ Performs burst forwarding between transceivers. @@ -63,6 +75,7 @@ def forward_msg(self, src_trx, rx_msg): if trx.get_rx_freq(rx_msg.fn) != tx_freq: continue + trace("%s\t<- fn=%d\ttn=%d" % (trx, rx_msg.fn, rx_msg.tn)) # Transform from TxMsg to RxMsg and forward tx_msg = rx_msg.trans(ver = trx.data_if._hdr_ver) trx.handle_data_msg(src_trx, rx_msg, tx_msg) --- b/src/target/trx_toolkit/fake_trx.py +++ a/src/target/trx_toolkit/fake_trx.py @@ -29,7 +29,7 @@ import re from app_common import ApplicationBase -from burst_fwd import BurstForwarder +from burst_fwd import BurstForwarder, trace from transceiver import Transceiver from data_msg import Modulation from clck_gen import CLCKGen @@ -473,6 +473,7 @@ def run(self): # This method will be called by the clock thread def clck_handler(self, fn): + trace("CLOCK\tfn=%d" % fn) # We assume that this list is immutable at run-time for trx in self.trx_list.trx_list: trx.clck_tick(self.burst_fwd, fn) --- b/src/target/trx_toolkit/transceiver.py +++ a/src/target/trx_toolkit/transceiver.py @@ -25,6 +25,7 @@ from data_if import DATAInterface from udp_link import UDPLink from trx_list import TRXList +from burst_fwd import trace from gsm_shared import HoppingParams @@ -198,6 +199,7 @@ def __init__(self, bind_addr, remote_addr, base_port, **kwargs): self._tx_queue = [] def __str__(self): + return self.name desc = "%s:%d" % (self.remote_addr, self.base_port) if self.child_idx > 0: desc += "/%d" % self.child_idx @@ -289,6 +291,7 @@ def recv_data_msg(self): return None # Enque the message, it will be sent later + trace("%s\t-> fn=%d\ttn=%d" % (self, msg.fn, msg.tn)) self.tx_queue_append(msg) return msg Change-Id: Icf0b4568b44eb75ee0733391d94b0af86f27ee2e --- M src/host/trxcon/src/trxcon_main.c 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/32/39532/1 diff --git a/src/host/trxcon/src/trxcon_main.c b/src/host/trxcon/src/trxcon_main.c index 3901e33..78963b7 100644 --- a/src/host/trxcon/src/trxcon_main.c +++ b/src/host/trxcon/src/trxcon_main.c @@ -79,7 +79,7 @@ .trx_remote_ip = "127.0.0.1", .trx_bind_ip = "0.0.0.0", .trx_base_port = 6700, - .trx_fn_advance = 0, + .trx_fn_advance = 2, .phyq_fbsb_extend_fns = 0, }; @@ -184,7 +184,7 @@ printf(" -b --trx-bind TRX bind IP address (default 0.0.0.0)\n"); printf(" -i --trx-remote TRX remote IP address (default 127.0.0.1)\n"); printf(" -p --trx-port Base port of TRX instance (default 6700)\n"); - printf(" -f --trx-advance Uplink burst scheduling advance (default 0)\n"); + printf(" -f --trx-advance Uplink burst scheduling advance (default 2)\n"); printf(" -F --fbsb-extend FBSB timeout extension (in TDMA FNs, default 0)\n"); printf(" -s --socket Listening socket for layer23 (default /tmp/osmocom_l2)\n"); printf(" -g --gsmtap-ip The destination IP used for GSMTAP (disabled by default)\n"); -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39532?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: Icf0b4568b44eb75ee0733391d94b0af86f27ee2e Gerrit-Change-Number: 39532 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

[S] Change in osmocom-bb[master]: trx_toolkit/transceiver: Do not forward nor log from under tx_queue_lock

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/39533?usp=email ) Change subject: trx_toolkit/transceiver: Do not forward nor log from under tx_queue_lock ...................................................................... trx_toolkit/transceiver: Do not forward nor log from under tx_queue_lock Even though for 1 BTS + 1 MS fake_trx works ok with tx-queuing, when I try to run two ccch_scan's with 1 BTS fake_trx starts occupy ~ 100% of CPU and emits lots of "Stale ..." messages: [WARNING] transceiver.py:317 (M2@127.0.0.1:7700) Stale TRXD message (fn=2793): fn=2791 tn=7 pwr=0 [WARNING] transceiver.py:317 (M2@127.0.0.1:7700) Stale TRXD message (fn=2793): fn=2792 tn=0 pwr=0 [WARNING] transceiver.py:317 (M2@127.0.0.1:7700) Stale TRXD message (fn=2793): fn=2792 tn=1 pwr=0 [WARNING] transceiver.py:317 (M2@127.0.0.1:7700) Stale TRXD message (fn=2793): fn=2792 tn=2 pwr=0 [WARNING] transceiver.py:317 (M2@127.0.0.1:7700) Stale TRXD message (fn=2793): fn=2792 tn=3 pwr=0 ... Inspecting a bit with a profiler showed that fake_trx simply cannot keep up with the load. Let's try to fix this with optimizing things a bit where it is easy to notice and easy to pick up low-hanging fruits. This is the first patch in that optimization series. It moves blocking calls from out of under tx_queue_lock on transmit path. The reason for this move is not to block receive path while the transmit path is busy more than necessary. I originally noticed tx_queue_lock.acquire being visible in profile of the rx thread which indicates that tx/rx contention on this lock can really happen if we do non-negligible tasks from under this lock. Here, in particular, it was forward_msg that was preparing and actually sending RxMsg to destination. tx_queue_lock is needed only to protect tx_queue itself and synchronize rx and tx threads access to it. Once necessary items are appended or popped, we can do everything else out of this lock. -> Move everything on the tx codepath, not actually needing access to tx_queue out of this lock: - only collect messages to be sent under the lock; actually forward them after releasing the log; - same for logging. Change-Id: I7d10c972c45b2b5765e7c3a28f8646508b3c8a82 --- M src/target/trx_toolkit/transceiver.py 1 file changed, 14 insertions(+), 5 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/33/39533/1 diff --git a/src/target/trx_toolkit/transceiver.py b/src/target/trx_toolkit/transceiver.py index 4e1cb87..7297abf 100644 --- a/src/target/trx_toolkit/transceiver.py +++ b/src/target/trx_toolkit/transceiver.py @@ -308,14 +308,23 @@ if not self.running: return + drop = [] + emit = [] + self._tx_queue_lock.acquire() for msg in self._tx_queue: - if msg.fn == fn: - fwd.forward_msg(self, msg) - elif msg.fn < fn: - log.warning("(%s) Stale TRXD message (fn=%u): %s" - % (self, fn, msg.desc_hdr())) + if msg.fn < fn: + drop.append(msg) + elif msg.fn == fn: + emit.append(msg) self._tx_queue = [msg for msg in self._tx_queue if msg.fn > fn] self._tx_queue_lock.release() + + for msg in emit: + fwd.forward_msg(self, msg) + + for msg in drop: + log.warning("(%s) Stale TRXD message (fn=%u): %s" + % (self, fn, msg.desc_hdr())) -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/39533?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I7d10c972c45b2b5765e7c3a28f8646508b3c8a82 Gerrit-Change-Number: 39533 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months, 3 weeks

1
0
0 0

2025

2024

2023

2022

gerrit-log February 2025