April 2025 - gerrit-log - lists.osmocom.org

[M] Change in osmocom-bb[master]: trx_toolkit/_fake_trx: Optimize IO loop fd polling

by Jenkins Builder

Jenkins Builder has posted comments on this change by kirr. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40064?usp=email ) Change subject: trx_toolkit/_fake_trx: Optimize IO loop fd polling ...................................................................... Patch Set 1: (1 comment) File src/target/trx_toolkit/_fake_trx.pyx: Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/osmocom-bb/+/40064/comment/93a8431a_d51fe8ea?u… : PS1, Line 485: cdef object trx # Tranceiver | None 'Tranceiver' may be misspelled - perhaps 'Transceiver'? -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40064?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I748810871601178cc97bcdaba41419949078c29d Gerrit-Change-Number: 40064 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: Jenkins Builder Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Comment-Date: Thu, 17 Apr 2025 09:53:12 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No

4 months

1
0
0 0

[L] Change in osmocom-bb[master]: trx_toolkit/fake_trx: Split it into fake_trx and _fake_trx modules

by Jenkins Builder

Jenkins Builder has posted comments on this change by kirr. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40051?usp=email ) Change subject: trx_toolkit/fake_trx: Split it into fake_trx and _fake_trx modules ...................................................................... Patch Set 1: (2 comments) File src/target/trx_toolkit/_fake_trx.py: Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/osmocom-bb/+/40051/comment/1b505f69_07fd4d5d?u… : PS1, Line 216: msg.burst = None # burst bits are omited 'omited' may be misspelled - perhaps 'omitted'? Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/osmocom-bb/+/40051/comment/28e7a449_edbf9a6e?u… : PS1, Line 218: # TODO: shoud we make these values configurable? 'shoud' may be misspelled - perhaps 'should'? -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40051?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: Iadffd49de8197564e57bfd9cb660b1d11136ffd4 Gerrit-Change-Number: 40051 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: Jenkins Builder Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Comment-Date: Thu, 17 Apr 2025 09:52:49 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No

4 months

1
0
0 0

[M] Change in osmocom-bb[master]: trx_toolkit/data_msg: Optimize RxMsg/TxMsg parsing

by Jenkins Builder

Jenkins Builder has posted comments on this change by kirr. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40078?usp=email ) Change subject: trx_toolkit/data_msg: Optimize RxMsg/TxMsg parsing ...................................................................... Patch Set 1: (2 comments) File src/target/trx_toolkit/data_msg.pyx: Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/osmocom-bb/+/40078/comment/840300e1_31f24f86?u… : PS1, Line 397: # NOTE header lenght is already pre-checked by _parse_msg 'lenght' may be misspelled - perhaps 'length'? Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/osmocom-bb/+/40078/comment/eb3a01c9_3c384345?u… : PS1, Line 710: # NOTE header lenght is already pre-checked by _parse_msg 'lenght' may be misspelled - perhaps 'length'? -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40078?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I661a414bf5091fa6b872831ba911465a8d073397 Gerrit-Change-Number: 40078 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: Jenkins Builder Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Comment-Date: Thu, 17 Apr 2025 09:51:44 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No

4 months

1
0
0 0

[S] Change in osmocom-bb[master]: trx_toolkit/burst_fwd: Optimize rx_msg.burst -> tx_msg.burst -> tx_ms...

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40087?usp=email ) Change subject: trx_toolkit/burst_fwd: Optimize rx_msg.burst -> tx_msg.burst -> tx_msg.payload by doing it in one go ...................................................................... trx_toolkit/burst_fwd: Optimize rx_msg.burst -> tx_msg.burst -> tx_msg.payload by doing it in one go fake_trx receives bursts and create TxMsg objects deserialized from received data. Then BurstForwarder is asked to forward that TxMsg to recipient transceivers. What happens on forwaring is: 1) TxMsg is converted to RxMsg. In particular TxMsg.burst is converted to RxMsg.burst by way of ubit2sbit transformation. 2) then TRX.handle_data_msg -> TRX.send_data_msg -> ... -> Msg._gen_msg -> RxMsg.append_burst_to serializes this burst to data to send via sbit2usbit And all that happens for every incoming burst multiplied by the number of recepient transceivers. We can save significant time by doing ubit -> sbit -> usbit transformation in one go instead of doing ubit -> sbit first, and then sbit -> usbit as second separate step. -> Adjust BurstForwarder to do so. It works by adding custom BurstForwarder-specific private field to RxMsg and by running added ubit2usbit transformation directly if RxMsg.append_burst_to sees this field set. Change-Id: Ie92ec20d0ba6bb3f78db50ddbee33bf9e259cc40 --- M src/target/trx_toolkit/burst_fwd.pyx M src/target/trx_toolkit/data_msg.pxd M src/target/trx_toolkit/data_msg.pyx M src/target/trx_toolkit/test_data_msg.py 4 files changed, 38 insertions(+), 3 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/87/40087/1 diff --git a/src/target/trx_toolkit/burst_fwd.pyx b/src/target/trx_toolkit/burst_fwd.pyx index db910ad..10b1b58 100644 --- a/src/target/trx_toolkit/burst_fwd.pyx +++ b/src/target/trx_toolkit/burst_fwd.pyx @@ -72,8 +72,8 @@ tx_msg.fn = rx_msg.fn tx_msg.tn = rx_msg.tn tx_msg.ver = trx.data_if._hdr_ver - if rx_msg.burst is not None: - tx_msg.burst = RxMsg._ubit2sbit(rx_msg.burst) + tx_msg.burst = None # skip creating tx_msg.burst - instead do + tx_msg._tburst = rx_msg.burst # rx_msg.burst -> tx_msg.payload.burst transformation in one go tx_msg.nope_ind = (rx_msg.burst is None) trx.handle_data_msg(src_trx, rx_msg, tx_msg) diff --git a/src/target/trx_toolkit/data_msg.pxd b/src/target/trx_toolkit/data_msg.pxd index 2ddd248..5dd14b9 100644 --- a/src/target/trx_toolkit/data_msg.pxd +++ b/src/target/trx_toolkit/data_msg.pxd @@ -48,6 +48,8 @@ cdef void __sbit2ubit(const int8_t *sbits, Py_ssize_t l, uint8_t *ubits) noexcept @staticmethod cdef void __ubit2sbit(const uint8_t *ubits, Py_ssize_t l, int8_t *sbits) noexcept + @staticmethod + cdef void __ubit2usbit(const uint8_t *ubits, Py_ssize_t l, uint8_t *usbits) noexcept @staticmethod cdef array.array[int8_t] _usbit2sbit(const uint8_t[::1] usbits) @@ -57,6 +59,8 @@ cdef array.array[uint8_t] _sbit2ubit(const int8_t[::1] sbits) @staticmethod cdef array.array[int8_t] _ubit2sbit(const uint8_t[::1] ubits) + @staticmethod + cdef array.array[uint8_t] _ubit2usbit(const uint8_t[::1] ubits) @final @@ -73,6 +77,7 @@ @no_gc # NOTE by default cython thinks that .burst might participate in cycle cdef class RxMsg(Msg): cdef public array.array burst # array.array[b] | None + cdef bytearray _tburst # bytearray | None untransformed burst from TxMsg from BurstForwarder cdef public int rssi # RSSI_UNSET - unset cdef public int toa256 # TOA256_UNSET - unset diff --git a/src/target/trx_toolkit/data_msg.pyx b/src/target/trx_toolkit/data_msg.pyx index 07b3d38..bdc3119 100644 --- a/src/target/trx_toolkit/data_msg.pyx +++ b/src/target/trx_toolkit/data_msg.pyx @@ -194,6 +194,15 @@ Msg.__ubit2sbit(&ubits[0], l, &sbits[0]) return sbits + @staticmethod + cdef array.array[uint8_t] _ubit2usbit(const uint8_t[::1] ubits): # array[B] -> array[B] + ''' Convert bits {1..0} to unsigned soft-bits {254..0}. ''' + l = len(ubits) + cdef array.array[uint8_t] usbits = array.clone(_Bempty, l, zero=False) + Msg.__ubit2usbit(&ubits[0], l, &usbits[0]) + return usbits + + # inplace versions of Xbit2Ybit @staticmethod cdef void __usbit2sbit(const uint8_t *usbits, Py_ssize_t l, int8_t *sbits) noexcept: @@ -219,6 +228,12 @@ b = ubits[i] sbits[i] = -127 if b else 127 + @staticmethod + cdef void __ubit2usbit(const uint8_t *ubits, Py_ssize_t l, uint8_t *usbits) noexcept: + for i in range(l): + b = ubits[i] + usbits[i] = 254 if b else 0 + # for tests (`@staticmethod cpdef` is not supported: github.com/cython/cython/issues/3327) @staticmethod @@ -229,6 +244,8 @@ def sbit2ubit(sbits): return Msg._sbit2ubit(sbits) @staticmethod def ubit2sbit(ubits): return Msg._ubit2sbit(ubits) + @staticmethod + def ubit2usbit(ubits): return Msg._ubit2usbit(ubits) cdef _validate(self): @@ -488,6 +505,7 @@ cdef _reset(self): Msg._reset(self) self.burst = None + self._tburst = None self.rssi = RSSI_UNSET self.toa256 = TOA256_UNSET @@ -749,7 +767,16 @@ ''' Generate message specific burst appending it to buf. ''' # Convert soft-bits to unsigned soft-bits + # + # but if we have burst coming from TxMsg in transformed form, do + # TxMsg (bits) -> RxMsg (signed soft-bits) -> unsigned soft-bits in one go l = PyByteArray_GET_SIZE(buf) + if self._tburst is not None: + PyByteArray_Resize(buf, l+len(self._tburst)) + Msg.__ubit2usbit(<uint8_t*>PyByteArray_AS_STRING(self._tburst), len(self._tburst), + <uint8_t*>&PyByteArray_AS_STRING(buf)[l]) + return + if self.burst is None: return PyByteArray_Resize(buf, l+len(self.burst)) diff --git a/src/target/trx_toolkit/test_data_msg.py b/src/target/trx_toolkit/test_data_msg.py index 8d1d014..8f73e11 100644 --- a/src/target/trx_toolkit/test_data_msg.py +++ b/src/target/trx_toolkit/test_data_msg.py @@ -146,13 +146,16 @@ self.assertEqual(usbits[:255], usbits_ref[:255]) self.assertEqual(usbits[255], 254) - # Test both sbit2ubit() and ubit2sbit() + # Test all sbit2ubit(), ubit2sbit() and ubit2usbit() ubits = Msg.sbit2ubit(sbits_ref) self.assertEqual(ubits, array('B', [1] * 127 + [0] * 128)) sbits = Msg.ubit2sbit(ubits) self.assertEqual(sbits, array('b', [-127] * 127 + [127] * 128)) + usbits = Msg.ubit2usbit(ubits) + self.assertEqual(usbits, array('B', [254] * 127 + [0]*128)) + def _test_transform(self, msg): # Prepare given messages msg.rand_hdr() -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40087?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: Ie92ec20d0ba6bb3f78db50ddbee33bf9e259cc40 Gerrit-Change-Number: 40087 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months

1
0
0 0

[S] Change in osmocom-bb[master]: trx_toolkit/data_msg: TODO random -> C

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40088?usp=email ) Change subject: trx_toolkit/data_msg: TODO random -> C ...................................................................... trx_toolkit/data_msg: TODO random -> C With this patch I'm completing my optimization work on fake_trx. I have not had time to convert random routines to C yet. In my testing I was not activating any FakeTRX random modes at all and so lack of C random support was not showing in the profiles. But if those random modes will be activated py-level random will certainly slow things down. Add TODO entries to fix that. For the reference compared to the state where Cython was just introduced in I2159a07bece13bda4f6ccd957063d4644d8b5e4f (trx_toolkit: Move FakeTRX-related performance-sensitive modules to Cython) we now perform significantly better: for 1 BTS + 2 ccch_scan fake_trx now takes ~ 25% of CPU without clock overruns. While at that Pyx-base state it was taking ~ 70% of CPU with frequent clock overruns. Profiles at pyx-base and hereby state can be seen here: http://navytux.spb.ru/~kirr/osmo/fake_trx/pyx-base.html http://navytux.spb.ru/~kirr/osmo/fake_trx/pyx-optimized.html and if comparing the effect of whole optimization work starting from Ic9e16720daeb348b5f9c535c24a682db53a93529 parented on 54a0052a (fake_trx: Remove unneeded field assignment) the effect is as follows: Running 1 BTS + 2 Mobiles without Frequency Hopping + 1 ccch_scan Before optimization work: fake_trx is occupying 2 CPU cores loading every core close to 100% with hitting clock overrun non-stop. After optimization work: fake_Trx is occupying 1 CPU core with loading it ~ 35% by only one thread without hitting any clock overrun. Change-Id: If2c091c0a729a04dcac954ab0905b6f3be32a533 --- M src/target/trx_toolkit/data_msg.pyx 1 file changed, 12 insertions(+), 12 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/88/40088/1 diff --git a/src/target/trx_toolkit/data_msg.pyx b/src/target/trx_toolkit/data_msg.pyx index bdc3119..d62dd63 100644 --- a/src/target/trx_toolkit/data_msg.pyx +++ b/src/target/trx_toolkit/data_msg.pyx @@ -131,11 +131,11 @@ def rand_fn(self): ''' Generate a random frame number. ''' - return random.randint(0, GSM_HYPERFRAME) + return random.randint(0, GSM_HYPERFRAME) # TODO -> C def rand_tn(self): ''' Generate a random timeslot number. ''' - return random.randint(0, 7) + return random.randint(0, 7) # TODO -> C def rand_hdr(self): ''' Randomize the message header. ''' @@ -393,7 +393,7 @@ if max is None: max = PWR_MAX - return random.randint(min, max) + return random.randint(min, max) # TODO -> C def rand_hdr(self): ''' Randomize message specific header. ''' @@ -457,7 +457,7 @@ def rand_burst(self, length = GMSK_BURST_LEN): ''' Generate a random message specific burst. ''' - self.burst = bytearray([random.randint(0, 1) for _ in range(length)]) + self.burst = bytearray([random.randint(0, 1) for _ in range(length)]) # TODO -> C def trans(self, ver = None): ''' Transform this message into RxMsg. ''' @@ -625,7 +625,7 @@ if max is None: max = RSSI_MAX - return random.randint(min, max) + return random.randint(min, max) # TODO -> C def rand_toa256(self, min = None, max = None): ''' Generate a random ToA (Time of Arrival) value. ''' @@ -636,7 +636,7 @@ if max is None: max = TOA256_MAX - return random.randint(min, max) + return random.randint(min, max) # TODO -> C def rand_hdr(self): ''' Randomize message specific header. ''' @@ -646,15 +646,15 @@ self.toa256 = self.rand_toa256() if self.ver >= 0x01: - self.mod_type = random.choice(_mod_registry) + self.mod_type = random.choice(_mod_registry) # TODO -> C if self.mod_type is ModGMSK: - self.tsc_set = random.randint(0, 3) + self.tsc_set = random.randint(0, 3) # TODO -> C else: - self.tsc_set = random.randint(0, 1) - self.tsc = random.randint(0, TSC_MAX) + self.tsc_set = random.randint(0, 1) # TODO -> C + self.tsc = random.randint(0, TSC_MAX) # TODO -> C # C/I: Carrier-to-Interference ratio - self.ci = random.randint(CI_MIN, CI_MAX) + self.ci = random.randint(CI_MIN, CI_MAX) # TODO -> C def desc_hdr(self): ''' Generate human-readable header description. ''' @@ -815,7 +815,7 @@ if length is None: length = self.mod_type.bl - self.burst = array.array('b', [random.randint(-127, 127) for _ in range(length)]) + self.burst = array.array('b', [random.randint(-127, 127) for _ in range(length)]) # TODO -> C def trans(self, ver = None): ''' Transform this message to TxMsg. ''' -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40088?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: If2c091c0a729a04dcac954ab0905b6f3be32a533 Gerrit-Change-Number: 40088 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months

1
0
0 0

[XS] Change in osmocom-bb[master]: trx_toolkit/*.pyx: Remove unused imports

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40089?usp=email ) Change subject: trx_toolkit/*.pyx: Remove unused imports ...................................................................... trx_toolkit/*.pyx: Remove unused imports Change-Id: Id16fe7cd83699c6e51f87c65e9a1969db92d13e5 --- M src/target/trx_toolkit/burst_fwd.pyx M src/target/trx_toolkit/data_msg.pyx 2 files changed, 0 insertions(+), 4 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/89/40089/1 diff --git a/src/target/trx_toolkit/burst_fwd.pyx b/src/target/trx_toolkit/burst_fwd.pyx index 10b1b58..7ebfb07 100644 --- a/src/target/trx_toolkit/burst_fwd.pyx +++ b/src/target/trx_toolkit/burst_fwd.pyx @@ -19,8 +19,6 @@ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. -import logging as log - cdef class BurstForwarder: """ Performs burst forwarding between transceivers. diff --git a/src/target/trx_toolkit/data_msg.pyx b/src/target/trx_toolkit/data_msg.pyx index d62dd63..dd8e05e 100644 --- a/src/target/trx_toolkit/data_msg.pyx +++ b/src/target/trx_toolkit/data_msg.pyx @@ -36,8 +36,6 @@ import random import abc -from typing import List - import gsm_shared cdef int GMSK_BURST_LEN = gsm_shared.GMSK_BURST_LEN cdef int EDGE_BURST_LEN = gsm_shared.EDGE_BURST_LEN -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40089?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: Id16fe7cd83699c6e51f87c65e9a1969db92d13e5 Gerrit-Change-Number: 40089 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months

1
0
0 0

[XS] Change in osmocom-bb[master]: trx_toolkit/transceiver: Use `msg is None` instead of `not msg`

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40090?usp=email ) Change subject: trx_toolkit/transceiver: Use `msg is None` instead of `not msg` ...................................................................... trx_toolkit/transceiver: Use `msg is None` instead of `not msg` `msg is None` are two pointers comparision. `not msg` goes to py-level and can generally invoke arbitrary python code. Change-Id: I1278bed0a83dfac13734467f5333fae30e19bd5a --- M src/target/trx_toolkit/transceiver.pyx 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/90/40090/1 diff --git a/src/target/trx_toolkit/transceiver.pyx b/src/target/trx_toolkit/transceiver.pyx index 901a637..2b408a5 100644 --- a/src/target/trx_toolkit/transceiver.pyx +++ b/src/target/trx_toolkit/transceiver.pyx @@ -277,7 +277,7 @@ cdef TxMsg recv_data_msg(self): # -> TxMsg | None # Read and parse data from socket msg = self.data_if.recv_tx_msg() - if not msg: + if msg is None: return None # Make sure that transceiver is configured and running -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40090?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I1278bed0a83dfac13734467f5333fae30e19bd5a Gerrit-Change-Number: 40090 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months

1
0
0 0

[M] Change in osmocom-bb[master]: trx_toolkit/data_msg: Optimize RxMsg/TxMsg serialization

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40080?usp=email ) Change subject: trx_toolkit/data_msg: Optimize RxMsg/TxMsg serialization ...................................................................... trx_toolkit/data_msg: Optimize RxMsg/TxMsg serialization Thread all calls related to serialization to be done at C-level and with explicit types. Replace struct.pack and bytearray .append and .extend with C-level custom routines to do so. Change-Id: Ia203d39c8906ca8bb2fac078a818219dcdcf2565 --- M src/target/trx_toolkit/data_if.pyx M src/target/trx_toolkit/data_msg.pxd M src/target/trx_toolkit/data_msg.pyx 3 files changed, 54 insertions(+), 13 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/80/40080/1 diff --git a/src/target/trx_toolkit/data_if.pyx b/src/target/trx_toolkit/data_if.pyx index 71a1878..a23a52b 100644 --- a/src/target/trx_toolkit/data_if.pyx +++ b/src/target/trx_toolkit/data_if.pyx @@ -103,7 +103,7 @@ cdef _send_msg(self, Msg msg, bint legacy = False): try: # Validate and encode a TRXD message - payload = msg.gen_msg(legacy) + payload = msg._gen_msg(legacy) except ValueError as e: log.error("Failed to encode a TRXD message ('%s') " "due to error: %s" % (msg.desc_hdr(), e)) diff --git a/src/target/trx_toolkit/data_msg.pxd b/src/target/trx_toolkit/data_msg.pxd index 6823397..29d1702 100644 --- a/src/target/trx_toolkit/data_msg.pxd +++ b/src/target/trx_toolkit/data_msg.pxd @@ -36,6 +36,7 @@ cdef parse_burst(self, bytearray burst) cdef int _parse_msg(self, bytearray msg) except -1 cdef _validate(self) + cdef bytearray _gen_msg(self, bint legacy) @staticmethod cdef void __usbit2sbit(const uint8_t *usbits, Py_ssize_t l, int8_t *sbits) noexcept diff --git a/src/target/trx_toolkit/data_msg.pyx b/src/target/trx_toolkit/data_msg.pyx index 34f6a12..98dbece 100644 --- a/src/target/trx_toolkit/data_msg.pyx +++ b/src/target/trx_toolkit/data_msg.pyx @@ -30,8 +30,10 @@ from cpython.bytearray cimport PyByteArray_AS_STRING, PyByteArray_GET_SIZE from xpy cimport PyByteArray_Resize +from libc.string cimport memcpy + + import random -import struct import abc from typing import List @@ -241,20 +243,22 @@ if self.tn < 0 or self.tn > 7: raise ValueError("TDMA time-slot %d is out of range" % self.tn) - def gen_msg(self, legacy = False): + def gen_msg(self, bint legacy = False): + return self._gen_msg(legacy) + cdef bytearray _gen_msg(self, bint legacy): ''' Generate a TRX DATA message. ''' # Validate all the fields self._validate() # Allocate an empty byte-array - buf = bytearray() + cdef bytearray buf = PyByteArray_FromStringAndSize(NULL, 0) # Put version (4 bits) and TDMA TN (3 bits) - buf.append((self.ver << 4) | (self.tn & 0x07)) + _append_u8(buf, (self.ver << 4) | (self.tn & 0x07)) # Put TDMA FN (4 octets, BE) - buf += struct.pack(">L", self.fn) + _append_u32(buf, self.fn) # Generate message specific header part self.append_hdr_to(buf) @@ -265,7 +269,7 @@ # This is a rudiment from (legacy) OpenBTS transceiver, # some L1 implementations still expect two dummy bytes. if legacy and self.ver == 0x00: - buf += bytearray(2) + _append_u16(buf, 0) return buf @@ -388,7 +392,7 @@ ''' Generate message specific header by appending it to buf. ''' # Put power - buf.append(self.pwr) + _append_u8(buf, self.pwr) cdef parse_hdr(self, bytearray msg): ''' Parse message specific header part. ''' @@ -404,7 +408,10 @@ # Copy burst 'as is' if self.burst is None: return - return buf.extend(self.burst) + l = PyByteArray_GET_SIZE(buf) + PyByteArray_Resize(buf, l+len(self.burst)) + memcpy(&PyByteArray_AS_STRING(buf)[l], + PyByteArray_AS_STRING(self.burst), len(self.burst)) cdef parse_burst(self, bytearray burst): ''' Parse message specific burst. ''' @@ -691,19 +698,20 @@ ''' Generate message specific header by appending it to buf. ''' # Put RSSI - buf.append(-self.rssi) + _append_u8(buf, -self.rssi) # Encode ToA (Time of Arrival) # Big endian, 2 bytes (int32_t) - buf += struct.pack(">h", self.toa256) + _append_s16(buf, self.toa256) if self.ver >= 0x01: # Modulation and Training Sequence info mts = self.gen_mts() - buf.append(mts) + _append_u8(buf, mts) + # C/I: Carrier-to-Interference ratio (in centiBels) - buf += struct.pack(">h", self.ci) + _append_s16(buf, self.ci) cdef parse_hdr(self, bytearray msg): ''' Parse message specific header part. ''' @@ -786,6 +794,38 @@ # ---- misc ---- +# _append_(u|s)(8|16|32) append to bytearray buf big-endian representation of specified integer v. + +cdef int _append_u8(bytearray buf, uint8_t v) except -1: + l = PyByteArray_GET_SIZE(buf) + PyByteArray_Resize(buf, l+1) + PyByteArray_AS_STRING(buf)[l] = v + + +cdef int _append_u16(bytearray buf, uint16_t v) except -1: + l = PyByteArray_GET_SIZE(buf) + PyByteArray_Resize(buf, l+2) + x = &PyByteArray_AS_STRING(buf)[l] + x[0] = (v >> 8) & 0xff + x[1] = (v >> 0) & 0xff + +cdef int _append_s16(bytearray buf, int16_t v) except -1: + l = PyByteArray_GET_SIZE(buf) + PyByteArray_Resize(buf, l+2) + x = &PyByteArray_AS_STRING(buf)[l] + x[0] = <uint8_t>((v >> 8) & 0xff) + x[1] = <uint8_t>((v >> 0) & 0xff) + +cdef int _append_u32(bytearray buf, uint32_t v) except -1: + l = PyByteArray_GET_SIZE(buf) + PyByteArray_Resize(buf, l+4) + x = &PyByteArray_AS_STRING(buf)[l] + x[0] = (v >> 24) & 0xff + x[1] = (v >> 16) & 0xff + x[2] = (v >> 8) & 0xff + x[3] = (v >> 0) & 0xff + + # _unpack_(u|s)(16|32) deserialize big-endian representation of corresponding integer type from memory pointed by b. cdef int16_t _unpack_s16(const uint8_t *b): -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40080?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: Ia203d39c8906ca8bb2fac078a818219dcdcf2565 Gerrit-Change-Number: 40080 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months

1
0
0 0

[S] Change in osmocom-bb[master]: trx_toolkit/data_msg: Switch Msg .fn/.tn from object to int64_t

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40081?usp=email ) Change subject: trx_toolkit/data_msg: Switch Msg .fn/.tn from object to int64_t ...................................................................... trx_toolkit/data_msg: Switch Msg .fn/.tn from object to int64_t This are integer fields, assigning them C-level type avoid py-level overhead when accessing and doing arithmetics with them. Change-Id: Iea31825f974734e727fe05da9e78ee67afb59d5b --- M src/target/trx_toolkit/_fake_trx.pyx M src/target/trx_toolkit/data_msg.pxd M src/target/trx_toolkit/data_msg.pyx M src/target/trx_toolkit/fake_pm.py M src/target/trx_toolkit/test_data_msg.py M src/target/trx_toolkit/transceiver.pxd M src/target/trx_toolkit/transceiver.pyx 7 files changed, 20 insertions(+), 18 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/81/40081/1 diff --git a/src/target/trx_toolkit/_fake_trx.pyx b/src/target/trx_toolkit/_fake_trx.pyx index aa56afa..bf24cda 100644 --- a/src/target/trx_toolkit/_fake_trx.pyx +++ b/src/target/trx_toolkit/_fake_trx.pyx @@ -432,7 +432,7 @@ self.burst_fwd = BurstForwarder(trx_list.trx_list) # This method will be called by the clock generator - def clck_handler(self, fn): + def clck_handler(self, int64_t fn): cdef Transceiver trx # We assume that this list is immutable at run-time for trx in self.trx_list: diff --git a/src/target/trx_toolkit/data_msg.pxd b/src/target/trx_toolkit/data_msg.pxd index 29d1702..823ec8d 100644 --- a/src/target/trx_toolkit/data_msg.pxd +++ b/src/target/trx_toolkit/data_msg.pxd @@ -3,7 +3,7 @@ from cython cimport final from cpython cimport array -from libc.stdint cimport int8_t, uint8_t +from libc.stdint cimport int8_t, int64_t, uint8_t cdef class Modulation: cdef int coding # Coding in TRXD header @@ -21,8 +21,8 @@ cdef class Msg: cdef readonly int ver - cdef readonly object fn # int | None - cdef readonly object tn # int | None + cdef readonly int64_t fn # -1 if not valid + cdef readonly int8_t tn # -1 if not valid cdef public bint nope_ind diff --git a/src/target/trx_toolkit/data_msg.pyx b/src/target/trx_toolkit/data_msg.pyx index 98dbece..64139a4 100644 --- a/src/target/trx_toolkit/data_msg.pyx +++ b/src/target/trx_toolkit/data_msg.pyx @@ -20,7 +20,7 @@ # GNU General Public License for more details. from cython cimport final -from libc.stdint cimport int16_t, uint16_t, uint32_t +from libc.stdint cimport int16_t, int64_t, uint16_t, uint32_t from cpython cimport array cdef array.array _Bempty = array.array('B', []) @@ -86,7 +86,7 @@ CHDR_VERSION_MAX = 0b1111 KNOWN_VERSIONS = tuple(range(KNOWN_VERSION_MAX+1)) - def __cinit__(self, fn = -1, tn = -1, int ver = 0): + def __cinit__(self, int64_t fn = -1, int64_t tn = -1, int ver = 0): self.ver = ver self.fn = fn self.tn = tn @@ -231,13 +231,13 @@ if not (0 <= self.ver <= KNOWN_VERSION_MAX): raise ValueError("Unknown TRXD header version %d" % self.ver) - if self.fn is None: + if self.fn == -1: raise ValueError("TDMA frame-number is not set") if self.fn < 0 or self.fn > GSM_HYPERFRAME: raise ValueError("TDMA frame-number %d is out of range" % self.fn) - if self.tn is None: + if self.tn == -1: raise ValueError("TDMA time-slot is not set") if self.tn < 0 or self.tn > 7: diff --git a/src/target/trx_toolkit/fake_pm.py b/src/target/trx_toolkit/fake_pm.py index 205596a..191b203 100644 --- a/src/target/trx_toolkit/fake_pm.py +++ b/src/target/trx_toolkit/fake_pm.py @@ -59,7 +59,7 @@ def rssi_trx(self): return randint(self.trx_min, self.trx_max) - def measure(self, freq, fn = None): + def measure(self, freq, fn = -1): # Iterate over all known transceivers for trx in self.trx_list: if not trx.running: @@ -67,7 +67,7 @@ # FIXME: we need to know current TDMA frame number here, # because some transceivers may use frequency hopping - if trx.fh is not None and fn is None: + if trx.fh is not None and fn == -1: continue # Match by given frequency diff --git a/src/target/trx_toolkit/test_data_msg.py b/src/target/trx_toolkit/test_data_msg.py index e6cb906..db9a998 100644 --- a/src/target/trx_toolkit/test_data_msg.py +++ b/src/target/trx_toolkit/test_data_msg.py @@ -67,12 +67,12 @@ msg = RxMsg() msg.validate() with self.assertRaises(ValueError): - msg = RxMsg(fn = None, tn = 0) + msg = RxMsg(fn = -1, tn = 0) msg.validate() # Out-of-range value(s) with self.assertRaises(ValueError): - msg = RxMsg(fn = -1, tn = 0) + msg = RxMsg(fn = -2, tn = 0) msg.validate() with self.assertRaises(ValueError): msg = RxMsg(fn = 0, tn = 10) diff --git a/src/target/trx_toolkit/transceiver.pxd b/src/target/trx_toolkit/transceiver.pxd index 2fb2fb7..562cb34 100644 --- a/src/target/trx_toolkit/transceiver.pxd +++ b/src/target/trx_toolkit/transceiver.pxd @@ -5,6 +5,8 @@ from burst_fwd cimport BurstForwarder from _clck_gen cimport CLCKGen +from libc.stdint cimport int64_t + cdef class Transceiver: cdef public: @@ -45,10 +47,10 @@ list[TxMsg] _tx_queue - cdef get_rx_freq(self, fn) - cpdef get_tx_freq(self, fn) + cdef get_rx_freq(self, int64_t fn) + cpdef get_tx_freq(self, int64_t fn) cdef TxMsg recv_data_msg(self) cdef send_data_msg(self, RxMsg msg) cdef handle_data_msg(self, Transceiver src_trx, TxMsg src_msg, RxMsg msg) - cdef clck_tick(self, BurstForwarder fwd, fn) + cdef clck_tick(self, BurstForwarder fwd, int64_t fn) diff --git a/src/target/trx_toolkit/transceiver.pyx b/src/target/trx_toolkit/transceiver.pyx index 2e8a064..b28dee8 100644 --- a/src/target/trx_toolkit/transceiver.pyx +++ b/src/target/trx_toolkit/transceiver.pyx @@ -213,7 +213,7 @@ return True - cdef get_rx_freq(self, fn): + cdef get_rx_freq(self, int64_t fn): if self.fh is None: return self._rx_freq @@ -221,7 +221,7 @@ (rx_freq, _) = self.fh.resolve(fn) return rx_freq - cpdef get_tx_freq(self, fn): + cpdef get_tx_freq(self, int64_t fn): if self.fh is None: return self._tx_freq @@ -297,7 +297,7 @@ cdef handle_data_msg(self, Transceiver src_trx, TxMsg src_msg, RxMsg msg): self.send_data_msg(msg) - cdef clck_tick(self, BurstForwarder fwd, fn): + cdef clck_tick(self, BurstForwarder fwd, int64_t fn): if not self.running: return -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40081?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: Iea31825f974734e727fe05da9e78ee67afb59d5b Gerrit-Change-Number: 40081 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months

1
0
0 0

[XS] Change in osmocom-bb[master]: trx_toolkit/data_msg: Switch TxMsg .pwr from object to int

by kirr

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40082?usp=email ) Change subject: trx_toolkit/data_msg: Switch TxMsg .pwr from object to int ...................................................................... trx_toolkit/data_msg: Switch TxMsg .pwr from object to int This is integer field, assigning C-level type avoid py-related overhead on access and arithmetics. Change-Id: Ibf07cd99d758e516317a16b4df3cd0dc90fc5c07 --- M src/target/trx_toolkit/data_msg.pxd M src/target/trx_toolkit/data_msg.pyx 2 files changed, 5 insertions(+), 4 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/82/40082/1 diff --git a/src/target/trx_toolkit/data_msg.pxd b/src/target/trx_toolkit/data_msg.pxd index 823ec8d..e5c605e 100644 --- a/src/target/trx_toolkit/data_msg.pxd +++ b/src/target/trx_toolkit/data_msg.pxd @@ -60,7 +60,8 @@ @final cdef class TxMsg(Msg): cdef public bytearray burst # | None - cdef readonly object pwr # int | None + cdef readonly int pwr # -1 if unset + cpdef validate(self) diff --git a/src/target/trx_toolkit/data_msg.pyx b/src/target/trx_toolkit/data_msg.pyx index 64139a4..1d40726 100644 --- a/src/target/trx_toolkit/data_msg.pyx +++ b/src/target/trx_toolkit/data_msg.pyx @@ -323,7 +323,7 @@ def __cinit__(self): self.burst = None - self.pwr = None + self.pwr = -1 cdef int HDR_LEN(self) except -1: ''' Calculate header length depending on its version. ''' @@ -345,7 +345,7 @@ # Validate common fields Msg._validate(self) - if self.pwr is None: + if self.pwr == -1: raise ValueError("Tx Attenuation level is not set") if self.pwr < PWR_MIN or self.pwr > PWR_MAX: @@ -382,7 +382,7 @@ # Describe the common part result = Msg.desc_hdr(self) - if self.pwr is not None: + if self.pwr != -1: result += ("pwr=%u " % self.pwr) # Strip useless whitespace and return -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40082?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: Ibf07cd99d758e516317a16b4df3cd0dc90fc5c07 Gerrit-Change-Number: 40082 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

4 months

1
0
0 0

2025

2024

2023

2022

gerrit-log April 2025