[S] Change in osmocom-bb[master]: trx_toolkit: Move FakeTRX-related performance-sensitive modules to Cy...

kirr has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmocom-bb/+/40052?usp=email ) Change subject: trx_toolkit: Move FakeTRX-related performance-sensitive modules to Cython ...................................................................... trx_toolkit: Move FakeTRX-related performance-sensitive modules to Cython Cython (https://cython.org/ , https://cython.readthedocs.io/en/latest/) is Python-compatible compiled language, that extends Python and allows to add type annotations which help to optimize generated code. It is based on Pyrex (https://www.csse.canterbury.ac.nz/greg.ewing/python/Pyrex/) which is no longer actively maintained this days. Using Cython allows one to take Python codebase and improve its speed incrementally step by step instead of doing full rewrite from scratch. Here we are only hooking Cython build system into the project and move *.py to *.pyx files for the following modules: - burst_fwd.c - _clck_gen.c - data_if.c - data_msg.c - _fake_trx.c - udp_link.c - transceiver.c Those modules turned up to require being optimized because their functionality is used by fake_trx loop non-stop at the very least ~2000 times a second if there is only one BTS attached to fake_trx. We will optimize those modules step by step in the follow-up patches. For now it is only plain code movement with adding # cython: language_level=3 comment to the top of each file because else Cython uses 3str language mode and emits a warning. We use language_level=3 because it has the same semantic as of regular Python3. The project now needs to be build before running: either `pip install -e .` or `python setup.py build_ext -i` when doing development, or simple `pip install .` for non-develop build. There is no performance increase just from the move to Cython. We will add optimizations step by step in the follow-up patches. Change-Id: I2159a07bece13bda4f6ccd957063d4644d8b5e4f --- M src/target/trx_toolkit/.gitignore R src/target/trx_toolkit/_clck_gen.pyx R src/target/trx_toolkit/_fake_trx.pyx R src/target/trx_toolkit/burst_fwd.pyx R src/target/trx_toolkit/data_if.pyx R src/target/trx_toolkit/data_msg.pyx A src/target/trx_toolkit/pyproject.toml A src/target/trx_toolkit/setup.py R src/target/trx_toolkit/transceiver.pyx R src/target/trx_toolkit/udp_link.pyx 10 files changed, 35 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/52/40052/1 diff --git a/src/target/trx_toolkit/.gitignore b/src/target/trx_toolkit/.gitignore index dc634e2..3ea0ef1 100644 --- a/src/target/trx_toolkit/.gitignore +++ b/src/target/trx_toolkit/.gitignore @@ -2,5 +2,14 @@ __pycache__/ *.py[cod] *$py.class +/build/ +/*.so /perf.data* /flamegraph.html +/burst_fwd.c +/_clck_gen.c +/data_if.c +/data_msg.c +/_fake_trx.c +/udp_link.c +/transceiver.c diff --git a/src/target/trx_toolkit/_clck_gen.py b/src/target/trx_toolkit/_clck_gen.pyx similarity index 98% rename from src/target/trx_toolkit/_clck_gen.py rename to src/target/trx_toolkit/_clck_gen.pyx index 9d9fd15..914b821 100644 --- a/src/target/trx_toolkit/_clck_gen.py +++ b/src/target/trx_toolkit/_clck_gen.pyx @@ -1,3 +1,5 @@ +# cython: language_level=3 + # TRX Toolkit # Simple TDMA frame clock generator # diff --git a/src/target/trx_toolkit/_fake_trx.py b/src/target/trx_toolkit/_fake_trx.pyx similarity index 99% rename from src/target/trx_toolkit/_fake_trx.py rename to src/target/trx_toolkit/_fake_trx.pyx index 87a352b..92573f8 100644 --- a/src/target/trx_toolkit/_fake_trx.py +++ b/src/target/trx_toolkit/_fake_trx.pyx @@ -1,3 +1,5 @@ +# cython: language_level=3 + # TRX Toolkit # Virtual Um-interface (fake transceiver) # diff --git a/src/target/trx_toolkit/burst_fwd.py b/src/target/trx_toolkit/burst_fwd.pyx similarity index 98% rename from src/target/trx_toolkit/burst_fwd.py rename to src/target/trx_toolkit/burst_fwd.pyx index 2824e0a..2737232 100644 --- a/src/target/trx_toolkit/burst_fwd.py +++ b/src/target/trx_toolkit/burst_fwd.pyx @@ -1,4 +1,5 @@ # -*- coding: utf-8 -*- +# cython: language_level=3 # TRX Toolkit # Burst forwarding between transceivers diff --git a/src/target/trx_toolkit/data_if.py b/src/target/trx_toolkit/data_if.pyx similarity index 98% rename from src/target/trx_toolkit/data_if.py rename to src/target/trx_toolkit/data_if.pyx index f59ca17..396987e 100644 --- a/src/target/trx_toolkit/data_if.py +++ b/src/target/trx_toolkit/data_if.pyx @@ -1,4 +1,5 @@ # -*- coding: utf-8 -*- +# cython: language_level=3 # TRX Toolkit # DATA interface implementation diff --git a/src/target/trx_toolkit/data_msg.py b/src/target/trx_toolkit/data_msg.pyx similarity index 99% rename from src/target/trx_toolkit/data_msg.py rename to src/target/trx_toolkit/data_msg.pyx index b1673f5..af0aac6 100644 --- a/src/target/trx_toolkit/data_msg.py +++ b/src/target/trx_toolkit/data_msg.pyx @@ -1,4 +1,5 @@ # -*- coding: utf-8 -*- +# cython: language_level=3 # TRX Toolkit # DATA interface message definitions and helpers diff --git a/src/target/trx_toolkit/pyproject.toml b/src/target/trx_toolkit/pyproject.toml new file mode 100644 index 0000000..e9f294d --- /dev/null +++ b/src/target/trx_toolkit/pyproject.toml @@ -0,0 +1,2 @@ +[build-system] +requires = ["setuptools", "wheel", "cython"] diff --git a/src/target/trx_toolkit/setup.py b/src/target/trx_toolkit/setup.py new file mode 100644 index 0000000..78de7ff --- /dev/null +++ b/src/target/trx_toolkit/setup.py @@ -0,0 +1,15 @@ +from setuptools import setup +from Cython.Build import cythonize + +setup( + name='trx_toolkit', + ext_modules = cythonize([ + "burst_fwd.pyx", + "_clck_gen.pyx", + "data_if.pyx", + "data_msg.pyx", + "_fake_trx.pyx", + "udp_link.pyx", + "transceiver.pyx" + ]) +) diff --git a/src/target/trx_toolkit/transceiver.py b/src/target/trx_toolkit/transceiver.pyx similarity index 99% rename from src/target/trx_toolkit/transceiver.py rename to src/target/trx_toolkit/transceiver.pyx index 2ebb29a..aaf3a90 100644 --- a/src/target/trx_toolkit/transceiver.py +++ b/src/target/trx_toolkit/transceiver.pyx @@ -1,4 +1,5 @@ # -*- coding: utf-8 -*- +# cython: language_level=3 # TRX Toolkit # Transceiver implementation diff --git a/src/target/trx_toolkit/udp_link.py b/src/target/trx_toolkit/udp_link.pyx similarity index 97% rename from src/target/trx_toolkit/udp_link.py rename to src/target/trx_toolkit/udp_link.pyx index 53a0bfb..779c0fc 100644 --- a/src/target/trx_toolkit/udp_link.py +++ b/src/target/trx_toolkit/udp_link.pyx @@ -1,4 +1,5 @@ # -*- coding: utf-8 -*- +# cython: language_level=3 # TRX Toolkit # UDP link implementation -- To view, visit https://gerrit.osmocom.org/c/osmocom-bb/+/40052?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmocom-bb Gerrit-Branch: master Gerrit-Change-Id: I2159a07bece13bda4f6ccd957063d4644d8b5e4f Gerrit-Change-Number: 40052 Gerrit-PatchSet: 1 Gerrit-Owner: kirr <kirr(a)nexedi.com> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: osmith <osmith(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>