kirr has uploaded this change for review.

View Change

trx_toolkit: Move FakeTRX-related performance-sensitive modules to Cython

Cython (https://cython.org/ , https://cython.readthedocs.io/en/latest/)
is Python-compatible compiled language, that extends Python and allows
to add type annotations which help to optimize generated code. It is
based on Pyrex (https://www.csse.canterbury.ac.nz/greg.ewing/python/Pyrex/)
which is no longer actively maintained this days.

Using Cython allows one to take Python codebase and improve its speed
incrementally step by step instead of doing full rewrite from scratch.

Here we are only hooking Cython build system into the project and move
*.py to *.pyx files for the following modules:

- burst_fwd.c
- _clck_gen.c
- data_if.c
- data_msg.c
- _fake_trx.c
- udp_link.c
- transceiver.c

Those modules turned up to require being optimized because their
functionality is used by fake_trx loop non-stop at the very least ~2000
times a second if there is only one BTS attached to fake_trx.

We will optimize those modules step by step in the follow-up patches.
For now it is only plain code movement with adding

# cython: language_level=3

comment to the top of each file because else Cython uses 3str language
mode and emits a warning. We use language_level=3 because it has the
same semantic as of regular Python3.

The project now needs to be build before running: either `pip install -e .`
or `python setup.py build_ext -i` when doing development, or simple
`pip install .` for non-develop build.

There is no performance increase just from the move to Cython. We will
add optimizations step by step in the follow-up patches.

Change-Id: I2159a07bece13bda4f6ccd957063d4644d8b5e4f
---
M src/target/trx_toolkit/.gitignore
R src/target/trx_toolkit/_clck_gen.pyx
R src/target/trx_toolkit/_fake_trx.pyx
R src/target/trx_toolkit/burst_fwd.pyx
R src/target/trx_toolkit/data_if.pyx
R src/target/trx_toolkit/data_msg.pyx
A src/target/trx_toolkit/pyproject.toml
A src/target/trx_toolkit/setup.py
R src/target/trx_toolkit/transceiver.pyx
R src/target/trx_toolkit/udp_link.pyx
10 files changed, 35 insertions(+), 0 deletions(-)

git pull ssh://gerrit.osmocom.org:29418/osmocom-bb refs/changes/52/40052/1
diff --git a/src/target/trx_toolkit/.gitignore b/src/target/trx_toolkit/.gitignore
index dc634e2..3ea0ef1 100644
--- a/src/target/trx_toolkit/.gitignore
+++ b/src/target/trx_toolkit/.gitignore
@@ -2,5 +2,14 @@
__pycache__/
*.py[cod]
*$py.class
+/build/
+/*.so
/perf.data*
/flamegraph.html
+/burst_fwd.c
+/_clck_gen.c
+/data_if.c
+/data_msg.c
+/_fake_trx.c
+/udp_link.c
+/transceiver.c
diff --git a/src/target/trx_toolkit/_clck_gen.py b/src/target/trx_toolkit/_clck_gen.pyx
similarity index 98%
rename from src/target/trx_toolkit/_clck_gen.py
rename to src/target/trx_toolkit/_clck_gen.pyx
index 9d9fd15..914b821 100644
--- a/src/target/trx_toolkit/_clck_gen.py
+++ b/src/target/trx_toolkit/_clck_gen.pyx
@@ -1,3 +1,5 @@
+# cython: language_level=3
+
# TRX Toolkit
# Simple TDMA frame clock generator
#
diff --git a/src/target/trx_toolkit/_fake_trx.py b/src/target/trx_toolkit/_fake_trx.pyx
similarity index 99%
rename from src/target/trx_toolkit/_fake_trx.py
rename to src/target/trx_toolkit/_fake_trx.pyx
index 87a352b..92573f8 100644
--- a/src/target/trx_toolkit/_fake_trx.py
+++ b/src/target/trx_toolkit/_fake_trx.pyx
@@ -1,3 +1,5 @@
+# cython: language_level=3
+
# TRX Toolkit
# Virtual Um-interface (fake transceiver)
#
diff --git a/src/target/trx_toolkit/burst_fwd.py b/src/target/trx_toolkit/burst_fwd.pyx
similarity index 98%
rename from src/target/trx_toolkit/burst_fwd.py
rename to src/target/trx_toolkit/burst_fwd.pyx
index 2824e0a..2737232 100644
--- a/src/target/trx_toolkit/burst_fwd.py
+++ b/src/target/trx_toolkit/burst_fwd.pyx
@@ -1,4 +1,5 @@
# -*- coding: utf-8 -*-
+# cython: language_level=3

# TRX Toolkit
# Burst forwarding between transceivers
diff --git a/src/target/trx_toolkit/data_if.py b/src/target/trx_toolkit/data_if.pyx
similarity index 98%
rename from src/target/trx_toolkit/data_if.py
rename to src/target/trx_toolkit/data_if.pyx
index f59ca17..396987e 100644
--- a/src/target/trx_toolkit/data_if.py
+++ b/src/target/trx_toolkit/data_if.pyx
@@ -1,4 +1,5 @@
# -*- coding: utf-8 -*-
+# cython: language_level=3

# TRX Toolkit
# DATA interface implementation
diff --git a/src/target/trx_toolkit/data_msg.py b/src/target/trx_toolkit/data_msg.pyx
similarity index 99%
rename from src/target/trx_toolkit/data_msg.py
rename to src/target/trx_toolkit/data_msg.pyx
index b1673f5..af0aac6 100644
--- a/src/target/trx_toolkit/data_msg.py
+++ b/src/target/trx_toolkit/data_msg.pyx
@@ -1,4 +1,5 @@
# -*- coding: utf-8 -*-
+# cython: language_level=3

# TRX Toolkit
# DATA interface message definitions and helpers
diff --git a/src/target/trx_toolkit/pyproject.toml b/src/target/trx_toolkit/pyproject.toml
new file mode 100644
index 0000000..e9f294d
--- /dev/null
+++ b/src/target/trx_toolkit/pyproject.toml
@@ -0,0 +1,2 @@
+[build-system]
+requires = ["setuptools", "wheel", "cython"]
diff --git a/src/target/trx_toolkit/setup.py b/src/target/trx_toolkit/setup.py
new file mode 100644
index 0000000..78de7ff
--- /dev/null
+++ b/src/target/trx_toolkit/setup.py
@@ -0,0 +1,15 @@
+from setuptools import setup
+from Cython.Build import cythonize
+
+setup(
+ name='trx_toolkit',
+ ext_modules = cythonize([
+ "burst_fwd.pyx",
+ "_clck_gen.pyx",
+ "data_if.pyx",
+ "data_msg.pyx",
+ "_fake_trx.pyx",
+ "udp_link.pyx",
+ "transceiver.pyx"
+ ])
+)
diff --git a/src/target/trx_toolkit/transceiver.py b/src/target/trx_toolkit/transceiver.pyx
similarity index 99%
rename from src/target/trx_toolkit/transceiver.py
rename to src/target/trx_toolkit/transceiver.pyx
index 2ebb29a..aaf3a90 100644
--- a/src/target/trx_toolkit/transceiver.py
+++ b/src/target/trx_toolkit/transceiver.pyx
@@ -1,4 +1,5 @@
# -*- coding: utf-8 -*-
+# cython: language_level=3

# TRX Toolkit
# Transceiver implementation
diff --git a/src/target/trx_toolkit/udp_link.py b/src/target/trx_toolkit/udp_link.pyx
similarity index 97%
rename from src/target/trx_toolkit/udp_link.py
rename to src/target/trx_toolkit/udp_link.pyx
index 53a0bfb..779c0fc 100644
--- a/src/target/trx_toolkit/udp_link.py
+++ b/src/target/trx_toolkit/udp_link.pyx
@@ -1,4 +1,5 @@
# -*- coding: utf-8 -*-
+# cython: language_level=3

# TRX Toolkit
# UDP link implementation

To view, visit change 40052. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-MessageType: newchange
Gerrit-Project: osmocom-bb
Gerrit-Branch: master
Gerrit-Change-Id: I2159a07bece13bda4f6ccd957063d4644d8b5e4f
Gerrit-Change-Number: 40052
Gerrit-PatchSet: 1
Gerrit-Owner: kirr <kirr@nexedi.com>
Gerrit-CC: fixeria <vyanitskiy@sysmocom.de>
Gerrit-CC: osmith <osmith@sysmocom.de>
Gerrit-CC: pespin <pespin@sysmocom.de>