From fa318991@gmail.com Tue Feb  6 11:25:42 2024
From: fa318991@gmail.com
To: openbsc@lists.osmocom.org
Subject: sadsadsa
Date: Tue, 06 Feb 2024 11:25:41 +0000
Message-ID: <170721874162.433.3399852799000483468@lists>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============6301385100785998390=="

--===============6301385100785998390==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

<a href="https://www.test.com">test</a>	
[url=https://www.test.com]test[/url]	
[[http://www.test.com|test]]	
url[http://www.domain.com][put text here]url
[Test](https://rentry.co)
[http://www.example.com link title]

--===============6301385100785998390==--


From fa318991@gmail.com Tue Feb  6 11:32:20 2024
From: fa318991@gmail.com
To: openbsc@lists.osmocom.org
Subject: CRM Assignment Help at Your Fingertips
Date: Tue, 06 Feb 2024 11:32:18 +0000
Message-ID: <170721913816.433.15217082770262825949@lists>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============7631493886294419705=="

--===============7631493886294419705==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Empower your journey in customer relationship management (CRM) with our compr=
ehensive support services. Our team specializes in providing expert guidance =
and assistance tailored to your CRM assignment needs. From analyzing customer=
 data to implementing effective strategies, our CRM Assignment Help ensures y=
ou're equipped with the knowledge and skills to excel. Trust us to be your pa=
rtner in success as you navigate the complexities. Let our expertise be at yo=
ur fingertips, guiding you toward academic achievement and more visit our web=
site Now.https://reportwritinghelp.com/assignment/consumer-behavior-assignmen=
t-help

--===============7631493886294419705==--


From jonathanpc@protonmail.com Thu Feb  8 20:15:56 2024
From: JPichel <jonathanpc@protonmail.com>
To: openbsc@lists.osmocom.org
Subject: ERROR Trying to dispatch event 17 to non-existent FSM instance!
Date: Thu, 08 Feb 2024 20:15:53 +0000
Message-ID: <170742335376.433.15742988272773725333@lists>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============7858182934086618223=="

--===============7858182934086618223==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hi,

I hope this is the place to ask this, if not I will delete this post. I am tr=
ying to run my own 2G network for researching purposes. I am running the foll=
owing elements:
- osmo-bts-trx (connecting to osmo-trx-uhd driving a USRP B205mini-i)
- osmo-bsc
- osmo-msc
- osmo-hlr
- osmo-mgw

I built all the programs from source on Ubuntu 22.04. Everything is running l=
ocally on one machine.

Everything seems to be running OK. I was able to register two different subsc=
ribers by IMSI (without providing K and OPc) and assign MSISDN tothem. They c=
an use the default USSD codes for seeing their IMSI and their MSISDN and I ca=
n make voice calls between both subscribers. I am mostly using the default co=
nfig files provided with the installation, the only changes I made to them wa=
s to point IPs to 127.0.0.1 to run everything on the same machine.

But when starting osmo-bsc and osmo-msc I get the same error on both:

jonathan(a)em680:~/tfg/osmo/config$ osmo-msc -c osmo-msc.cfg
DLGLOBAL NOTICE Available via telnet 127.0.0.1 4254 (telnet_interface.c:88)
DLCTRL NOTICE CTRL at 127.0.0.1 4255 (control_if.c:1014)
DLGSUP NOTICE GSUP connecting to 127.0.0.1:4222 (gsup_client.c:74)
DDB NOTICE Init database connection to 'sms.db' using SQLite3 lib version 3.3=
7.2 (db.c:521)
DLMGCP NOTICE MGW(mgw) MGCP client: using endpoint domain '@mgw' (mgcp_client=
.c:933)
DMSC NOTICE MGW pool with 1 pool members configured, (ignoring MGW configurat=
ion in VTY node 'msc'). (msc_main.c:596)
DLSCCP NOTICE OsmoMSC-A: Creating SS7 instance (sccp_user.c:536)
DLSCCP NOTICE OsmoMSC-A: Using SS7 instance 0, pc:0.23.1 (sccp_user.c:563)
DLSCCP NOTICE OsmoMSC-A: Creating AS instance (sccp_user.c:570)
DLSCCP NOTICE OsmoMSC-A: Using AS instance as-clnt-OsmoMSC-A (sccp_user.c:581)
DLSCCP NOTICE OsmoMSC-A: Creating default route (sccp_user.c:587)
DLSCCP NOTICE OsmoMSC-A: No unassociated ASP for m3ua, creating new ASP asp-c=
lnt-OsmoMSC-A (sccp_user.c:626)
DLGLOBAL ERROR Trying to dispatch event 17 to non-existent FSM instance! (osm=
o_ss7_as.c:118)
DLGLOBAL ERROR backtrace() returned 11 addresses (backtrace.c:42)
DLGLOBAL ERROR  /usr/local/lib/libosmocore.so.21(osmo_log_backtrace+0x24) [0x=
7fc1ddd9fdcc] (backtrace.c:53)
DLGLOBAL ERROR  /usr/local/lib/libosmocore.so.21(_osmo_fsm_inst_dispatch+0x14=
4) [0x7fc1dddaade7] (backtrace.c:53)
DLGLOBAL ERROR  /usr/local/lib/libosmo-sigtran.so.9(osmo_ss7_as_add_asp+0x1f9=
) [0x7fc1ddcfb80c] (backtrace.c:53)
DLGLOBAL ERROR  /usr/local/lib/libosmo-sigtran.so.9(osmo_sccp_simple_client_o=
n_ss7_id+0xa48) [0x7fc1ddd1dd5e] (backtrace.c:53)
DLGLOBAL ERROR  osmo-msc(+0x26fd8) [0x55fa5388bfd8] (backtrace.c:53)
DLGLOBAL ERROR  osmo-msc(+0x27043) [0x55fa5388c043] (backtrace.c:53)
DLGLOBAL ERROR  osmo-msc(+0x27bed) [0x55fa5388cbed] (backtrace.c:53)
DLGLOBAL ERROR  /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc1dd95fd90] (b=
acktrace.c:53)
DLGLOBAL ERROR  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f=
c1dd95fe40] (backtrace.c:53)
DLGLOBAL ERROR  osmo-msc(+0x26565) [0x55fa5388b565] (backtrace.c:53)
DLSCCP NOTICE OsmoMSC-A: Using ASP instance asp-clnt-OsmoMSC-A (sccp_user.c:6=
95)
DLSS7 NOTICE 0: Creating SCCP instance (osmo_ss7.c:408)
DSGS NOTICE SGs socket bound to r=3DNULL<->l=3D0.0.0.0:29118 (sgs_server.c:18=
6)
DMSC NOTICE A-interface: SCCP user OsmoMSC-A:RI=3DSSN_PC,PC=3D(no PC),SSN=3DB=
SSAP, cs7-instance 0 ((null)) (msc_main.c:806)
DLINP NOTICE 127.0.0.1:4222 connection done (ipa.c:141)
DLINP NOTICE received ID_GET for unit ID 0/0/0 (ipaccess.c:919)
DLM3UA NOTICE 0: asp-asp-clnt-OsmoMSC-A: Received NOTIFY Type State Change:AS=
 Inactive () (m3ua.c:625)
DLSS7 NOTICE xua_default_lm(asp-clnt-OsmoMSC-A)[0x55fa542bb080]{ACTIVE}: Igno=
ring primitive M-ASP_ACTIVE.confirm (xua_default_lm_fsm.c:400)
DLM3UA NOTICE 0: asp-asp-clnt-OsmoMSC-A: Received NOTIFY Type State Change:AS=
 Active () (m3ua.c:625)
DLM3UA NOTICE 0: asp-asp-clnt-OsmoMSC-A: Rx DAVA() for 0.23.3/0,  (xua_snm.c:=
403)
...

jonathan(a)em680:~/tfg/osmo/config$ osmo-bsc -c osmo-bsc.cfg
DLGLOBAL NOTICE Available via telnet 127.0.0.1 4242 (telnet_interface.c:88)
DLINP NOTICE enabling ipaccess BSC mode on 0.0.0.0 with OML 3002 and RSL 3003=
 TCP ports (ipaccess.c:1055)
DLCTRL NOTICE CTRL at 127.0.0.1 4249 (control_if.c:1014)
DLMGCP NOTICE MGW(mgw) MGCP client: using endpoint domain '@mgw' (mgcp_client=
.c:933)
DNM NOTICE MGW pool with 1 pool members configured, (ignoring MGW configurati=
on in VTY node 'msc'). (osmo_bsc_main.c:889)
DMSC NOTICE To auto-configure msc 0, creating cs7 instance 0 implicitly (osmo=
_bsc_sigtran.c:600)
DMSC NOTICE Initializing SCCP connection for A/m3ua on cs7 instance 0 (osmo_b=
sc_sigtran.c:647)
DLSCCP ERROR SS7 instance 0: no primary point-code set, using default point-c=
ode (sccp_user.c:557)
DLSCCP NOTICE A-0-m3ua: Using SS7 instance 0, pc:0.23.3 (sccp_user.c:563)
DLSCCP NOTICE A-0-m3ua: Creating AS instance (sccp_user.c:570)
DLSCCP NOTICE A-0-m3ua: Using AS instance as-clnt-A-0-m3ua (sccp_user.c:581)
DLSCCP NOTICE A-0-m3ua: Creating default route (sccp_user.c:587)
DLSCCP NOTICE A-0-m3ua: No unassociated ASP for m3ua, creating new ASP asp-cl=
nt-A-0-m3ua (sccp_user.c:626)
DLGLOBAL ERROR Trying to dispatch event 17 to non-existent FSM instance! (osm=
o_ss7_as.c:118)
DLGLOBAL ERROR backtrace() returned 10 addresses (backtrace.c:42)
DLGLOBAL ERROR  /usr/local/lib/libosmocore.so.21(osmo_log_backtrace+0x24) [0x=
7f990ff0ddcc] (backtrace.c:53)
DLGLOBAL ERROR  /usr/local/lib/libosmocore.so.21(_osmo_fsm_inst_dispatch+0x14=
4) [0x7f990ff18de7] (backtrace.c:53)
DLGLOBAL ERROR  /usr/local/lib/libosmo-sigtran.so.9(osmo_ss7_as_add_asp+0x1f9=
) [0x7f990fe9380c] (backtrace.c:53)
DLGLOBAL ERROR  /usr/local/lib/libosmo-sigtran.so.9(osmo_sccp_simple_client_o=
n_ss7_id+0xa48) [0x7f990feb5d5e] (backtrace.c:53)
DLGLOBAL ERROR  osmo-bsc(+0x144035) [0x556b310d7035] (backtrace.c:53)
DLGLOBAL ERROR  osmo-bsc(+0x2aef0) [0x556b30fbdef0] (backtrace.c:53)
DLGLOBAL ERROR  /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f990fc55d90] (b=
acktrace.c:53)
DLGLOBAL ERROR  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f=
990fc55e40] (backtrace.c:53)
DLGLOBAL ERROR  osmo-bsc(+0x28265) [0x556b30fbb265] (backtrace.c:53)
DLSCCP NOTICE A-0-m3ua: Using ASP instance asp-clnt-A-0-m3ua (sccp_user.c:695)
DLSS7 NOTICE 0: Creating SCCP instance (osmo_ss7.c:408)
DMSC NOTICE A-0-m3ua msc-0: local (BSC) SCCP address: RI=3DSSN_PC,PC=3D0.23.3=
,SSN=3DBSSAP (osmo_bsc_sigtran.c:703)
DMSC NOTICE A-0-m3ua msc-0: remote (MSC) SCCP address: RI=3DSSN_PC,PC=3D0.23.=
1,SSN=3DBSSAP (osmo_bsc_sigtran.c:705)
DMSC NOTICE A-0-m3ua msc-0: binding SCCP user (osmo_bsc_sigtran.c:710)
DLSS7 ERROR XUA_AS(as-clnt-A-0-m3ua)[0x556b31b42cc0]{AS_INACTIVE}: Event AS-T=
RANSFER.req not permitted (m3ua.c:510)
DLM3UA NOTICE 0: asp-asp-clnt-A-0-m3ua: Received NOTIFY Type State Change:AS =
Inactive () (m3ua.c:625)
DLSS7 NOTICE xua_default_lm(asp-clnt-A-0-m3ua)[0x556b31b445c0]{ACTIVE}: Ignor=
ing primitive M-ASP_ACTIVE.confirm (xua_default_lm_fsm.c:400)
DLM3UA NOTICE 0: asp-asp-clnt-A-0-m3ua: Received NOTIFY Type State Change:AS =
Active () (m3ua.c:625)
DMSC NOTICE RESET ACK from MSC: RI=3DSSN_PC,PC=3D0.23.1,SSN=3DBSSAP (osmo_bsc=
_bssap.c:84)
DRESET NOTICE bssmap_reset(msc-0)[0x556b31b45790]{CONNECTED}: link up (bssmap=
_reset.c:83)
DMSC NOTICE (msc0) BSSMAP association is up (a_reset.c:45)
DLINP NOTICE 0.0.0.0:3002 accept()ed new link from 127.0.0.1:38477 (ipa.c:318)
DLINP NOTICE Keepalive is set: 0 (ipaccess.c:612)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
GPRS' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
EGPRS' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
HOPPING' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
MULTI_TSC' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
OML_ALERTS' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
CBCH' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
SPEECH_F_V1' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
SPEECH_H_V1' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
SPEECH_F_EFR' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
SPEECH_F_AMR' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
SPEECH_H_AMR' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
ETWS_PN' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
PAGING_COORDINATION' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
IPV6_NSVC' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
ACCH_REP' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
CCN' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
VAMOS' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
ABIS_OSMO_PCU' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
BCCH_PWR_RED' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
DYN_TS_SDCCH8' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
ACCH_TEMP_OVP' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
VBS' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Get Attributes Response: feature '=
VGCS' is supported (abis_nm.c:599)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): ARI reported sw[0/2]: osmobts is 1=
.7.0.54-82a2 (abis_nm.c:652)
DNM NOTICE OC=3DBTS(01) INST=3D(00,ff,ff): Reported variant: osmo-bts-trx (ab=
is_nm.c:534)
DNM ERROR OC=3DRADIO-CARRIER(02) INST=3D(00,00,ff): Attribute SW Configuratio=
n is unreported (abis_nm.c:557)
DNM ERROR OC=3DBASEBAND-TRANSCEIVER(04) INST=3D(00,00,ff): Attribute Manufact=
urer Dependent State is unreported (abis_nm.c:557)
DNM NOTICE OC=3DBASEBAND-TRANSCEIVER(04) INST=3D(00,00,ff): ARI reported sw[0=
/1]: TRX_PHY_VERSION is Unknown (abis_nm.c:652)
DLINP NOTICE 0.0.0.0:3003 accept()ed new link from 127.0.0.1:37491 (ipa.c:318)
DLINP NOTICE Keepalive is set: 0 (ipaccess.c:612)
...

Is it something I should worry about?

Thanks

--===============7858182934086618223==--


From nhofmeyr@sysmocom.de Thu Feb  8 22:29:21 2024
From: Neels Hofmeyr <nhofmeyr@sysmocom.de>
To: openbsc@lists.osmocom.org
Subject: Re: ERROR Trying to dispatch event 17 to non-existent FSM instance!
Date: Thu, 08 Feb 2024 23:29:14 +0100
Message-ID: <ZcVVunP48UuFGnLg@my.box>
In-Reply-To: <170742335376.433.15742988272773725333@lists>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============0761663766235929850=="

--===============0761663766235929850==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

> DLGLOBAL ERROR Trying to dispatch event 17 to non-existent FSM instance! (o=
smo_ss7_as.c:118)

Thanks for reporting this!

You write that your network is working as expected, and that you see this err=
or
only at startup: that means you don't need to worry about this error.

Nevertheless, it seems to be a newly introduced non-fatal bug, from this fair=
ly
recent patch: https://gerrit.osmocom.org/c/libosmo-sccp/+/35348

We should fix that: https://osmocom.org/issues/6356

So thanks again for reporting!
~N

--===============0761663766235929850==--


From nhofmeyr@sysmocom.de Sun Feb 25 21:40:06 2024
From: Neels Hofmeyr <nhofmeyr@sysmocom.de>
To: openbsc@lists.osmocom.org
Subject: SDP API choices
Date: Sun, 25 Feb 2024 22:39:59 +0100
Message-ID: <Zduzr7QSGk1aZITe@my.box>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2906896554981168645=="

--===============2906896554981168645==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hi all,

I'm currently working on 2G to 3G TrFO voice negotiation. In practice, that
means, to be able to negotiate a set of AMR rates via SDP. And that, in turn,
means that we handle fmtp properly in SDP strings.

SDP is not only used in SIP to the other call leg, but also in the MGCP to the
media gateway.
- So we have a bunch of SDP and fmtp related code in osmo-mgw.git.
- Separately, we have an SDP implementation in osmo-msc.git, because the SDP
  code in osmo-mgw.git was too tightly coupled with other code to be able to
  reuse it for MNCC->SIP.
For fmtp negotiation, I now want similar things in both mgw and msc.

What I want to do now, is move the SDP implementation from osmo-msc.git to
osmo-mgw.git/libosmo-mgcp-client and make it public API: a common place that
will also enable all other MGCP clients to work with AMR fmtp.

All of this was just the intro =3D) the question I would like to put out ther=
e is
about API design: static or dynamic, and opaque or transparent.
Am I overthinking this? Or would you like to overthink it with me?


=3D=3D static or dynamic =3D=3D

In osmo-msc.git, the SDP structs so far are fully static and self-contained.
The cool thing is that they can be copied around quite easily and there is no
need for passing talloc ctx pointers, no need to worry about ownership,
memleaks and double frees. But it also means that it uses fixed sizes:
- char encoding_name[16],
- char fmtp[256],
- struct sdp_audio_codec codecs[32]
and so on.

Unfortunately, RFC-8866 does not define any limits whatsoever on number of
characters or number of codecs in an SDP message. Quoting:
"although no length limit is given, it is recommended that they be short"
Thanks for nothing! >:(

So for static API design, we need to now pick a good size for every single it=
em
in advance.

When fmtp has max 256 bytes and codec lists have max 32 entries, even an empty
list will use 32 * 256+ bytes: lots of heap allocations.

And the limits are hard: anyone wanting to correctly express more than 32
different codec variants can simply not use this library.

Dynamic seems a much better fit for RFC-8866.
There is practically no limit at all on string length / list length.
Also the caller allocates not one byte more memory than is actually needed.

The argument has been going back and forth in my head for weeks. Static makes
things so much simpler. But what sizes should we dictate. Ok then dynamic. But
these days we have "infinite" memory, the bottleneck is CPU time, so if we can
be static, we are gracious on very cheap real estate (memory) while saving a
lot of the precious stuff (cycles). Ok then static. But with all those AMR
variants that are coming up, I want to at least be able to represent all of
them in one list, so I need at least 64 entries. And let's allow long fmtp
strings, who knows. Then we'll end up with ~20kb of data that every codec list
out there will use up, even if it is empty. Even if it is just a temporary
local variable. Even if it is just dragged along for a rare use case and never
actually needed. Is it really faster to allocate 100 times more memory
statically versus allocating just enough but dynamically?

In the end I decided for dynamic allocation.
When I started adding talloc ctx args, I wanted the API to be simple, so every
single part of the puzzle is now its own dynamic allocation:

 struct osmo_sdp_msg {
        char *username;      // talloc_strdup()
        char *session_name;  // talloc_strdup()
        [...]
        struct osmo_sdp_codec_list *codecs;  // osmo_sdp_codec_list_alloc()
           // llist_head
           //    -> osmo_sdp_codec_alloc()
           //          \___ char *encoding_name  -> talloc_strdup()
           //             _ char *fmtp           -> talloc_strdup()
           //          -next-> osmo_sdp_codec_alloc()
           //                     \___ ...
           //                            -next-> osmo_sdp_codec_alloc()
           //                                         \___ ...
 };

All of this used to be just a single self-contained struct.

You might ask, why is 'codecs', which is just the llist_head, a separate
allocation? Because I want the API to trivially figure out the talloc parent
context to allocate items from:

  osmo_codec_list_add(my_list, my_codec);

and not have to worry about the individual code paths' parent context to
allocate new entries from:

  osmo_codec_list_add(&my_obj->my_list, my_obj->backpointer.backpointer.ctx, =
my_codec);

So the only functions with a talloc ctx argument are
 osmo_codec_alloc(ctx) and
 osmo_codec_list_alloc(ctx)

This style has a tendency to spread:
Now I want to also make osmo-msc's struct codec_filter a dynamic allocation,
because the codec_filter_foo() currently have no ctx argument; they still use
the static SDP structs. When the codec_filter pointer itself is the neat
logical talloc parent of the codec lists it uses, then I don't need to add a
separate ctx arg everywhere now.

Am I taking this too far now?
Am I overthinking it and either way is fine?


=3D=3D opaque vs transparent =3D=3D

I can either have the full struct definition in the .h file =3D transparent, =
or I
can hide the struct in the .c file and provide lots of getter and setter
functions =3D opaque.

Both ways have a major advantage over the other:

- opaque means that we can freely extend the API without any ABI breakage,
  ever.

- transparent means that callers can easily define const arrays of data
  structures. In my case, I want to keep const lists of const codecs for e.g.
  test data in unit tests, for listing known codecs in codec_mapping.c, for
  defining default codec configuration.

transparent allows:

        const struct osmo_sdp_codec ran_defaults =3D {
                        { .encoding_name =3D "AMR", .rate =3D 8000, .fmtp =3D=
 "octet-align=3D1;mode-set=3D0,2,4,7", .payload_type =3D 112, },
                        { .encoding_name =3D "AMR", .rate =3D 8000, .fmtp =3D=
 "octet-align=3D1;mode-set=3D0,2,4", .payload_type =3D 112, },
                        { .encoding_name =3D "AMR", .rate =3D 8000, .fmtp =3D=
 "mode-set=3D0,2,4,7", .payload_type =3D 112, },
                        { .encoding_name =3D "AMR", .rate =3D 8000, .fmtp =3D=
 "mode-set=3D0,2,4", .payload_type =3D 112, },
                        { .encoding_name =3D "GSM-EFR", .rate =3D 8000, .payl=
oad_type =3D 110, },
                        { .encoding_name =3D "GSM", .rate =3D 8000, .payload_=
type =3D 3, },
                        { .encoding_name =3D "GSM-HR-08", .rate =3D 8000, .pa=
yload_type =3D 111, },
                        };
        int i;
        for (i =3D 0; i < ARRAY_SIZE(ran_defaults); i++)
                osmo_sdp_codec_list_add(g_ran_codecs_cfg, &defaults[i])

        printf("%s", codec->encoding_name);
        if (!strcmp(codec->encoding_name, "AMR"))
                printf(":%s", codec->fmtp);

same in opaque:

        osmo_sdp_codec_list_add(g_ran_codecs_cfg, "AMR", 8000, "octet-align=
=3D1;mode-set=3D0,2,4,7",  112);
        osmo_sdp_codec_list_add(g_ran_codecs_cfg, "AMR", 8000, "octet-align=
=3D1;mode-set=3D0,2,4",  112);
        osmo_sdp_codec_list_add(g_ran_codecs_cfg, "AMR", 8000, "mode-set=3D0,=
2,4,7",  112);
        osmo_sdp_codec_list_add(g_ran_codecs_cfg, "AMR", 8000, "mode-set=3D0,=
2,4",  112);
        osmo_sdp_codec_list_add(g_ran_codecs_cfg, "GSM-EFR", 8000, NULL, 110);
        osmo_sdp_codec_list_add(g_ran_codecs_cfg, "GSM", 8000, NULL, 3);
        osmo_sdp_codec_list_add(g_ran_codecs_cfg, "GSM-HR-08", 8000, NULL, 11=
1);

        printf("%s", osmo_sdp_codec_get_encoding_name(codec));
        if (!strcmp(osmo_sdp_codec_get_encoding_name(codec), "AMR"))
                printf(":%s", osmo_sdp_codec_get_ftmp(codec));

The opaque looks alright in comparison, but
- being able to enlist a struct osmo_sdp_codec in a const array is powerful,
  allowing the paradigm to replace code complexity by good data structures.
  So used in codec_mapping.c in osmo-msc.
- requiring getters and setters for each end every member creates a long list
  of API functions: 'get' and 'set' for each orthogonal member; we might be
  tempted to "dupe" many llist_*() functions for osmo_sdp_codec_list.
- having to call functions for each end every member makes the code a lot more
  noisy.
- it forces all allocation decisions down onto the caller =3D only using the =
API
  author's favorite dynamic allocation, hello memleaks etc.

IMHO, opaque APIs are often lots of code with very low density of actually
important stuff.

  /* This function gets foo. */
  struct foo *get_foo(x)
  {
     return x->foo;
  }

This is so plump and boring!!

Currently I am following the transparent way,
and putting 'bool v2;' extension flags for a distant future in the structs'
ends.

But there has been a discussion here that the truly good APIs are opaque.
talloc, libSDL, libsndfile come to mind.


=3D=3D both =3D=3D

Finally, these two aspects are interdependent:
An opaque API *has* to be dynamic.

So far, I am moving from the transparent+static implementation to
transparent+dynamic.

Should I also go the "next" step to opaque+dynamic? I don't really want to...


I've had these considerations many times in my life, and there never seems to
be the one best way.
Thanks for your thoughts!


If you need more code reference for what I am talking about:

old static:
https://cgit.osmocom.org/osmo-msc/tree/include/osmocom/msc/sdp_msg.h?id=3D1b1=
a39bea1c528830d7c752c7c8d7c9a33141046
https://cgit.osmocom.org/osmo-msc/tree/src/libmsc/sdp_msg.c?id=3D1b1a39bea1c5=
28830d7c752c7c8d7c9a33141046

new dynamic+transparent:
https://cgit.osmocom.org/osmo-mgw/tree/include/osmocom/sdp?h=3Dneels/sdp&id=
=3D24c09fbacbc93a60dfbb932e1a985931249d4f7b
https://cgit.osmocom.org/osmo-mgw/tree/src/libosmo-sdp?h=3Dneels/sdp&id=3D24c=
09fbacbc93a60dfbb932e1a985931249d4f7b

~N

--=20
- Neels Hofmeyr <nhofmeyr(a)sysmocom.de>          http://www.sysmocom.de/
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Gesch=C3=A4ftsf=C3=BChrer / Managing Directors: Harald Welte

--===============2906896554981168645==--


From pespin@sysmocom.de Mon Feb 26 09:57:39 2024
From: Pau Espin Pedrol <pespin@sysmocom.de>
To: openbsc@lists.osmocom.org
Subject: Re: SDP API choices
Date: Mon, 26 Feb 2024 10:57:16 +0100
Message-ID: <dc5d4db9-8df6-4857-aa19-ae6b13d3ee66@sysmocom.de>
In-Reply-To: <Zduzr7QSGk1aZITe@my.box>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============6700590176445304335=="

--===============6700590176445304335==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Hi Neels,

not sure what kind of answer you expect here, probably just some sort of 
poll since most of the "rant" was already provided :)

As usual, I'd go for opaque. Since opaque means the implementation can 
be changed, then it doesn't matter much whether you want to use dynamic 
or static. It can be a mix of both, use whatever fits more in each place.
But I'd definitely make the lib allocate the structs and have them as 
opaque pointers.

Regarding "benefits of iterating over llist", you can simply have an API 
return the llist pointer and have some API to get the object from the 
llist per-item pointer which is internally implemented by means of 
container_of() or a cast or similar.

Regards,
Pau

-- 
- Pau Espin Pedrol <pespin(a)sysmocom.de>         http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte

--===============6700590176445304335==--


From nhofmeyr@sysmocom.de Mon Feb 26 18:36:36 2024
From: Neels Hofmeyr <nhofmeyr@sysmocom.de>
To: openbsc@lists.osmocom.org
Subject: Re: SDP API choices
Date: Mon, 26 Feb 2024 19:36:31 +0100
Message-ID: <ZdzaL/KQ+CoeCQCw@my.box>
In-Reply-To: <dc5d4db9-8df6-4857-aa19-ae6b13d3ee66@sysmocom.de>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============0311970483963056883=="

--===============0311970483963056883==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Yeah I am having the landscape of choices mapped out, and I'd like to get a
feel for the general opinion to fine tune my heading.
"At what degree of which aspects do you guys see the optimum?"

On Mon, Feb 26, 2024 at 10:57:16AM +0100, Pau Espin Pedrol wrote:
> As usual, I'd go for opaque. Since opaque means the implementation can be
> changed, then it doesn't matter much whether you want to use dynamic or
> static. It can be a mix of both, use whatever fits more in each place.

An opaque API has to be dynamic.
There is always a struct foo * that was returned by a dynamic foo_alloc().

> But I'd definitely make the lib allocate the structs and have them as opaque
> pointers.

Ok, poll taken: the benefit of easily modifying the API later weighs more than
having to write dozens of foo_get_x() and foo_set_x() and calling functions a=
ll
the time instead of using plain pointer arithmetic.

> Regarding "benefits of iterating over llist", you can simply have an API
> return the llist pointer and have some API to get the object from the llist
> per-item pointer which is internally implemented by means of container_of()
> or a cast or similar.

Yes. But is it really opaque to return an llist_head? Should the API maybe hi=
de
the particular list implementation and provide functions for walking the list?


What I'd also like your opinion on is, how far would you take the dynamic-nes=
s?
Is it fine to have many detailed allocation "stages", for the benefit of logic
and opacity? Example:

I like it when the object itself is the talloc context for its members:

  struct codec_list *my_list =3D codec_list_alloc(ctx);
  codec_list_add(my_list, item); //  <-- use my_list as talloc parent for the=
 new entry

That means I want to allocate an llist_head instance dynamically. Going this
way makes a lot more, small dynamic allocations.

(benefit: the logical structure of the data shows very nicely in the talloc
reports!)



An alternative would be: use an llist_head that the caller simply has as stat=
ic
member of its state:

  struct llist_head my_list;
  codec_list_add(&my_list, ctx, item); // <-- always provide the talloc ctx s=
eparately

It has less separate dynamic allocations, but there are more ways to shoot
oneself in the foot, and more ctx arguments all over the API.
Also it's not really opaque anymore.

At the moment I want to go for many small dynamic allocations.
Fine?


Ah and also, when the osmo_sdp_codec{} is opaque, how would we set up the data
structures in codec_mapping.c? I guess with a function that procedurally buil=
ds
an allocated list? Currently it's an (IMHO nice) const array of pure data:
https://gerrit.osmocom.org/plugins/gitiles/osmo-msc/+/master/src/libmsc/codec=
_mapping.c
If struct osmo_sdp_codec is transparent, it can stay this way. If I make it
opaque, every part here that says .sdp =3D {...} would have to be dynamically
created via API instead.

(In my current status, osmo_sdp_codec is transparent. If the opinions are
strong I'll add effort to make it opaque)

Another idea would be to invent a separate SDP struct in osmo-msc, but that
also needs more API to convert one to the other... I don't want that.

Thanks!!
~N

--===============0311970483963056883==--


From pespin@sysmocom.de Mon Feb 26 19:42:14 2024
From: Pau Espin Pedrol <pespin@sysmocom.de>
To: openbsc@lists.osmocom.org
Subject: Re: SDP API choices
Date: Mon, 26 Feb 2024 20:42:09 +0100
Message-ID: <9dccaf32-fd15-4275-ba94-47faac574441@sysmocom.de>
In-Reply-To: <ZdzaL/KQ+CoeCQCw@my.box>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2811361858404811300=="

--===============2811361858404811300==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hi,

On 2/26/24 19:36, Neels Hofmeyr wrote:
> An opaque API has to be dynamic.
> There is always a struct foo * that was returned by a dynamic foo_alloc().

Yeah, sure, I meant the structs below those, the ones which are/may be=20
allocated internally.

>=20
>> But I'd definitely make the lib allocate the structs and have them as opaq=
ue
>> pointers.
>=20
> Ok, poll taken: the benefit of easily modifying the API later weighs more t=
han
> having to write dozens of foo_get_x() and foo_set_x() and calling functions=
 all
> the time instead of using plain pointer arithmetic.

Yes.

>=20
>> Regarding "benefits of iterating over llist", you can simply have an API
>> return the llist pointer and have some API to get the object from the llist
>> per-item pointer which is internally implemented by means of container_of()
>> or a cast or similar.
>=20
> Yes. But is it really opaque to return an llist_head? Should the API maybe =
hide
> the particular list implementation and provide functions for walking the li=
st?

I see no problem with returning an llist_head (when it makes sense to do=20
so).
I'd say it really depends on the very specific use case.

>=20
>=20
> What I'd also like your opinion on is, how far would you take the dynamic-n=
ess?
> Is it fine to have many detailed allocation "stages", for the benefit of lo=
gic
> and opacity? Example:
>=20
> I like it when the object itself is the talloc context for its members:
>=20
>    struct codec_list *my_list =3D codec_list_alloc(ctx);
>    codec_list_add(my_list, item); //  <-- use my_list as talloc parent for =
the new entry

Again I'd say this kind of decision is really specific to each use case,=20
depending on the relations between objects, ownserships, etc.
I'd say in general it's fine as long as parent/child relations between=20
them are clear, so that the child makes no sense out of the parent.

>=20
> That means I want to allocate an llist_head instance dynamically. Going this
> way makes a lot more, small dynamic allocations.

I think you actually want to say is (or I want to see it this way): "I=20
want to allocate an object which is a container of other items".
That object being implemented as a llist (having a field llist_head) is=20
an implementation detail. The fact that you add an API to return an=20
pointer to an llist_head which allows iterating over the items contained=20
in the parent object is *not* an implementation detail, it's simply an=20
API of the object.

>=20
> (benefit: the logical structure of the data shows very nicely in the talloc
> reports!)
>=20
>=20
>=20
> An alternative would be: use an llist_head that the caller simply has as st=
atic
> member of its state:
>=20
>    struct llist_head my_list;
>    codec_list_add(&my_list, ctx, item); // <-- always provide the talloc ct=
x separately
>=20
> It has less separate dynamic allocations, but there are more ways to shoot
> oneself in the foot, and more ctx arguments all over the API.
> Also it's not really opaque anymore.

It really depends on the usercase. If you really only want an llist=20
which is not to be managed from the library, then you could go this way.=20
But then freeing everything becomes more cumbersome.

In general I'd gor for the other example you shared first.

>=20
> At the moment I want to go for many small dynamic allocations.
> Fine?

You can always add an object pool if small dynamic allocations become a=20
problem (malloc locks, fragmentation, etc.).
The good part is that this is all implementation details encapsulated in=20
the allocation functions.

>=20
>=20
> Ah and also, when the osmo_sdp_codec{} is opaque, how would we set up the d=
ata
> structures in codec_mapping.c? I guess with a function that procedurally bu=
ilds
> an allocated list? Currently it's an (IMHO nice) const array of pure data:
> https://gerrit.osmocom.org/plugins/gitiles/osmo-msc/+/master/src/libmsc/cod=
ec_mapping.c
> If struct osmo_sdp_codec is transparent, it can stay this way. If I make it
> opaque, every part here that says .sdp =3D {...} would have to be dynamical=
ly
> created via API instead.

I see not much of a problem with it. That's created once at startup anyway?
If you need to refer to it from different places, it can be global and=20
just take the pointer to one of them.

>=20
> (In my current status, osmo_sdp_codec is transparent. If the opinions are
> strong I'll add effort to make it opaque)

My opinion/experience is in general go for opaque, specially for complex=20
objects/structures which can be updated in the future.

And from the start, add a _internal.h header and add there all function=20
declarations there until you see they are really needed somewhere=20
outside of the library.

Regards,
Pau

--=20
- Pau Espin Pedrol <pespin(a)sysmocom.de>         http://www.sysmocom.de/
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte

--===============2811361858404811300==--


From mauro.levra@viavisolutions.com Tue Feb 27 14:38:13 2024
From: Mauro Levra <mauro.levra@viavisolutions.com>
To: openbsc@lists.osmocom.org
Subject: Re: GSMTAP new version (3?)
Date: Tue, 27 Feb 2024 14:38:11 +0000
Message-ID: <170904469163.433.498305159599353662@lists>
In-Reply-To: =?utf-8?q?=3CMN0PR18MB59004FA6D02B3445CD37D1DC98732=40MN0PR18MB?=
 =?utf-8?q?5900=2Enamprd18=2Eprod=2Eoutlook=2Ecom=3E?=
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============3812555447229766219=="

--===============3812555447229766219==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Hi,

Since no one has replied to my previous very generic message, I come forward
with a proposal hoping to trigger a discussion.

  As you know, the current definition of the GSMTAP header has a version
field that could enable new versions to be safely introduced. Unfortunately,
Wireshark decoder does not check the value of the version field, it just
presents the value in the decoded output. It seems that the only way to
define a new GSMTAP version that does not break Wireshark decoder is to
preserve the initial 16 header octets.

  This is the current gsmtap_hdr definition. Some fields are only relevant
in the context of a GSM capture, while the meaning of others (notably type,
sub_type) has changed over time to describe a payload of a newer technology.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    version    |    hdr_len    |     type      |    timeslot   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             arfcn             |  signal_dbm   |    snr_db     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         frame_number                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   sub_type    |   antenna_nr  |   sub_slot    |   reserved    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   ~                                                               ~
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                            Payload                            |
   ~                                                               ~
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  When hdr_len is greater than 4 (the length is 4 * 4 = 16 octets), it is
possible to store additional information in the header, without breaking
any compliant decoder implementation. This new header section could contain
a sequence of TLV entries with a 2 octet tag, a 2 octet length field, and a
value field padded to the nearest 32 bit boundary.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |              tag              |            length             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             value                             |
   ~                                                               ~
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The length field allows decoders to skip unknown tags. To improve storage
performance, a subset of tags for values with a length less or equal to 2
octets can be defined (something like this was proposed in 2012 during the
GSMTAPv3 session). Like this:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|            tag              |   value (padded to 16 bits)   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |1|            tag              |            length             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             value                             |
   ~                                                               ~
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

In this last variant, we have 32k tags for "small" values and 32k tags for
full TLV entries.

  Finally, an additional bit can be used to distinguish between official
Osmocom tag definitions, and application specific tags, that are valid only
within the boundaries of a specific implementation and could be used to
embed metadata that has not yet been assigned an official tag.

What are your thoughts about this?

Mauro

--===============3812555447229766219==--


From tel@peremen.name Tue Feb 27 19:22:49 2024
From: Shinjo Park <tel@peremen.name>
To: openbsc@lists.osmocom.org
Subject: Re: GSMTAP new version (3?)
Date: Tue, 27 Feb 2024 20:22:42 +0100
Message-ID: <7635228.EvYhyI6sBW@ainazi>
In-Reply-To: <170904469163.433.498305159599353662@lists>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============7100462765777212007=="

--===============7100462765777212007==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hello Mauro,

Thanks for this message. I was preparing my suggestion at https://
gist.github.com/peremen/f8f199bf89e8f4c3e0b5e3f9e141b275#file-gsmtapv3-md whi=
ch=20
I targeted to migrate to the Osmocom's gitea some time ago.

> It seems that the only way to
> define a new GSMTAP version that does not break Wireshark decoder is to
> preserve the initial 16 header octets.

That was open also in my suggestion, but if older Wireshark will continue to =

parse the GSMTAP header for the whole 16 octets when the type is unknown, the=
n=20
I am also aligning towards this idea. I was thinking allocating a "protective=
=20
type" on GSMTAPv2 to not parse GSMTAPv3 packets by GSMTAPv2-only=20
implementation.

> This new header section could contain
> a sequence of TLV entries with a 2 octet tag, a 2 octet length field, and a
> value field padded to the nearest 32 bit boundary.

What I was thinking is defining a new data structure for each payload type an=
d=20
optionally extend them in the future using TLV entries. See my proposal for=20
the type-specific data structure.

The compact storage format to cover both TV and TLV entries sounds also good.=
=20
Given that 1) existing GSM applications can still use GSMTAPv2 (correct me if=
=20
I am wrong - whether GSMTAPv2 and v3 will be coexisting) and 2) for 4G and 5G=
=20
RRC/NAS, payloads could be longer than 1000 octets, where 2 additional length=
=20
octets may not actually be relevant at all, using TLV exclusively might not b=
e=20
so problematic regarding the performance.

> Finally, an additional bit can be used to distinguish between official
> Osmocom tag definitions, and application specific tags, that are valid only
> within the boundaries of a specific implementation and could be used to
> embed metadata that has not yet been assigned an official tag.

I was thinking similar usage by reserving a range of types for this purpose. =

Tags should be also included too.

In addition, I will going to present a talk in this OsmoDevCon [1], and I am =

currently planning to finalize the proposal before the conference.

[1] https://osmocom.org/projects/osmo-dev-con/wiki/OsmoDevCon2024

Shinjo

2024=EB=85=84 2=EC=9B=94 27=EC=9D=BC =ED=99=94=EC=9A=94=EC=9D=BC =EC=98=A4=ED=
=9B=84 3=EC=8B=9C 38=EB=B6=84 11=EC=B4=88 CET=EC=97=90 Mauro Levra =EB=8B=98=
=EC=9D=B4 =EC=93=B4 =EA=B8=80:
> Hi,
>=20
> Since no one has replied to my previous very generic message, I come forward
> with a proposal hoping to trigger a discussion.
>=20
>   As you know, the current definition of the GSMTAP header has a version
> field that could enable new versions to be safely introduced. Unfortunately,
> Wireshark decoder does not check the value of the version field, it just
> presents the value in the decoded output. It seems that the only way to
> define a new GSMTAP version that does not break Wireshark decoder is to
> preserve the initial 16 header octets.
>=20
>   This is the current gsmtap_hdr definition. Some fields are only relevant
> in the context of a GSM capture, while the meaning of others (notably type,
> sub_type) has changed over time to describe a payload of a newer technology.
>=20
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |    version    |    hdr_len    |     type      |    timeslot   |
>=20
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |             arfcn             |  signal_dbm   |    snr_db     |
>=20
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |                         frame_number                          |
>=20
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |   sub_type    |   antenna_nr  |   sub_slot    |   reserved    |
>=20
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    ~                                                               ~
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |                            Payload                            |
>=20
>    ~                                                               ~
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>   When hdr_len is greater than 4 (the length is 4 * 4 =3D 16 octets), it is
> possible to store additional information in the header, without breaking
> any compliant decoder implementation. This new header section could contain
> a sequence of TLV entries with a 2 octet tag, a 2 octet length field, and a
> value field padded to the nearest 32 bit boundary.
>=20
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |              tag              |            length             |
>=20
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |                             value                             |
>=20
>    ~                                                               ~
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
> The length field allows decoders to skip unknown tags. To improve storage
> performance, a subset of tags for values with a length less or equal to 2
> octets can be defined (something like this was proposed in 2012 during the
> GSMTAPv3 session). Like this:
>=20
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |0|            tag              |   value (padded to 16 bits)   |
>=20
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |1|            tag              |            length             |
>=20
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>    |                             value                             |
>=20
>    ~                                                               ~
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
> In this last variant, we have 32k tags for "small" values and 32k tags for
> full TLV entries.
>=20
>   Finally, an additional bit can be used to distinguish between official
> Osmocom tag definitions, and application specific tags, that are valid only
> within the boundaries of a specific implementation and could be used to
> embed metadata that has not yet been assigned an official tag.
>=20
> What are your thoughts about this?
>=20
> Mauro





--===============7100462765777212007==--


From nhofmeyr@sysmocom.de Thu Feb 29 15:22:30 2024
From: Neels Hofmeyr <nhofmeyr@sysmocom.de>
To: openbsc@lists.osmocom.org
Subject: Re: GSMTAP new version (3?)
Date: Thu, 29 Feb 2024 16:22:24 +0100
Message-ID: <ZeChMJcX0lgoHIHI@my.box>
In-Reply-To: <170904469163.433.498305159599353662@lists>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2913719279721026237=="

--===============2913719279721026237==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

I'm not deeply familiar with the topic, but reading along, the below would be
my bikeshed feedback =3D)

On Tue, Feb 27, 2024 at 02:38:11PM -0000, Mauro Levra wrote:
>   As you know, the current definition of the GSMTAP header has a version
> field that could enable new versions to be safely introduced. Unfortunately,
> Wireshark decoder does not check the value of the version field, it just
> presents the value in the decoded output. It seems that the only way to
> define a new GSMTAP version that does not break Wireshark decoder is to
> preserve the initial 16 header octets.

Just to understand the aim: we can of course fix the wireshark dissector to
handle the version properly. The idea to keep v3 backwards compatible is only
about making older wireshark installations "magically" compatible with the new
gsmtap version? I find that this is just a nice-to-have, and i have often
upgraded wireshark in order to support a specific protocol. What makes v3 to =
v2
compatibility such an important requirement? For me it is "just" a short
transition phase until all users have gotten around to upgrade their wireshark
installation?
(If someone back in 2012 had fixed wireshark to handle the version number
properly, then no-one would even mention it today. Seems silly to design a
protocol around such a short-lived peculiarity of one specific program.)

> The length field allows decoders to skip unknown tags. To improve storage
> performance, a subset of tags for values with a length less or equal to 2
> octets can be defined (something like this was proposed in 2012 during the
> GSMTAPv3 session). Like this:
>=20
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |0|            tag              |   value (padded to 16 bits)   |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |1|            tag              |            length             |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |                             value                             |
>    ~                                                               ~
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

We already have at least two TLV parsing APIs in osmocom, and having written
one of them, I'd like to give this feedback: things become a lot easier when
there is no variance in the sizes of the length field. If the aim of this is =
to
save one byte of payload, I find this quite anachronistic. In 2024, we don't
need to optimize for network bandwidth, instead we want to optimize for CPU
load. It makes for far simpler (optimizable) decoders when T and L have a
single fixed size. I guess there is a long history behind this proposal, but
the voice in my head says: in 2024, Why not pick 16bit for both T and L from
the start and be done with it. Allowing 64k tags of 64k octets without special
implications seems more appropriate for the future than still dabbling with
designs that have a limit of 256 on a very low protocol level.

For example, PFCP is one of the newer 3GPP protocols, and it features a
T16L16V.

Looking at libosmo-gtlv as an example for a TLV coder, it is of course possib=
le
to have an osmo_gtlv_cfg.load_tl() function that knows which tag is longer and
which is shorter -- it then will decide on a size for every single tag value
that is encountered. Just, it seems to me more desirable that iterating tags
should be fast, just copying fixed sizes without code branches.

It is then also easier to skip unknown tags.

> In this last variant, we have 32k tags for "small" values and 32k tags for
> full TLV entries.
>=20
>   Finally, an additional bit can be used to distinguish between official
> Osmocom tag definitions, and application specific tags, that are valid only
> within the boundaries of a specific implementation and could be used to
> embed metadata that has not yet been assigned an official tag.

Slight nuance of definition:

I would recommend not assigning a specific single bit to a specific vendor,
because it is too specific. Instead I'd reserve a tag range for gsmtapv3, and
leave the rest "application specific" for any vendors to do what they please.

For example, in case of T16, we can reserve 0-0x7fff for future "native"
protocol extensions. By defining this way around, instead of having 0x8000 to
0xffff reserved specifically for Osmocom, we have 0x8000-0xffff available to
any vendor. The vendors can, within that non-reserved tag range, decide betwe=
en
each other to decide for a vendor specific tag range (or not).

It's essentially the same as you suggest, but it unloads the protocol
definition from having to coordinate specific vendors' tag ranges.

For the record, TLV is excellent for allowing vendor specific extensions,
because any parser can trivially skip parts of the message that it doesn't
understand, without any prior knowledge.

There is still the possibility that two vendors do insufficient research and
happen to use the same tag values for distinct meanings; if both vendors gain
traction, then at some point wireshark will be confused between them. If we
want to guard against this beyond doubt, there could be an officially defined
tag indicating the nature of tag extensions. Like a simple string:

  T: TAG_EXTENSION_VENDOR
  L: 9
  V: "Osmocom-1"

For example, an Osmocom vendor could be required to include this TLV,
indicating that extension tags should be interpreted as Osmocom version 1. In
the lack of such, any parser can skip or error unknown tags.

At some point, "Osmocom-2" could reshuffle the used tag values, etc.

(This tag could itself be an osmocom specific tag, but it would be much more
elegant protocol design if there is one common mechanism across all vendors to
define usage of the vendor-specific range.)


Finally, I have yet another idea for vendor specific extensions: in PFCP, the=
re
is a concept of "grouped IEs". In essence, a nested TLV structure inside an
outer V.

So we could define a *single* vendor specific extension tag, that can contain
within it any number of vendor specific TLVs, like this:

   T: TAG_EXTENSION
   L: 456
   V: {
      octet 0: vendor_id_len=3D9
      octet 1..n: "Osmocom-1"
      octet n+1..456: nested TLV structure

      T: ...
      L: ...
      V: { ... }

      T: ...
      L: ...
      V: { ... }
      }

In this variant, the "TAG_EXTENSION" is a fixed official tag, and it defines
that the start of V defines a vendor id string.

Any parsers that don't know extensions skip the entire vendor-specific
subsection in one jump (in above example, skip 456 octets once instead of each
vendor tag one by one).

A parser that knows Osmocom extensions will look at the TAG_EXTENSION V, veri=
fy
that it starts with a supported "Osmocom-1" string, and only then evaluate the
TLV within.

(For libosmo-gtlv, this is already supported, because we already implemented
this for PFCP parsing.)

Those are my two cents! Just some ideas from the off.

~N

--===============2913719279721026237==--


From mauro.levra@viavisolutions.com Thu Feb 29 17:16:31 2024
From: Mauro Levra <mauro.levra@viavisolutions.com>
To: openbsc@lists.osmocom.org
Subject: Re: GSMTAP new version (3?)
Date: Thu, 29 Feb 2024 17:16:29 +0000
Message-ID: <170922698916.433.6193123022205384771@lists>
In-Reply-To: <ZeChMJcX0lgoHIHI@my.box>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============3165023967790879850=="

--===============3165023967790879850==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Thank you, Neels and Shinjo, for your insights.


> What I was thinking is defining a new data structure for each payload type
> and optionally extend them in the future using TLV entries. See my
> proposal for the type-specific data structure.

@Shinjo, I did not know about your existing proposal. I understand the idea
about type specific headers, but I think it may lead to problems and
increased compatibility issues in the future, as more types are added. I
would suggest to use TLV exclusively.

> In addition, I will going to present a talk in this OsmoDevCon, and I am 
> currently planning to finalize the proposal before the conference.

That is good news, good luck for your proposal.


---

@Neels, do not say you are bikeshedding, because your comments are not
trivial at all... :D

> Just to understand the aim: we can of course fix the wireshark dissector
> to handle the version properly. The idea to keep v3 backwards compatible
> is only about making older wireshark installations "magically" compatible
> with the new gsmtap version?

Yes, it is only about that. For me it is not a strong requirement, just a
nice-to-have, as you say.

> What makes v3 to v2 compatibility such an important requirement?

It can be a design constraint, or not. In my initial message I tried to lay
down the most safe and backward compatibile proposal possible.

> (If someone back in 2012 had fixed wireshark to handle the version number
> properly, then no-one would even mention it today. Seems silly to design a
> protocol around such a short-lived peculiarity of one specific program.)

I can send a merge request to Wireshark team to address this. The sooner we
add this version check the better.

> things become a lot easier when there is no variance in the sizes of the
> length field. If the aim of this is to save one byte of payload, I find
> this quite anachronistic.

From the notes of the 2012 talk, I understood that someone would be happy
with this level of optimization. Having a single format (T16L16V) will of
course be the most simple and efficient (CPU wise) solution. Also, 12 years
have passed.

> It's essentially the same as you suggest, but it unloads the protocol
> definition from having to coordinate specific vendors' tag ranges.

I agree, it is a nice change of perspective.

  The final part of your email about extensions and vendor specific tags is
really interesting, but I do not have enough experience to give you a useful
feedback. I wonder if we can copy this mechanism from an existing protocol
definition.

Thanks,

Mauro

--===============3165023967790879850==--