13 Jan
2018
13 Jan
'18
11:12 p.m.
Dear Harald,
In this mail I would like to express my thoughts and ideas
regarding to the 'external interface for USSD'. To provide
some background for other mailing list members, who are not
aware of this topic, I'll explain it in a few sentences.
We already have an external interface for SMS - SMPP. But
unfortunately, the same cannot be said about USSD. There is
no built-in support of any interface, which could be used to
handle USSD requests in a separate application. So, at the
moment we only handle *#100# (own number request), which is
implemented as a part of both OsmoMSC and legacy OsmoNiTB.
And if one would like to implement additional USSD commands,
it's required to modify and recompile the whole project :/
Fairwaves team already have some draft implementation for legacy
OsmoNiTB, which relays on the libosmocore's GSM 04.80 API. And
having the external interface in the mainline would be great
not only for Fairwaves, but for the whole Osmocom project
and its users.
The problem is that the current libosmocore's GSM 04.80 API
is not complete and requires some critical modifications,
which of course cannot be integrated immediately. Moreover,
this implementation doesn't follow Osmocom coding style,
for example, unlike other functions, where return code
rc = 0 means success, there it means error...
So, first of all, we need to know, how many projects are
using the current API, especially non-Osmocom projects.
After that (taking it into account), it shall be decided:
- Should we keep the old API / ABI and maintain all new
features among with it. This way would force us to
duplicate the existing code and use different symbols.
Then the old API would continuously became deprecated...
- Should we make all the critical changes in the current
API (increasing the libosmocore version?). This way would
break builds of dependent projects and would suppose
fixing compatibility with the new API. But, at the same
time, the implementation would be cleaner and closer to
the specification, without any deprecated duplication...
Personally, I prefer the second way. At the same time, I want
to make the modification as much transparent as possible.
Speaking about Osmocom projects, the only users of the GSM
04.80 API are OsmoMSC and legacy OsmoNiTB. Both projects
utilize it for '*#100#' handling only :/
To be more convincing, let's look at the following code:
#define MAX_LEN_USSD_STRING 31
struct ss_request {
uint8_t opcode;
uint8_t ss_code;
uint8_t ussd_text[MAX_LEN_USSD_STRING + 1];
uint8_t transaction_id;
uint8_t invoke_id;
};
This is what the current API allows you to obtain from a
SS-request (e.g. '*#100#') coming from user. And what's
wrong here?
- The 'ss_request' doesn't indicate a part of the library
it comes from. Would be better to use 'gsm0480_ss_request'.
- The 'ussd_text' length doesn't follow the specification,
where USSD OCTET STRING length is 160 bytes. So, the amount
of characters depends on used coding scheme.
- The information about SS-message type, component type, text
length and used DCS (Data Coding Scheme), etc. is missing.
Despite it could be important for the external application.
So, let's discuss together all the 'pros and cons', and decide
together, how should we facilitate the external USSD interface
development.
With best regards,
Vadim Yanitskiy.
14 Jan
14 Jan
midnight
Hi Vadim,
I'm sorry to respond to your elaborate mail in a very short way:
We do not want to break building old versions of OsmoNITB & Co
with new library versions. Every time we do this, we are creating
problems to our users. We have done this several times in the
past, and it's always a pain to handle.
Let's assume anyone wants to build an old OsmoNITB version, for example
in order to test for a regression (e.g. using 'git bisect'). How
would you do this, if for every version you want to test, you need
also *all* the matching libosmo* of that time. To make things worse,
you don't even know which of the versions you need to use, making
it impossible to hunt down when a regression was introduced in an effective
way.
It would be really nice to have some automatic jenkins build jobs that
actually test at least building old code (e.g. openbsc.git) against
current 'master' libraries to avoid accidents, and to even clean up some
of the incompatibilities we may have introduced since, if possible.
It's bad enough if we break this by accident. It's much worse if we
do it knowingly.
So whatever we do, we have to try our best to keep old APIs stable
or backwards-compatible. In most cases, the old API would simply
be a compatibility wrapper around the new API. So there's one
implementation of a given functionality, but the new users use it
directly, while the old users go through a compatibility wrapper.
I don't have the details of the USSD code in my head, but I'd be surprised
if it wouldn't be possible to simply wrap the new API behind an old
API compat wrapper? If you encounter any specific problems, feel
free to discuss ways how we can do this.
Regards,
Harald
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
7:16 a.m.
Hi Vadim, Harald,
I wanted to speak up and second the need to fix the USSD system, as Vadim
notes it's actually just broken at the moment and doesn't meet the
standard. A student in my lab (Rowan, cc'd) recently had to extend it to be
able to interface with ussd_airflow, I believe building off of earlier
fairwaves patches. We'd also like to be able to upstream those fixes to the
community, but it seems that Harald's legacy support issue is the big
hangup? Wouldn't it be easier to put related library versions into the
changelog or something similar? Git submodules for instance allow you to
specify a specific submodule hash and a mechanism like that could free up
the interface development.
Just some thoughts. Thanks to everyone for bringing this issue up.
On Sat, Jan 13, 2018 at 3:00 PM, Harald Welte <laforge(a)gnumonks.org> wrote:
Hi Vadim,
I'm sorry to respond to your elaborate mail in a very short way:
We do not want to break building old versions of OsmoNITB & Co
with new library versions. Every time we do this, we are creating
problems to our users. We have done this several times in the
past, and it's always a pain to handle.
Let's assume anyone wants to build an old OsmoNITB version, for example
in order to test for a regression (e.g. using 'git bisect'). How
would you do this, if for every version you want to test, you need
also *all* the matching libosmo* of that time. To make things worse,
you don't even know which of the versions you need to use, making
it impossible to hunt down when a regression was introduced in an effective
way.
It would be really nice to have some automatic jenkins build jobs that
actually test at least building old code (e.g. openbsc.git) against
current 'master' libraries to avoid accidents, and to even clean up some
of the incompatibilities we may have introduced since, if possible.
It's bad enough if we break this by accident. It's much worse if we
do it knowingly.
So whatever we do, we have to try our best to keep old APIs stable
or backwards-compatible. In most cases, the old API would simply
be a compatibility wrapper around the new API. So there's one
implementation of a given functionality, but the new users use it
directly, while the old users go through a compatibility wrapper.
I don't have the details of the USSD code in my head, but I'd be surprised
if it wouldn't be possible to simply wrap the new API behind an old
API compat wrapper? If you encounter any specific problems, feel
free to discuss ways how we can do this.
Regards,
Harald
--
- Harald Welte <laforge(a)gnumonks.org>
http://laforge.gnumonks.org/
============================================================
================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch.
A6)
16 Jan
16 Jan
1:53 a.m.
On Sun, Jan 14, 2018 at 3:16 PM, Kurtis Heimerl
<kheimerl(a)cs.washington.edu> wrote:
I wanted to speak up and second the need to fix the
USSD system, as Vadim
notes it's actually just broken at the moment and doesn't meet the standard.
A student in my lab (Rowan, cc'd) recently had to extend it to be able to
interface with ussd_airflow, I believe building off of earlier fairwaves
patches. We'd also like to be able to upstream those fixes to the community,
Do you have a repo with this integration? This might be interesting
for us as well. We're currently implementing USSD services in Erlang
but a more non-programmer friendly YAML might be a good option in some
cases.
--
Regards,
Alexander Chemeris.
CTO/Founder, Fairwaves, Inc.
Skype: Alexander.Chemeris
Mobile: +1(424)400-7626
https://fairwaves.co
Subscribe to Fairwaves news: http://eepurl.com/baL_pf
8:21 a.m.
Hi Alexander,
Most of the work I did was based off the libosmocore fairwaves/sup-ussd
branch. As that branch stands, the first USSD message and its response
are handled fine, where it breaks down, is if the service requests more
information from the user. On subsequent messages, it looks like the phone
leaves out a few critical bytes including the opcode. After reading the
standards documents I still do not understand why this is the case but it
was observed on multiple handsets and seems to be normal. Most of the
modifications I had to make involved handling the other USSD message types
and dealing with the fallout from having the opcode missing. I also added
some additional fields to the ss_request struct to track the text length,
message type and component type. I changed the max string length from 31
to 160 but I'm not sure if it should be longer. USSD strings are
transmitted in a 7 bit packed format where the maximum length is 160
bytes. When expanded out into standard ascii this ends up around 182
characters so maybe the max size should be 182 characters.
To tie into an external USSD service, I set up osmo-nitb to forward USSD
requests to a python script using sup. This script would then forward the
USSD message to another program, ussd_airflow, via a http post request.
Ussd_airflow is a Django web server that handles USSD requests and can be
configured using YAML files.
The version of libosmocore used was forked from the master branch in
October and has not been kept up to date since then, although I doubt any
of the USSD code has been changed much since then.
link: https://github.com/rowanphipps/libosmocore
The version of openbsc used was forked from the fairwaves/sup-ussd branch.
It was not modified very much although that branch had not been touched
since October 2015 so it is reasonably old.
link: https://github.com/rowanphipps/openbsc
I made some modifications to the ussd_airflow web server to allow it it
handle multiple different service codes but the original would work fine as
well.
documentation: https://django-ussd-airflow.readthedocs.io/en/latest/
original: https://github.com/mwaaas/ussd_airflow
modified: https://github.com/rowanphipps/ussd_airflow
And here is the repo with my python script to link all the parts together:
https://github.com/rowanphipps/sup_airflow
As for creating an external interface, the current sup interface could be
left exposed with the idea that people could just write scripts to parse
these messages and forward them as needed. If this is chosen then the sup
protocol needs to be better documented and standardized. Sup seems to have
been changed in more recent branches and now functions differently and in a
way that the old fairwaves code does not work with. The main advantage of
keeping sup is that it is very simple to extend to use with other USSD
systems.
Let me know if you have any questions regarding the changes I made and how
it works.
On Mon, Jan 15, 2018 at 4:54 PM Alexander Chemeris <
Alexander.Chemeris(a)fairwaves.co> wrote:
On Sun, Jan 14, 2018 at 3:16 PM, Kurtis Heimerl
<kheimerl(a)cs.washington.edu> wrote:
--
Thanks!
- Rowan Phipps
I wanted to speak up and second the need to fix
the USSD system, as Vadim
notes it's actually just broken at the moment and doesn't meet the
standard.
A student in my lab (Rowan, cc'd) recently
had to extend it to be able to
interface with ussd_airflow, I believe building off of earlier fairwaves
patches. We'd also like to be able to upstream those fixes to the
community,
Do you have a repo with this integration? This might be interesting
for us as well. We're currently implementing USSD services in Erlang
but a more non-programmer friendly YAML might be a good option in some
cases.
--
Regards,
Alexander Chemeris.
CTO/Founder, Fairwaves, Inc.
Skype: Alexander.Chemeris
Mobile: +1(424)400-7626 <(424)%20400-7626>
https://fairwaves.co
Subscribe to Fairwaves news: http://eepurl.com/baL_pf
1:52 a.m.
Hi, Harald,
I'm surprised to read this because I'm quite sure that you suggested
breaking the API/ABI and increasing the library number yourself, and
Vadim worked under the assumption that this is the suggested way to
go.
On Sun, Jan 14, 2018 at 8:00 AM, Harald Welte <laforge(a)gnumonks.org> wrote:
Let's assume anyone wants to build an old OsmoNITB
version, for example
in order to test for a regression (e.g. using 'git bisect'). How
would you do this, if for every version you want to test, you need
also *all* the matching libosmo* of that time.
I personally believe that switching all libraries versions is the only
right way to do in this situation because there are quite a lot of
synchronized changes in both OsmoNITB and the underlying libraries.
And specifically with hunting a regression example I would want to use
a version of all the code in the exact state as it was before.
To make things worse,
you don't even know which of the versions you need to use, making
it impossible to hunt down when a regression was introduced in an effective
way.
I thought that library versions serve exactly this purpose - to know
which version to build against?
Another way is something similar to what we've done in osmo-combo repo
which allows you to specify specific commits in each sub-module to
build against.
So whatever we do, we have to try our best to keep old
APIs stable
or backwards-compatible. In most cases, the old API would simply
be a compatibility wrapper around the new API. So there's one
implementation of a given functionality, but the new users use it
directly, while the old users go through a compatibility wrapper.
I'll let Vadim comment here but from what I remember, the old
implementation is broken enough to make it difficult to maintain. I
would rather focus on making sure the new API is good enough and we
don't need to break things again in the foreseeable future.
--
Regards,
Alexander Chemeris.
CTO/Founder, Fairwaves, Inc.
Skype: Alexander.Chemeris
Mobile: +1(424)400-7626
https://fairwaves.co
Subscribe to Fairwaves news: http://eepurl.com/baL_pf
2299
days inactive
2302
days old
5 comments
5 participants
participants (5)
-
Alexander Chemeris -
Harald Welte -
Kurtis Heimerl -
Rowan Phipps -
Vadim Yanitskiy