Hi All!
I'm wondering if there are any users of the Osmocom cellular stack in Pakistan
around. If so, please kindly follow-up by private mail, as there might be
some projects in which your help would be appreciated. Thanks in advance.
Regards,
Harald
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
Dear all,
I've started to implement some test cases that validate the accuracy of
OsmoBTS and OsmoBSC SYSTEM INFORMATION generation.
Those test cases are written in TTCN-3, which is a special programming
language specifically designed for testing and validation of protocol
stacks. For those present at OsmoDevCon, I've briefly presented about
it there.
In terms of general core infrastructure, I've written:
* Definition of GSM RR and some other commonly used GSM types using the
TTCN-3 type description language and the TITAN RAW (binary) encoder/decoder.
The beauty of this is that one can simply describe the message
structure without having to generate any code for encoding/decoding
the message. See
http://git.osmocom.org/osmo-ttcn3-hacks/tree/library/GSM_Types.ttcn
and
http://git.osmocom.org/osmo-ttcn3-hacks/tree/library/GSM_SystemInformation.…
* A GSMTAP Test Port, whcih is basically how a TTCN-3 testcase/testsuite
interacts with a given IUT (implementation under test). It is
implemented as dual-faced port on top of the TITAN IPL4asp (ip layer 4)
port and implements GSMTAP header encoding/decoding from/to TTCN-3
structured data types:
http://git.osmocom.org/osmo-ttcn3-hacks/tree/library/GSMTAP_PortType.ttcn
which uses
http://git.osmocom.org/osmo-ttcn3-hacks/tree/library/GSMTAP_Types.ttcn
describing the GSMTAP header format.
* Helper functions to emulate interactive use of the VTY based on the
existing TITAN TELNETasp Test Port:
http://git.osmocom.org/osmo-ttcn3-hacks/tree/library/Osmocom_VTY_Functions.…
* L1CTL type definition (again using auto-generated encoder/decoders!)
http://git.osmocom.org/osmo-ttcn3-hacks/tree/library/L1CTL_Types.ttcn
as well as an L1CTL Test Port at
http://git.osmocom.org/osmo-ttcn3-hacks/tree/library/L1CTL_PortType.ttcn
Using the above one can easily interface from TTCN3 with L1CTL of
virtphy in order to e.g. sync to a BTS on a given ARFCN and establish
a dedicated radio channel. Here is where the beautoy of the TTCN3
type language, templates and matching can be seen. It's very easy to
perform something like "does this IMMEDIATE ASSIGNMENT match the RA +
FN of the RACH request that I sent?
Based on all of the above, there's two test suites:
== Test Suites
1) SYSTEM INFORMATION contents and scheduling validation
I've written some SI validation code which can be found at
http://git.osmocom.org/osmo-ttcn3-hacks/tree/sysinfo
SI scheduling is quite complex, particularly when you have many of the
2ter/2bis/2quater/13/2n/... enabled. At some point actually you will
overflow what's possible to comply with the rules in a BCCH Norm. and
you have to go to a BCCH Extd. (OsmoBTS doesn't implement the latter
yet, btw).
See e.g. function f_validate_si_scheduling() for validation of scheduling
constraints:
http://git.osmocom.org/osmo-ttcn3-hacks/tree/sysinfo/Test.ttcn#n229
Later in the file you can find test cases TC_cellid() and below for the
individual test cases that validate that individual VTY configured
values such as BSIC, CellId, LAC, ... are actually broadcast in the SI
message. IT will update its internal matching template and then perform
a sequence of VTY commands to e.g. change the LAC followed by verifying
that the LAC of the decoded SI has actually changed.
The above tools have helped to discover (and fix) the following bugs:
* http://osmocom.org/issues/2368
* http://osmocom.org/issues/2367
* http://osmocom.org/issues/2365 (very critical, I believe)
and also the not-yet-fixed but low-importance
* http://osmocom.org/issues/2364
2) LAPDm testing
I've written some (early) LAPDm testing code at
http://git.osmocom.org/osmo-ttcn3-hacks/tree/lapdm
The idea here is to test for valid and invalid transactions on the Um
interface and see how OsmoBTS and the libosmcore lapdm code perform.
With very few lines, this has already uncovered the following issues:
* http://osmocom.org/issues/2370
* http://osmocom.org/issues/2380
Unfortunately my progress was severely affected over the weekend as I
spent about one working day wrapping my head around something that
turned out to be a bug in the Eclipse TITAN RAW coder, see
https://www.eclipse.org/forums/index.php/t/1087557/ - fortuantely the
Titan team has provided both a work-around and a fix today, which I
count as excellent service.
If you're curious about TTCN-3, I strongly recommend
http://www.ttcn-3.org/files/TTCN3_P.pdf as an introductory slide deck.
== Outlook
I want to add more test cases to the sysinfo and lapdm test suites and
get hands-on experience with the Junit XML output plugin. At that
point, a Debian 9 build slave of our jenkins should be able to
automatically build + run the latest test cases from
osmo-ttcn3-hacks.git and present the individual test case results inside
jenkins.
I also want to add test suites for validation of:
* proper generation + scheduling of of PAGING REQUEST in the BTS,
considering paging groups particularly in the context of different
BS_AG_BLKS_RES values
* CBCH validation
* proper generation of CCCH load indications by the BTS, i.e. generate a
certian load e.g. on uplink RACH and check the reports
* TA loop verification (add timing information to GSMTAP and then
dynamically change timing from MS side to see that BTS is compensating
as expected
* power control loop verification: GSMTAP already has a field for
transmitting RSSI in dBm. We can adjust this from the MS side to
validate the BTS will adjus the instructed transmit power level of the
virtual MS accordingly
* actually exhaust all channels of a multi-TRX virtual BTS in different
channel combinations, verify we can establish all existing logical
channels in each configuration
* combine the last topic with configurations that have dynamic PDCH/TCH
channels to verify we properly switch forward and backward
* various link failure conditions, i.e. does the BTS close the logical
channel in the expected timing after uplink loss?
Despite my current excitement about all of this, I'm not sure how much
time I will be able to spend on this, given that the sysmocom customer
priority is typically not the development of automatic test suites. But
I'll try my best.
If anyone wants to join: TITAN is just one "apt-get install eclipse-titan"
away!
== What's not good to test with TTCN-3 / TITAN:
While the existing infrastructure I've built is very suitable to test
low-level protocol topics, I think it's not particularly
useful/applicable in other areas:
* SI REST OCTETS and GPRS RLC/MAC, as they're defined in CSN.1 and TITAN
has no CSN.1 encoder/decoder. For easy mathing/generation of those
messages, we'd have to find something like an external
CSN.1-to-ASN.1-BER or CSN.1-to-XML converter which we could use. I'd
be happy to consider paying a reasonable amount for a proprietary
product, just as long as we are sure that it will provde the kind of
productivity that's useful here.
* full end-to-end tests with actual mobile phones, RF and BTSs in the
picture. We already have osmo-gsm-tester for this, and Titan with its
powerful type language, templates and matching capabilities doesn't
provide all that much benefit here over other languages/approaches
* higher-layer protocol testing. We do have OsmocomBB as a L2 + L3
MS-side implementation, and I don't think it makes sense to replicate
this. Even while the message encoding/decoding is super easy with
TITAN, we still have all the related state machines. So I'd rather
extend OsmocomBB where needed for automatic test cases than reinvent
the wheel here.
Regards,
Harald
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
Hi.
Just a heads-up on recent changes related to Osmocom releases.
In short: read about it at
https://osmocom.org/projects/cellular-infrastructure/wiki/Make_a_new_release
Somewhat longer explanation:
In attempt to simplify and structure release process (see
https://projects.osmocom.org/issues/1861 ) I've added helper script (osmo-release.mk,
installed by recent libosmocore-dev) which attempts to automate as much as possible:
* treat library and non-library projects differently
* autogenerate debian/changelog
* automatically bump version
* cleanup TODO-RELEASE
* checks for missing *_LIBVERSION update
* automatically commit, sign and tag the result
Of course it might fail to do the right thing for multitude of reasons (incorrectly
filled TODO-RELEASE, missing tags, missing git-version-gen helper, weird versions in
debian/changelog etc) so nothing is pushed automatically. You're expected to inspect
the release commit and push it manually (don't forget to push the tag too!).
In general, it works: made libosmo-abis 0.4.0 and osmo-bts 0.6.0 releases using it.
The patches to make it available to other *osmo* projects are under review.
Now, if you've read that far, you must be really interested in making release for
some Osmocom project (or just really bored :) Either way, please do any of the following:
* read the wiki page linked above and see if anything unclear/poorly written/needs
corrections
* suggest further updates to release policy described in there
* review corresponding patches in gerrit
* suggest further automation step (ideally, as a patch to osmo-release.mk ;)
* try "make release" for one of the projects not mentioned above (if you have
sufficient rights to push tags to gerrit)
* have a look at recent libosmo-abis (>=0.4.0) and osmo-bts (>=0.6.0) and see if
anything is broken
N. B: release helper expects version to follow http://semver.org/ - this has been
raised in ML and no objections followed. Please adhere to it for all future versions.
--
Max Suraev <msuraev(a)sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte
I rebased the recent openbsc.git patches to osmo-msc.git and am submitting
all of them on gerrit, to make sure each passes the tests. Since they are
already accepted on openbsc.git, I'll also directly +2 and merge them.
We will be re-using the same Change-Ids from openbsc.git, and I checked:
if multiple commits match the same change-id, a search on gerrit will
simply list all matches, so we're not losing any information.
https://gerrit.osmocom.org/#/q/Ie7cdf16232181d4b8093e61f2d8a3faed9010d4f
~N
Dear all,
I had to spend a large portion of my spare time last night in order to
add a new jenkins build slave in a way that it works. What should have
been very simple by following the wiki at
https://osmocom.org/projects/osmocom-servers/wiki/Jenkins_Node_Setup
turned out into a long process of failures that still continue, see
http://jenkins.osmocom.org/jenkins/job/osmo-bts-gerrit/1325/
I'm really not happy about the lack of discipline in keeping the documentation
in sync - or even bothering to state in the wiki "FIXME, some stuff
needs to be done which is not documented". Having very incomplete
documentation that suggests something is a very easy and
straight-forward task is possibly worse than having no documentation,
at which point the reader/user is clear about this being a
time-consuming procedure that involved manual recreation of a manual
setup.
We do now have "build-2.osmocom.org", which is a Ryzen 1700X eight-core
CPU with 64GB RAM. It natively runs Debian 9 (stretch) and has a
Debian8 build slave inside a lxc container.
I've updated the wiki page, but I don't think "copy over the ~/docker directory
whose contents is not under revision control and then run ./update.sh" is
all that good an idea either.
Adding more build slaves to a jenkins setup should be the most natural and
normal thing to do. After all, the entire system is designed to scale out
by adding more build slaves.
I think the right process here would be to have a script that generates
the build slave[s]. My preference would be to have a lxc template for
it, so any new build host simply needs to lxc-create from that template.
Is anyone willing to contribute in that area?
Regards,
Harald
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
Hello openbsc community!
We are encountering the ff issue: When using basic phones, we have observed
that the timestamps on the SMS are in UTC, even if we changed the linux
system timezone to a different one. Also, if the basic phone will do an
auto-update of date and time from the network, it will use the time in UTC.
Do you have suggestions on how to change the network timezone configuration
for the osmo-nitb? There is a timezone command mentioned in the osmo-nitb
vty reference manual (timezone <-19-19> (0|15|30|45), ex. timezone 8 0 ).
If we put that config under the BTS level (in osmo-nitb.cfg), osmo-nitb
will complain that there is a config error. If we put that under the
network level, osmo-nitb does not complain, but it doesn't seem to work as
expected.
Let me know if addtl info is needed. Thank you in advance!
Trying to get the newest 2G+3G developments thru the test suites (including the
vty ones), I face a problem with this VTY definition from libosmo-sccp:
routing-key RCONTEXT DPC [si (aal2|bicc|b-isup|h248|isup|sat-isup|sccp|tup)] [ssn SSN]
It turns out the square braces indicating optional parameters cannot contain
spaces.
To test, I created
foo [a] [b]
which works as
OsmoMSC(config-msc)# foo ?
[a] a
OsmoMSC(config-msc)# foo b
% Unknown command.
OsmoMSC(config-msc)# foo a
ok
OsmoMSC(config-msc)# foo a ?
[b] b
OsmoMSC(config-msc)# foo a b
ok
So far so good, but with:
foo [a AA] [b]
I get
OsmoMSC(config-msc)# foo ?
[a a
OsmoMSC(config-msc)# foo a
% There is no matched command.
OsmoMSC(config-msc)# foo a val
% Unknown command.
The way this would work is
foo [a] [AA] [b]
and means that I can issue either 'foo', 'foo a', 'foo a val' or 'foo a val b'.
Not that helpful really.
With above command
routing-key RCONTEXT DPC [si (aal2|bicc|b-isup|h248|isup|sat-isup|sccp|tup)] [ssn SSN]
it seems to me it is intended as optionally providing none, si or both si and ssn?
I guess we need separate command definitions:
routing-key RCONTEXT DPC
routing-key RCONTEXT DPC si (aal2|bicc|b-isup|h248|isup|sat-isup|sccp|tup)
routing-key RCONTEXT DPC si (aal2|bicc|b-isup|h248|isup|sat-isup|sccp|tup) ssn SSN
Does that make sense?
Until we fix it, the vty tests will not be able to match up the vty doc
parameters and the python tests will fail.
I grepped for all square brace vty definitions we have; the only ones attempting to include multiple args in square braces are in osmo_ss7_vty.c:
./libosmo-sccp/src/osmo_ss7_vty.c-265- "update route POINT_CODE MASK linkset LS_NAME [priority PRIO] [qos-class (CLASS|default)]",
./libosmo-sccp/src/osmo_ss7_vty.c-781- "routing-key RCONTEXT DPC [si (aal2|bicc|b-isup|h248|isup|sat-isup|sccp|tup)] [ssn SSN]}",
They aren't usable.
(the 'update route...' is part of the _sg vty command set and thus not caught
by osmo-msc or -bsc vty tests; osmo-stp has no vty tests so far, AFAICT)
These appear to be all square brace vty definitions, most enclose only a single
element and are fine:
▶ grep -n '\<DEFUN\>(' -A1 $(find . -name "*.c") | grep '[[]'
./osmo-sgsn/src/gprs/sgsn_vty.c-534- "show mm-context tlli HEX [pdp]",
./osmo-sgsn/src/gprs/sgsn_vty.c-553- "show mm-context imsi IMSI [pdp]",
./osmo-sgsn/src/gprs/sgsn_vty.c-570- "show mm-context all [pdp]",
./osmo-sgsn/src/gprs/gb_proxy_vty.c:461:DEFUN(show_gbproxy, show_gbproxy_cmd, "show gbproxy [stats]",
./osmo-sgsn/src/gprs/gb_proxy_vty.c-552- "delete-gbproxy-peer <0-65534> (only-bvc|only-nsvc|all) [dry-run]",
./osmo-bsc/src/osmo-bsc/osmo_bsc_vty.c-63- "msc [<0-1000>]", "Configure MSC details\n" "MSC connection to configure\n")
./osmo-bsc/src/libbsc/bsc_vty.c:319:DEFUN(show_bts, show_bts_cmd, "show bts [<0-255>]",
./osmo-bsc/src/libbsc/bsc_vty.c:1629:DEFUN(cfg_bts_dtxu, cfg_bts_dtxu_cmd, "dtx uplink [force]",
./osmo-bsc/src/libbsc/bsc_vty.c-3947- "bts <0-255> trx <0-255> timeslot <0-7> sub-slot <0-7> (activate|deactivate) (hr|fr|efr|amr) [<0-7>]",
./osmo-mgw/src/libosmo-legacy-mgcp/mgcp_vty.c-229- "show mgcp [stats]",
./libosmo-sccp/src/osmo_ss7_vty.c-110- "point-code format <1-24> [<1-23>] [<1-22>]",
./libosmo-sccp/src/osmo_ss7_vty.c-265- "update route POINT_CODE MASK linkset LS_NAME [priority PRIO] [qos-class (CLASS|default)]",
./libosmo-sccp/src/osmo_ss7_vty.c-781- "routing-key RCONTEXT DPC [si (aal2|bicc|b-isup|h248|isup|sat-isup|sccp|tup)] [ssn SSN]}",
./libosmo-sccp/examples/sccp_test_vty.c-39- "connect-req <0-16777216> [DATA]",
./libosmo-sccp/examples/sccp_test_vty.c-53- "connect-resp <0-16777216> [DATA]",
./libosmocore/src/gb/gprs_ns_vty.c:216:DEFUN(show_nse, show_nse_cmd, "show ns (nsei|nsvc) <0-65535> [stats]",
./libosmocore/src/gb/gprs_bssgp_vty.c:153:DEFUN(show_bvc, show_bvc_cmd, "show bssgp nsei <0-65535> [stats]",
./libosmocore/src/vty/command.c-2965- "no service terminal-length [<0-512>]",
./libosmocore/src/vty/logging_vty.c-520- "log gsmtap [HOSTNAME]",
~N
I just added a new repository: osmo-dev.git.
For a long time I have been using hacked-up tools to help me in daily
Osmocom development. A recent addition was a generator for a top-level
makefile that allows one-line do-what-I-mean rebuild.
That's the point where I thought it made sense to share, in case anyone
else is interested in this kind of convenience.
The name is taken from osmo-ci. osmo-ci is for continuous integation on
our jenkins, osmo-dev is for development at home. I considered adding to
osmo-ci, but the aims are actually different from osmo-ci.
There are two README files, one explaining how the top-level makefile
works, one in src/ explains my little git robots that are also included.
The future will bring more *.deps and *.opts configs...
osmo-dev is on gerrit, feel free to contribute / test / fix!
If you don't agree with the naming and separate repos decision, I'd be
fine to change that...
~N
Hi all,
Yesterday and today I've been working to get the test suites for M3UA,
SUA and GGSN(GTP) integrated into jenkins.osmocom.org.
Some more background can be found at
http://laforge.gnumonks.org/blog/20170820-osmocom-testsuites/ with links
to the respective jenkins jobs and git repositories of test cases and
Dockerfiles + Makefiles.
It's important to note that those tests are neither a competition nor a
replacement for both the unit tests as well as osmo-gsm-tester. I'm not
quite sure how to call them, maybe "connected tests" or the like, as
they interact with our running network elements over the various
protocols/interfaces.
The parts that I'm not entirely sure about yet is:
1) does it really make sense to build docker images for both tester and IUT
(osmo-stp in cases of M3UA/SUA tests, openggsn in case of ggsn tests)?
Running both in docker containers attached to the same (internal,
non-routed/public) network allows the processes to talk to each other
without interfering the build host networking in any way, so it
seemed like a convenient choice.
Whether or not it makes sense to update both the tester and IUT
container in a single jenkins job - I'm not sure. Any ideas?
2) trigger. So far I went for the nightly trigger at ~ 4am GMT. This
is sufficient for now, but it could of course be triggered by polling
git of both the tester and the IUT.
3) version information. I would like to get proper version information
into jenkins, so we don't just know "nightly test on day X" but see
on the jenkins Web UI which particular git version of IUT (and
tester) was tested. Any help appreciated!
4) archiving logs. It would be great to keep the logs of not only the
last test execution around. Sure, jenkins has stdout (of the tester)
and junit-xml (of the tester). But things like the log file
generated by the IUT is currently only present for the last test
execution. Any help appreciated!
I'm currently wokring on docker images for osmo-sgsn, osmo-nitb,
osmo-bts-virt and virtphy, so we can extend above-mentioned setup
to cover also the 'sysinfo + nitb vty' as well as the lapdm and pcu
tests that I wrote in TTCN-3.
After that, I plan to direct my attention more to the development of
more test casese in those respective test suites, rather than optimizing
the Jenkins integration.
Any help both with more test cases as well as jenkins would be much
appreciated. I think the hard part has been creating the entire
toolchain and access to the various interfaces (Gp, Gb, Um, VTY, ...)
and now it should be rather easy to write more tests.
Regards,
Harald
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
