TTCN-3 test suite for OsmoBTS

[Harald Welte]

Hi all,

today I've docker-ized the latest TTCN-3 test suite I've been developing over
the last week or so.  They're now running nightly in our Jenkins, you can see
the results analyzer at
https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-bts-test/test_results_analyzer/

(don't forget the "Expand All" button)

Out of all the test suites I've written in recent months, the OsmoBTS
test suite is what I'm particularly happy about.  While of course not
achieving 100% coverage it actually tests a lot of things, including
various cases that would be very hard to impossible to achieve in manual
testing.o

A larger number of tests is also relate to the BTS<->PCU socket
interface, where [almost] each of the message types occurring in Rx/Tx
is triggered + tested accordingly.

The system information scheduling test cases were working for me 2 days
ago, I'm not sure what I did wrong to break them again, I'll
investigate.

In terms of important needed OsmoBTS tests, I see:
* verification of PCU -> Um for all different coding schemes
  (requires PDCH support in trxcon first, or fall-back to virt_phy
  instead)
* testing of timing advance loop
* testing of uplink power control loop (needs fake_trx extension)
* testing of voice channels in all codec modes
* memory leak related testing

It was a loooong week full of late night and early morning shifts, but
I'm happy that progress was possible in a rather short amount of time.

Thanks to Vadim "fixeria" for his work on trxcon + fake_trx, which means
we can test a fully unmodified osmo-bts-trx ("the real thing"), rather
than osmo-bts-virtual, which is a BTS model that doesn't exist this way
in reality.

The nice part is that the test cases only use L1CTL on one hand side and
RSL/PCUIF/VTY/CTRL on the other side, and as such they should in theory
work equally with the followign setups:

a) TTCN3 -L1CTL-> trxcon -> fake_trx -> osmo-bts-trx -> TTCN3 (current)
b) TTCN3 -L1CTL-> virt_phy osmo-bts-virtual -> TTCN3
c) TTCN3 -L1CTL-> OsmocomBB-firmware -RF-> osmo-bts-{sysmo,lc15,octphy} -> TTCN3

Only 'a)' is currently used, but it would definitely be interesting to
be able to use the exact same test cases (minus TA/power-control loop
test) with real hardware and the different PHYs.

Any contributions are as always very much welcome.  With all the
existing test cases out there and the dockerized setup, it should be
super easy to reproduce the test results on your local machine, and to
copy+paste+edit an existing test to extend test coverage even further.

The development of the test suite has already resulted in dozens (!) of
bugs being discovered some of them quite severe.  Check the increased
activity since february 22nd at http://osmocom.org/projects/osmobts/issues?utf8=%E2%9C%93&set_filter=1&f%5B%5D=&c%5B%5D=tracker&c%5B%5D=status&c%5B%5D=priority&c%5B%5D=subject&c%5B%5D=assigned_to&c%5B%5D=updated_on&group_by=&t%5B%5D= for more information

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)