TTCN-3 test suite for OsmoBTS

Hi all, today I've docker-ized the latest TTCN-3 test suite I've been developing over the last week or so. They're now running nightly in our Jenkins, you can see the results analyzer at https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-bts-test/test_resu… (don't forget the "Expand All" button) Out of all the test suites I've written in recent months, the OsmoBTS test suite is what I'm particularly happy about. While of course not achieving 100% coverage it actually tests a lot of things, including various cases that would be very hard to impossible to achieve in manual testing.o A larger number of tests is also relate to the BTS<->PCU socket interface, where [almost] each of the message types occurring in Rx/Tx is triggered + tested accordingly. The system information scheduling test cases were working for me 2 days ago, I'm not sure what I did wrong to break them again, I'll investigate. In terms of important needed OsmoBTS tests, I see: * verification of PCU -> Um for all different coding schemes (requires PDCH support in trxcon first, or fall-back to virt_phy instead) * testing of timing advance loop * testing of uplink power control loop (needs fake_trx extension) * testing of voice channels in all codec modes * memory leak related testing It was a loooong week full of late night and early morning shifts, but I'm happy that progress was possible in a rather short amount of time. Thanks to Vadim "fixeria" for his work on trxcon + fake_trx, which means we can test a fully unmodified osmo-bts-trx ("the real thing"), rather than osmo-bts-virtual, which is a BTS model that doesn't exist this way in reality. The nice part is that the test cases only use L1CTL on one hand side and RSL/PCUIF/VTY/CTRL on the other side, and as such they should in theory work equally with the followign setups: a) TTCN3 -L1CTL-> trxcon -> fake_trx -> osmo-bts-trx -> TTCN3 (current) b) TTCN3 -L1CTL-> virt_phy osmo-bts-virtual -> TTCN3 c) TTCN3 -L1CTL-> OsmocomBB-firmware -RF-> osmo-bts-{sysmo,lc15,octphy} -> TTCN3 Only 'a)' is currently used, but it would definitely be interesting to be able to use the exact same test cases (minus TA/power-control loop test) with real hardware and the different PHYs. Any contributions are as always very much welcome. With all the existing test cases out there and the dockerized setup, it should be super easy to reproduce the test results on your local machine, and to copy+paste+edit an existing test to extend test coverage even further. The development of the test suite has already resulted in dozens (!) of bugs being discovered some of them quite severe. Check the increased activity since february 22nd at http://osmocom.org/projects/osmobts/issues?utf8=%E2%9C%93&set_filter=1&… for more information Regards, Harald -- - Harald Welte &lt;laforge(a)gnumonks.org&gt; http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)