Hi all,

Yesterday and today I've been working to get the test suites for M3UA, SUA and GGSN(GTP) integrated into jenkins.osmocom.org.

Some more background can be found at http://laforge.gnumonks.org/blog/20170820-osmocom-testsuites/ with links to the respective jenkins jobs and git repositories of test cases and Dockerfiles + Makefiles.

It's important to note that those tests are neither a competition nor a replacement for both the unit tests as well as osmo-gsm-tester. I'm not quite sure how to call them, maybe "connected tests" or the like, as they interact with our running network elements over the various protocols/interfaces.

The parts that I'm not entirely sure about yet is:

1) does it really make sense to build docker images for both tester and IUT (osmo-stp in cases of M3UA/SUA tests, openggsn in case of ggsn tests)?

Running both in docker containers attached to the same (internal, non-routed/public) network allows the processes to talk to each other without interfering the build host networking in any way, so it seemed like a convenient choice.

Whether or not it makes sense to update both the tester and IUT container in a single jenkins job - I'm not sure. Any ideas?

2) trigger. So far I went for the nightly trigger at ~ 4am GMT. This is sufficient for now, but it could of course be triggered by polling git of both the tester and the IUT.

3) version information. I would like to get proper version information into jenkins, so we don't just know "nightly test on day X" but see on the jenkins Web UI which particular git version of IUT (and tester) was tested. Any help appreciated!

4) archiving logs. It would be great to keep the logs of not only the last test execution around. Sure, jenkins has stdout (of the tester) and junit-xml (of the tester). But things like the log file generated by the IUT is currently only present for the last test execution. Any help appreciated!

I'm currently wokring on docker images for osmo-sgsn, osmo-nitb, osmo-bts-virt and virtphy, so we can extend above-mentioned setup to cover also the 'sysinfo + nitb vty' as well as the lapdm and pcu tests that I wrote in TTCN-3.

After that, I plan to direct my attention more to the development of more test casese in those respective test suites, rather than optimizing the Jenkins integration.

Any help both with more test cases as well as jenkins would be much appreciated. I think the hard part has been creating the entire toolchain and access to the various interfaces (Gp, Gb, Um, VTY, ...) and now it should be rather easy to write more tests.

Regards, Harald