Hi Andre,
On Tue, May 23, 2017 at 07:20:08PM +0200, André Boddenberg wrote:
Long story short, here's a Dockerfile (attached) capable of running tests on top of a compiled (via ./contrib/jenkins.sh) libosmo-sctp repository. Only one container is used and IPs can be created on loopback interface in fact of using "--add-cap=NET_ADMIN", which imo is a mighty feature - able of pushing docker beyond its proposed use cases.
After building the Dockerfile execute following "line" inside compiled libosmo-sctp repo:
docker run -it --rm --cap-add=NET_ADMIN \ -v $(pwd):/workdir/libosmo-sctp \ <NAME_OF_IMAGE> ./test-m3ua.sh 2>&1 | tee result.plain
# docker run -it --rm --cap-add=NET_ADMIN -v $(pwd):/workdir/libosmo-sctp 16e576fa31f9 ./test-m3ua.sh 2>&1 | tee result.plain Test m3ua-sgp-aspsm-v-001 /workdir/libosmo-sctp/stp/.libs/osmo-stp: error while loading shared libraries: libosmovty.so.3: cannot open shared object file: No such file or directory
I guess there's a ldconfig missing somewhere? How are you actually building the code if there's no libosmovty installed from source in your Dockerfile?
BTW: should we add your Docker file to the collection of Dockerfiles in http://git.osmocom.org/docker-playground/ ?
Does the container work for you, too?
unfortunately not, see above :/
What do you think about such approach in general?
I'd like to play with it, but first have to get it to work.
In general, I don't really care too much *how* the tests are excecuted, just as long as it is based on technology that works well and is present on our buildhosts/jenkins.
What was wrong with my simple 'unshare' based script?
A test job [2] has been created showing how such a "publishing" looks like. One simply clicks on "Latest Test Result" and sees a list of all failures.
looks great!
Each failure can be expanded to see the "Stacktrace:/Backtrace:". Furthermore, the first line of such a "Stacktrace" specifies whether it is a "pure" failure or a timeout.
interesting
In addition I'd like to mention the possibility of saving duration of each test case inside XML report. This could enable performance statistics!?
yes, but I don't think it is of much use, as the build hosts might be doing all kinds of other things in parallel?
What do you think about such approach of creating and publishing XML reports in general?
Looks fine to me. Maybe there's even a simple/straightforward way to do that for the gnu autotest based unit tests we have in the various osmocom projects?
Please note that create_xml_report.sh still needs a clean up, but at the same time I thought it's already worth sharing my hands-on results. Furthermore, feel free to request more details on specific topics e.g. JUnit XML reports, Jenkins, Docker (tried to keep mail crisp).
wouldn't it make sense to simply replace the 'runm3uatest' executor and the shell script that repeatedly executes it with something that generates the required XML directly? It could be a python script directly, doing away with the C program that does nothing but executing a guile interpreter ;)