Hi Harald,
they are not yet listed on the index, but you should be able to clone them via git://git.osmocom.org/nplab/m3ua-testtool and git://git.osmocom.org/nplab/sua-testtool already. I just tried, works for me [tm].
Thanks, works for me too. Now I can see the test output, although tests are failing. Probably "culprit" is line 29 in test-m3ua.sh:
Line29 => $STP_DIR/osmo-stp -c $STP_CONFIG &
Error=> ./test-m3ua.sh: 29: ./test-m3ua.sh: ../..//stp/osmo-stp: not found
Can you please help me to fix this issue?
ok. For the m3ua/sua testtool, the 'runm3uatest' program already contains some custom colorized output, you might simply be able to patch that to generate something useful to jenkins?
That was my first approach, but I didn't manage to pipe the colorized output to a file. Any ideas?
So I (temporarily) changed following line in run-all-sgp-tests.sh:
$ (./runm3uatest -d . -t $timeout $testcase > /dev/tty) >& /dev/null
to
$ ./runm3uatest -d . -t $timeout $testcase
Then executing
$ ./run-in-ns.sh ./test-m3ua.sh 2>&1 | tee m3ua-test-report.plain
gives a very verbose output, which is saved to a "m3ua-test-report.plain" for XML generation after all tests have been run. At this point I got the feeling that a more verbose output could also be more helpful in a Jenkins job than prompting the summary, which is probably more suited for builds run by a developer.(?)
What is your general opinion about the verboseness of the Jenkins log and the XML report?
Here is how a XML report looks like when we would simply evaluate the "test output summary":
<testsuites> <testsuite name="" tests="32" failures="32" errors="32"/> <testcase name="m3ua-sgp-aspsm-v-002"/> <testcase name="m3ua-sgp-aspsm-v-001"/> </testsuite> </testsuites>
This XML snippet represent the current state after my hands on yesterday:
<testsuites> <testsuite name="" tests="32" failures="32" errors="32"/> <testcase name="m3ua-sgp-aspsm-v-001" time="51011250"/> <error type=" In procedure sctp-sendmsg">
Some deprecated features have been used. Set the environment variable GUILE_WARN_DEPRECATED to "detailed" and rerun the program to get more information. Set it to "no" to suppress this message. FAILED </error> <testcase name="m3ua-sgp-aspsm-v-003" time="58336593"/> ...... </testsuite> </testsuites>
I personally always prefer when Jenkins provides some information about the failure rather than only pointing out that something failed and referring to the console log.
Something else, can I may request a failing test or information how to change one to let it fail? :) Without such test case it's hard to test the script, which shall produce the XML report.
Regards, André