On jenkins and opensuse, we have builds where we cannot easily analyse the testsuite.dir after a build failed. So we only see which tests failed and not why.
On jenkins I have in most places added a script invocation that prints all testsuite.dir/*/*.log in case of build failure, so we can see it in the jenkins log right away.
On opensuse.org, IIUC, the 'make check' is run by dh or something, so it's not trivial (to me) to make it output the test logs in case of failure.
I'm thinking now, don't we always want to see these failures anyway? We could include in our Makefile.am a tweak to print the test logs in case of testsuite failure.
Sometimes this output can become fairly large though; e.g. when a long test fails early and prints "nothing", the test log shows a huge diff with all expected output as minus-lines.
My main goal right now is to see why/how the libosmo-netif OBS build failed (osmux test fails) without further local reproduction effort. This situation will certainly repeat over and over, for all of the other projects as well.
What do you think? Can we get that easily?
~N
On 25 Jan 2017, at 12:23, Neels Hofmeyr nhofmeyr@sysmocom.de wrote:
On jenkins and opensuse, we have builds where we cannot easily analyse the testsuite.dir after a build failed. So we only see which tests failed and not why.
On jenkins I have in most places added a script invocation that prints all testsuite.dir/*/*.log in case of build failure, so we can see it in the jenkins log right away.
dh_override_auto_test: make check || (cat ... && false)?
into the debian/rules or such?
My main goal right now is to see why/how the libosmo-netif OBS build failed (osmux test fails) without further local reproduction effort. This situation will certainly repeat over and over, for all of the other projects as well.
most likely timing... the test is using realtime... we should consider mocking gettimeofday in some way?
holger
On Wed, Jan 25, 2017 at 01:14:40PM +0100, Holger Freyther wrote:
On 25 Jan 2017, at 12:23, Neels Hofmeyr nhofmeyr@sysmocom.de wrote:
On jenkins and opensuse, we have builds where we cannot easily analyse the testsuite.dir after a build failed. So we only see which tests failed and not why.
On jenkins I have in most places added a script invocation that prints all testsuite.dir/*/*.log in case of build failure, so we can see it in the jenkins log right away.
dh_override_auto_test: make check || (cat ... && false)?
into the debian/rules or such?
(and this in all of our source trees) Thanks! Now someone has to still write the patches and test them...
most likely timing... the test is using realtime... we should consider mocking gettimeofday in some way?
Related: https://git.osmocom.org/libosmocore/commit/?id=8e2f7e87f4d854e697c40545326a1... and its first and so far only caller https://git.osmocom.org/libosmocore/commit/?id=7b4d727ec5debc3d057686a0a3f0f... from http://osmocom.org/issues/1809.
~N
-
Holger Freyther -
Neels Hofmeyr