Hi Holger,
On Sun, Nov 13, 2011 at 01:30:16AM +0100, Holger Hans Peter Freyther wrote:
Hi Pablo, all
I have pushed GNU autotest[1] integration of libosmocore into the
zecke/gnu-autotest branch and invoking make check will execute the testsuite.
The output looks like this:
## ------------------------------------- ##
## libosmocore 0.4.0.10-c015 test suite. ##
## ------------------------------------- ##
Regression tests.
1: bits ok
2: msgfile ok
3: sms ok
4: smscb ok
5: timer FAILED (testsuite.at:38)
6: ussd FAILED (testsuite.at:44)
GNU autotest will execute an external application and then can check the exit
code, compare the stdout/stderr to a file. In this case the timer test fails
as the test itself is randomized and does not always provide the same output.
Interesting, never played with this autotest stuff so far.
Pablo if your time permits it would be nice if you
could:
- Provide a cli option to make the test have less iterations (to make
make check run faster)
Patch attached for this.
- Provide a cli option to produce a repeatable output
(e.g. by
omitting the expired output).
Can we tell the tool to compare stdout but to ignore stderr? If so, we
can display the repeatable output in stdout and the non-repeatable output
in stderr.
What do you think? In some ways I think that executing
the timer test as part
of our regression tests makes sense but maybe specially on a loaded machine
the test might be flaky...
Yes, with lots of timers, the expiration may not be done in time on
a loaded machine. If we can ignore the stderr output, we can put the
information about timers not expiring in time to stderr.