osmo-trx test setup & convolve_test failures on different machines

Pau Espin Pedrol pespin at sysmocom.de
Mon Jan 15 19:08:43 UTC 2018


Hi everybody,

as some of you may have noticed due to the generated noise in gerrit ml 
during last days, I have been adding some autotest setup to osmo-trx in 
order to have several tests being run during gerrit review. Since 
osmo-trx has some specific code built with NEON instructions, I also 
added some infrastructure to jenkin.sh to generate and use a prebuilt 
debian armhf rootfs so we can easily build and run tests using qemu-arm 
from our local workstations or from jenkins slaves, which will check 
that support for those architectures doesn't become broken.

In order to set up the make check tests, I re-used existing tests and 
moved them to the tests/ directory. I also moved 
utils/convolvtest/main.c into "tests/Transceiver52M/convolve_test" [1] 
to become a test that checks "convolve_{real,complex}" APIs.

I noticed, however, that this test prints slightly (or not that 
slightly) different outputs on different machine architectures, which of 
course make tests fail as it matches the exact values.
So far the output is different in my x86_64 workstation, my i686 netbook 
(or i586 OBS host) and in my qemu-arm (armhf).
As each machine uses different instructions sets, I indeed can 
understand that there appear small differences due to floating 
operations and rounding, but sometimes I can see relatively big 
difference between one implementation and the other one (eg 7.032490 vs 
6.655972).

I'm thinking about modifying the test to, instead of matching the output 
exactly, check that for each value in the vector it is similar to a 
value with a maximum epsilon distance from a reference vector. However, 
as I don't have any knowledge regarding the maths and theory behind the 
convolution codes, I must admit I am not sure which epsilon should I be 
using, or which should be the expected output for the test. It may 
perhaps make sense to have separate reference output for different 
architectures, I don't know. Can somebody shed some light on this topic? 
Any advise or hint is welcome.

It would also be nice to have some tests for the "convert" API, which 
uses optimized instructions sets too. If somebody wants to provide those 
or any hint on how to implement them that's going to be handy.

All the test infrastructure is merged now, only the patch adding the 
qemu part is not merged but already working in [2], I'll merge it 
tomorrow if nobody is angry with it. You can find related output for the 
test I was speaking of for different machine arch (x86_64, i686, arm) in 
[3] and [4].

[1]https://git.osmocom.org/osmo-trx/tree/tests/Transceiver52M/convolve_test.c
[2] https://gerrit.osmocom.org/#/c/5763
[3] https://osmocom.org/issues/2826
[4] https://osmocom.org/issues/2828

-- 
- Pau Espin Pedrol <pespin at sysmocom.de>         http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte


More information about the OpenBSC mailing list