osmo-trx test setup & convolve_test failures on different machines

Hi everybody, as some of you may have noticed due to the generated noise in gerrit ml during last days, I have been adding some autotest setup to osmo-trx in order to have several tests being run during gerrit review. Since osmo-trx has some specific code built with NEON instructions, I also added some infrastructure to jenkin.sh to generate and use a prebuilt debian armhf rootfs so we can easily build and run tests using qemu-arm from our local workstations or from jenkins slaves, which will check that support for those architectures doesn't become broken. In order to set up the make check tests, I re-used existing tests and moved them to the tests/ directory. I also moved utils/convolvtest/main.c into "tests/Transceiver52M/convolve_test" [1] to become a test that checks "convolve_{real,complex}" APIs. I noticed, however, that this test prints slightly (or not that slightly) different outputs on different machine architectures, which of course make tests fail as it matches the exact values. So far the output is different in my x86_64 workstation, my i686 netbook (or i586 OBS host) and in my qemu-arm (armhf). As each machine uses different instructions sets, I indeed can understand that there appear small differences due to floating operations and rounding, but sometimes I can see relatively big difference between one implementation and the other one (eg 7.032490 vs 6.655972). I'm thinking about modifying the test to, instead of matching the output exactly, check that for each value in the vector it is similar to a value with a maximum epsilon distance from a reference vector. However, as I don't have any knowledge regarding the maths and theory behind the convolution codes, I must admit I am not sure which epsilon should I be using, or which should be the expected output for the test. It may perhaps make sense to have separate reference output for different architectures, I don't know. Can somebody shed some light on this topic? Any advise or hint is welcome. It would also be nice to have some tests for the "convert" API, which uses optimized instructions sets too. If somebody wants to provide those or any hint on how to implement them that's going to be handy. All the test infrastructure is merged now, only the patch adding the qemu part is not merged but already working in [2], I'll merge it tomorrow if nobody is angry with it. You can find related output for the test I was speaking of for different machine arch (x86_64, i686, arm) in [3] and [4]. [1]https://git.osmocom.org/osmo-trx/tree/tests/Transceiver52M/convolve_test.c [2] https://gerrit.osmocom.org/#/c/5763 [3] https://osmocom.org/issues/2826 [4] https://osmocom.org/issues/2828 -- - Pau Espin Pedrol <pespin(a)sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte