Hi Max and Harald,
As I mentioned above, my suggestion is to keep the source code "as portable as possible", which doesn't mean absolutely portable. I don't suggest to add other UNIX platforms (Darwin, Solaris, etc.) to our portability scope, I suggest to perform checks on the same platforms as we currently do, but additionally to have both Clang and GCC on both slaves.
Why Clang?
- it's default compiler on FreeBSD (recent distributions); - it has its own big community; - it has growing compatibility with GCC, see: en.wikipedia.org/wiki/Clang#Performance_and_GCC_compatibility
Why different versions?
- different distributions provide different versions available out of the box, i.e. some (such as FreeBSD) provide the latest version, while some other provide a bit dated versions; - some things may change in newer versions, e.g. from my recent experience: Clang 4.0 has support of the __builtin_cpu_supports built-in, while older versions don't.
I see the only disadvantage of this approach - longer total build time. So, if you guys support this idea, I could do all configuration and installation related work. Also, I'll try to contribute some computing power to Jenkins: there are some servers in my university, which power isn't used anyhow, so they mostly idle all the time.
Recently I decided to dig into FreeBSD's world, because one of my changes in Gerrit does build fine on Debian slave, but doesn't on FreeBSD.
Here I found the problem. Cite from GCC's documentation page x86-Built-in-Functions.html at gcc.gnu.org:
If you specify command-line switches such as -msse, the compiler could use the extended instruction sets even if the built-ins are not used explicitly in the program. For this reason, applications that perform run-time CPU detection must compile separate files for each supported architecture, using the appropriate flags. In particular, the file containing the CPU detection code should be compiled without these options.
Adding SIMD flags to a whole project breaks portability between different CPUs, so I added these flags for viterbi_sse.lo only. Now the problem seems solved.
With best regards, Vadim Yanitskiy.
Hi Vadim,
On Tue, May 09, 2017 at 04:09:22AM +0700, Vadim Yanitskiy wrote:
As I mentioned above, my suggestion is to keep the source code "as portable as possible", which doesn't mean absolutely portable.
I suggest to perform checks on the same platforms as we currently do, but additionally to have both Clang and GCC on both slaves. [...]
- different distributions provide different versions available out of the box, i.e. some (such as FreeBSD) provide the latest version, while some other provide a bit dated versions;
- some things may change in newer versions, e.g. from my recent experience: Clang 4.0 has support of the __builtin_cpu_supports built-in, while older versions don't.
I'm not quite sure if I follow your argument here. From my point of view, the importance is that we test build on those platforms/compilers that most of our users actually use. So I would rather argue for testing with (the gcc versions of) various recent releases of popular GNU/Linux distributions or on different CPUs such as ARM, than to introduce a compiler version that isn't even included in latest FreeBSD (11).
I see the only disadvantage of this approach - longer total build time. So, if you guys support this idea, I could do all configuration and installation related work.
Thanks. I guess the question is if people generally agree that it is useful. I'm not entirely convinced that we should do build testing on more compiler versions.
Also, as a compromise proposal: we don't necessarily have to do it as part of the testing of each and every commit. It might be sufficient to do this e.g. once as part of a daily/nightly job which sends e-mails to the author of the commit that broke?
Also, I'll try to contribute some computing power to Jenkins: there are some servers in my university, which power isn't used anyhow, so they mostly idle all the time.
Thanks for your offer. However, I'm not quite sure if we should do something like that without official permission. Paying for another rented server should generally not be a problem: An AMD Ryzen hexa-core 3.6GHz machine with 32GB RAM and SSD is available for 58.31 EUR per month.
Here I found the problem. [...]
Adding SIMD flags to a whole project breaks portability between different CPUs, so I added these flags for viterbi_sse.lo only. Now the problem seems solved.
Yes, that's basically the same error as osmo-trx did with using "-march=native": Using optimization flags of the build CPU and assuming that the CPU you later run on will be the exact same CPU type. That's really only useful on systems like Gentoo where everybody builds everything from source on the very system they run the code on.
Regards, Harald
Hi Harald,
I'm not quite sure if I follow your argument here. From my point of view, the importance is that we test build on those platforms/compilers that most of our users actually use. So I would rather argue for testing with (the gcc versions of) various recent releases of popular GNU/Linux distributions or on different CPUs such as ARM, than to introduce a compiler version that isn't even included in latest FreeBSD (11).
Thanks for response, your point of view makes sense.
Also, as a compromise proposal: we don't necessarily have to do it as part of the testing of each and every commit. It might be sufficient to do this e.g. once as part of a daily/nightly job which sends e-mails to the author of the commit that broke?
Great idea! Let's see, what other developers think about that...
With best regards, Vadim Yanitskiy.
-
Harald Welte -
Vadim Yanitskiy