Deprecating support for big endian in Osmocom
Hi all, during recent patch review (https://gerrit.osmocom.org/c/osmo-trx/+/33737) the topic of continuing to maintain support for big endian machines has come up. While traditionally I've always been a strong proponent of writing portable code that can run also on big endian systems, it is not the year 2003 or 2008 anymore, and PowerPC (the main big endian platform) is dead by now, as is SPARC. Not just in newly-built processors, but also in existing and still operating machines, at least of the class that would run our code. So unless somebody objects with strong arguments, I'd propose to officially and explicitly drop supporting big endian systems from osmocom CNI projects. From what I can tell, this would primarily mean * drop the struct_endianness check from the commit verification * removing all our struct_endianness-generated or other code that explicitly adds big endian support * adding some kind of #warning or even #error to a common libosmocore header file if anyone tries to build on big endian This obviously doesn't mean we can abandon using [osmo_]{htonl,ntohl}, as network byte order is still network byte order. Regards, Harald -- - Harald Welte <laforge@gnumonks.org> https://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Hi, if we want to drop big endian support officially let that be it, but I would keep the struct_endianness check since it's not really a lot of work running it to generate the structs and it's only seldomly used. Moreover, I would for sure not spend time in *removing* big endian support. Maybe let's simply avoid spending extra time in adding specific big endian support in places where it would require some time to implement, and leave a FIXME/TODO commit instead, like the patch which originated this topic discussion. Regards, Pau -- - Pau Espin Pedrol <pespin@sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte
TLDR; I lean towards dropping BE support. From my personal, practical POV, I have no need ever of using a BE platform. It's true that running the struct_endianness.py is not much effort, but that script is not very intelligent and can only handle very specific cases (IIRC unions don't work, and the data type needs to be <= 8bit). I remember that recently there was a case of that script not working properly. So there is at leat *some* quirkiness that remains, even though we have this script. So my first thought is: yes, let's just drop this ugly stuff that clutters our source code, may complicate how we lay out our structs, and that literally no-one is using anywhere. Simplify, simplify. Another thought is that a good friend of mine likes to operate legacy hardware, and there I've often heard complaints that some upstream X just completely dropped perfectly good support for hardware Y because they falsely assumed no-one is using it. Now I'm not so sure if that friend of mine is literally the only person complaining about it, or if it is a bad thing in general? That would be my only point for keeping big endian support. My final thought is that I wonder how other projects handle this. Every C/C++ code is having this problem. I find it curious that the endianness of packed structs is not abstracted away by the *compiler*, where it probably should be? BTW, TIL where the term Endian originated -- despite it seeming so obvious in hindsight -- and if you don't know yet: https://en.wikipedia.org/wiki/Endianness#Etymology "The adjective endian has its origin in the writings of 18th century Anglo-Irish writer Jonathan Swift. In the 1726 novel Gulliver's Travels, he portrays the conflict between sects of Lilliputians divided into those breaking the shell of a boiled egg from the big end or from the little end..." ~N On Mon, Jul 17, 2023 at 08:49:02AM +0200, Harald Welte wrote:
Hi all,
during recent patch review (https://gerrit.osmocom.org/c/osmo-trx/+/33737) the topic of continuing to maintain support for big endian machines has come up.
While traditionally I've always been a strong proponent of writing portable code that can run also on big endian systems, it is not the year 2003 or 2008 anymore, and PowerPC (the main big endian platform) is dead by now, as is SPARC. Not just in newly-built processors, but also in existing and still operating machines, at least of the class that would run our code.
So unless somebody objects with strong arguments, I'd propose to officially and explicitly drop supporting big endian systems from osmocom CNI projects.
From what I can tell, this would primarily mean * drop the struct_endianness check from the commit verification * removing all our struct_endianness-generated or other code that explicitly adds big endian support * adding some kind of #warning or even #error to a common libosmocore header file if anyone tries to build on big endian
This obviously doesn't mean we can abandon using [osmo_]{htonl,ntohl}, as network byte order is still network byte order.
Regards, Harald -- - Harald Welte <laforge@gnumonks.org> https://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
-- - Neels Hofmeyr <nhofmeyr@sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschäftsführer / Managing Directors: Harald Welte
Hi Neels and others, On Mon, Jul 17, 2023 at 07:58:31PM +0200, Neels Hofmeyr wrote:
From my personal, practical POV, I have no need ever of using a BE platform.
For me its different: I have been using BE in the past and I may still use BE platforms at some point in time. But that's my personal retrocomputing intrest, and I don't need to run osmo* on it. And if I want, it is my personal choice to invest time into it, I don't want to burden other developers with it.
It's true that running the struct_endianness.py is not much effort, but that script is not very intelligent and can only handle very specific cases (IIRC unions don't work, and the data type needs to be <= 8bit). I remember that recently there was a case of that script not working properly. So there is at leat *some* quirkiness that remains, even though we have this script.
yes, we have that script, it doesn't handle all cases, and we have zero testing. So rather than providing the impression we have BE support, let's just drop it and be clear about it. If anyone actually shows up and wants to maintain it, we wouldn't mind related patches, I guess. Similar to the statement at the time we dropped FreeBSD support: If somebody wants to work on that and provide alternatives for the various linux-specific bits in libosmocore (timerfd, etc.) - fine.
Another thought is that a good friend of mine likes to operate legacy hardware, [...]
I am such a person myself, see above. But it is not the job of the average osmocom developer to care for the peculiar fetishes of 0.0001% of a user base that's already working in a niche industry.
Every C/C++ code is having this problem.
Well, every C/C++ code dealing with external binary data formats stored in a non-native endianess. So pure application code dealing only with textual protocols (HTTP, SMTP, ...) or data formats (XML, JSON) is unaffected. -- - Harald Welte <laforge@gnumonks.org> https://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Hi, On 17.07.23 08:49, Harald Welte wrote:
From what I can tell, this would primarily mean * drop the struct_endianness check from the commit verification * removing all our struct_endianness-generated or other code that explicitly adds big endian support * adding some kind of #warning or even #error to a common libosmocore header file if anyone tries to build on big endian
Created this issue to follow up: https://osmocom.org/issues/6122 Best regards, Oliver -- - Oliver Smith <osmith@sysmocom.de> https://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte
Just a thought - would this imply that the distributions might consider removing libosmo* because of build failures in some architectures? Rafael On 7/31/23 10:50, Oliver Smith wrote:
Hi,
On 17.07.23 08:49, Harald Welte wrote:
From what I can tell, this would primarily mean * drop the struct_endianness check from the commit verification * removing all our struct_endianness-generated or other code that explicitly adds big endian support * adding some kind of #warning or even #error to a common libosmocore header file if anyone tries to build on big endian
Created this issue to follow up: https://osmocom.org/issues/6122
Best regards, Oliver
On 31.07.23 12:00, Rafael Diniz wrote:
Just a thought - would this imply that the distributions might consider removing libosmo* because of build failures in some architectures?
Rafael
In my experience, distributions would just disable building for the now unsupported architectures and keep the packages for all other arches. -- - Oliver Smith <osmith@sysmocom.de> https://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte
participants (5)
-
Harald Welte -
Neels Hofmeyr -
Oliver Smith -
Pau Espin Pedrol -
Rafael Diniz