On Thu, Nov 15, 2018 at 04:23:55PM +0100, Neels Hofmeyr wrote: I actually hacked up one of those borderline-insane py scripts to mangle our C code into big-endian-reversed structs. See https://gerrit.osmocom.org/#/c/libosmocore/+/11786 for the script and https://gerrit.osmocom.org/#/c/libosmocore/+/11787 for what it did to libosmocore The script implementation itself is mad and convoluted, of course. How could it be different when handling and mangling C code. It works by looking at all packed structs that have sub-byte ints, and composes a big-endian counterpart. If there already is a big-endian ifdef, it strips it away and takes the little-endian part as authoritative. Non-packed structs are ignored, because we sometimes have weird boolean fields in the form of 'int flag:1;' which don't add up to full bytes. So, we can have auto-generated big-endian structs, and we can also verify in gerrit that they are correct (by runnning the script over the files and erroring if there are any local modifications; not implemented yet). But we get once-off cosmetic changes, to reach a state where running the struct_endianess.py script over the code results in zero changes. A simple example of what the script does is diff --git a/include/osmocom/gprs/protocol/gsm_04_60.h b/include/osmocom/gprs/protocol/gsm_04_60.h index 96e9ab78..5d5fca9a 100644 --- a/include/osmocom/gprs/protocol/gsm_04_60.h +++ b/include/osmocom/gprs/protocol/gsm_04_60.h @@ -7,10 +7,12 @@ #pragma once #include <stdint.h> +#include <osmocom/core/endian.h> #if OSMO_IS_LITTLE_ENDIAN == 1 /* TS 04.60 10.3a.4.1.1 */ struct gprs_rlc_ul_header_egprs_1 { +#if OSMO_IS_LITTLE_ENDIAN uint8_t r:1, si:1, cv:4, @@ -26,10 +28,20 @@ struct gprs_rlc_ul_header_egprs_1 { spare_hi:1; uint8_t spare_lo:6, dummy:2; +#elif OSMO_IS_BIG_ENDIAN +/* auto-generated from the little endian part above (libosmocore/contrib/struct_endianess.py) */ + uint8_t tfi_hi:2, cv:4, si:1, r:1; + uint8_t bsn1_hi:5, tfi_lo:3; + uint8_t bsn2_hi:2, bsn1_lo:6; + uint8_t bsn2_lo:8; + uint8_t spare_hi:1, pi:1, rsb:1, cps:5; + uint8_t dummy:2, spare_lo:6; +#endif } __attribute__ ((packed)); A more complex example is: Currently struct amr_hdr looks like this, manual big-endian stuff: struct amr_hdr { #if OSMO_IS_BIG_ENDIAN /* Payload Header */ uint8_t cmr:4, /* Codec Mode Request */ pad1:4; /* Table of Contents */ uint8_t f:1, /* followed by another speech frame? */ ft:4, /* coding mode */ q:1, /* OK (not damaged) at origin? */ pad2:2; #elif OSMO_IS_LITTLE_ENDIAN /* Payload Header */ uint8_t pad1:4, cmr:4; /* Table of Contents */ uint8_t pad2:2, q:1, /* OK (not damaged) at origin? */ ft:4, /* coding mode */ f:1; /* followed by another speech frame? */ #endif } __attribute__((packed)); After the script has done its mangling, it will look like this -- the little endian part is unchanged, and the big endian stuff is completely autogenerated: struct amr_hdr { #if OSMO_IS_LITTLE_ENDIAN /* Payload Header */ uint8_t pad1:4, cmr:4; /* Table of Contents */ uint8_t pad2:2, q:1, /* OK (not damaged) at origin? */ ft:4, /* coding mode */ f:1; /* followed by another speech frame? */ #elif OSMO_IS_BIG_ENDIAN /* auto-generated from the little endian part above (libosmocore/contrib/struct_endianness.py) */ uint8_t cmr:4, pad1:4; uint8_t f:1, ft:4, q:1, pad2:2; #endif } __attribute__((packed)); Notice that little endian is on top now, the big-endian part features a comment mentioning the script, and has different formatting. I intuitively added the script in libosmocore/contrib/ and not in osmo-ci, ymmv. I still think libosmocore-specific struct mangling belongs in libosmocore, and not in osmo-ci. (The value string verification script is in osmo-ci, but a programmer wanting to auto-generate a big-endian struct before submitting a patch would prefer to have the script there already, without having to download osmo-ci first). I have pushed a lot of neels/big_endian branches, the libosmocore one is on gerrit already, let's see what the review gets me before submitting the others as well: http://git.osmocom.org/libosmocore/commit/?h=neels/big_endian&id=8670bd… http://git.osmocom.org/libosmo-netif/commit/?h=neels/big_endian&id=3c21… http://git.osmocom.org/libosmo-sccp/commit/?h=neels/big_endian&id=6470e… http://git.osmocom.org/libosmo-abis/commit/?h=neels/big_endian&id=ca3e6… http://git.osmocom.org/osmo-pcu/commit/?h=neels/big_endian&id=7a99dcaef… http://git.osmocom.org/osmo-sgsn/commit/?h=neels/big_endian&id=f1620286… ~N