Failing test cases (and builds) on other architectures in Debian

Neels Hofmeyr nhofmeyr at sysmocom.de
Fri Nov 16 00:09:44 UTC 2018


On Thu, Nov 15, 2018 at 04:23:55PM +0100, Neels Hofmeyr wrote:
> > Maybe the trick is to just start going over all structs in libosmocore
> 
> I thought about scripting it, because editing manually takes forever and is
> error prone.

I actually hacked up one of those borderline-insane py scripts to mangle our C
code into big-endian-reversed structs.

See
https://gerrit.osmocom.org/#/c/libosmocore/+/11786 for the script and
https://gerrit.osmocom.org/#/c/libosmocore/+/11787 for what it did to libosmocore

The script implementation itself is mad and convoluted, of course. How could it
be different when handling and mangling C code.

It works by looking at all packed structs that have sub-byte ints, and composes
a big-endian counterpart. If there already is a big-endian ifdef, it strips it
away and takes the little-endian part as authoritative.

Non-packed structs are ignored, because we sometimes have weird boolean fields
in the form of 'int flag:1;' which don't add up to full bytes.

So, we can have auto-generated big-endian structs, and we can also verify in
gerrit that they are correct (by runnning the script over the files and
erroring if there are any local modifications; not implemented yet).

But we get once-off cosmetic changes, to reach a state where running the
struct_endianess.py script over the code results in zero changes.

A simple example of what the script does is


diff --git a/include/osmocom/gprs/protocol/gsm_04_60.h b/include/osmocom/gprs/protocol/gsm_04_60.h
index 96e9ab78..5d5fca9a 100644
--- a/include/osmocom/gprs/protocol/gsm_04_60.h
+++ b/include/osmocom/gprs/protocol/gsm_04_60.h
@@ -7,10 +7,12 @@
 #pragma once

 #include <stdint.h>
+#include <osmocom/core/endian.h>

 #if OSMO_IS_LITTLE_ENDIAN == 1
 /* TS 04.60  10.3a.4.1.1 */
 struct gprs_rlc_ul_header_egprs_1 {
+#if OSMO_IS_LITTLE_ENDIAN
        uint8_t r:1,
                 si:1,
                 cv:4,
@@ -26,10 +28,20 @@ struct gprs_rlc_ul_header_egprs_1 {
                 spare_hi:1;
        uint8_t spare_lo:6,
                 dummy:2;
+#elif OSMO_IS_BIG_ENDIAN
+/* auto-generated from the little endian part above (libosmocore/contrib/struct_endianess.py) */
+       uint8_t tfi_hi:2, cv:4, si:1, r:1;
+       uint8_t bsn1_hi:5, tfi_lo:3;
+       uint8_t bsn2_hi:2, bsn1_lo:6;
+       uint8_t bsn2_lo:8;
+       uint8_t spare_hi:1, pi:1, rsb:1, cps:5;
+       uint8_t dummy:2, spare_lo:6;
+#endif
 } __attribute__ ((packed));




A more complex example is:

Currently struct amr_hdr looks like this, manual big-endian stuff:

struct amr_hdr {
#if OSMO_IS_BIG_ENDIAN
        /* Payload Header */
        uint8_t cmr:4,  /* Codec Mode Request */
                pad1:4;
        /* Table of Contents */
        uint8_t f:1,    /* followed by another speech frame? */
                ft:4,   /* coding mode */
                q:1,    /* OK (not damaged) at origin? */
                pad2:2;
#elif OSMO_IS_LITTLE_ENDIAN
        /* Payload Header */
        uint8_t pad1:4,
                cmr:4;
        /* Table of Contents */
        uint8_t pad2:2,
                q:1,    /* OK (not damaged) at origin? */
                ft:4,   /* coding mode */
                f:1;    /* followed by another speech frame? */
#endif
} __attribute__((packed));


After the script has done its mangling, it will look like this -- the little
endian part is unchanged, and the big endian stuff is completely autogenerated:

struct amr_hdr {
#if OSMO_IS_LITTLE_ENDIAN
        /* Payload Header */
        uint8_t pad1:4,
                cmr:4;
        /* Table of Contents */
        uint8_t pad2:2,
                q:1,    /* OK (not damaged) at origin? */
                ft:4,   /* coding mode */
                f:1;    /* followed by another speech frame? */
#elif OSMO_IS_BIG_ENDIAN
/* auto-generated from the little endian part above (libosmocore/contrib/struct_endianness.py) */
        uint8_t cmr:4, pad1:4;
        uint8_t f:1, ft:4, q:1, pad2:2;
#endif
} __attribute__((packed));


Notice that little endian is on top now, the big-endian part features a comment
mentioning the script, and has different formatting.


I intuitively added the script in libosmocore/contrib/ and not in osmo-ci,
ymmv. I still think libosmocore-specific struct mangling belongs in
libosmocore, and not in osmo-ci.
(The value string verification script is in osmo-ci, but a programmer wanting
to auto-generate a big-endian struct before submitting a patch would prefer to
have the script there already, without having to download osmo-ci first).

I have pushed a lot of neels/big_endian branches, the libosmocore one is on
gerrit already, let's see what the review gets me before submitting the others
as well:
http://git.osmocom.org/libosmocore/commit/?h=neels/big_endian&id=8670bd025a47d462a7b76a811e66dee42afacc1f
http://git.osmocom.org/libosmo-netif/commit/?h=neels/big_endian&id=3c21004bfba30054bf0069c5b4bc27447453178d
http://git.osmocom.org/libosmo-sccp/commit/?h=neels/big_endian&id=6470e9577137d3e674d5d433bf27b9543972a420
http://git.osmocom.org/libosmo-abis/commit/?h=neels/big_endian&id=ca3e6bbcfd52d84e2f7dad68f9c1c3c13611ebf7
http://git.osmocom.org/osmo-pcu/commit/?h=neels/big_endian&id=7a99dcaeff6553de107a416b89cf1249fb89dc01
http://git.osmocom.org/osmo-sgsn/commit/?h=neels/big_endian&id=f1620286371b2c463a5d77896d40282ddaf31b36

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20181116/7e28d0e2/attachment.bin>


More information about the OpenBSC mailing list