laforge has submitted this change. ( https://gerrit.osmocom.org/c/libosmo-abis/+/38182?usp=email )
Change subject: trau2rtp TW-TS-001: handle corner case of BFI with stale SID bits
......................................................................
trau2rtp TW-TS-001: handle corner case of BFI with stale SID bits
Many BTS models will emit the previous content of their internal
buffer, perhaps corrupted in some peculiar way, when they need to
emit "BFI with no data" because they received FACCH, or because
they received nothing at all and no channel decoding attempt was
made. This situation is described in detail here:
https://osmocom.org/projects/retro-gsm/wiki/No-data_output
However, when these TRAU-UL BFIs (from Abis or from incoming TFO)
are to be converted to TW-TS-001 with osmo_trau2rtp(), a corner case
arises: if the BTS indicated SID=0 by way of C13 & C14 bits but the
stale payload bit content indicates SID != 0, the full decoder or
TFO transform on the receiving end of the RTP stream will get an
erroneous "invalid SID" indication (instead of "plain" BFI) if the
marked-bad payload is passed through without alteration.
Handle this corner case by modifying marked-bad (BFI=1) payloads
to make them non-SID in the special case when the BTS indicated SID=0,
but the bit pattern says otherwise.
Change-Id: I2800301f7ac25537c4d7fb14acefd73a80590d7c
---
M src/trau/trau_rtp_conv.c
M tests/trau_conv/trau16_efr_twts001.ok
M tests/trau_conv/trau16_fr_twts001.ok
3 files changed, 137 insertions(+), 15 deletions(-)
Approvals:
pespin: Looks good to me, but someone else must approve
laforge: Looks good to me, approved
Jenkins Builder: Verified
diff --git a/src/trau/trau_rtp_conv.c b/src/trau/trau_rtp_conv.c
index 1a200c7..24ea70c 100644
--- a/src/trau/trau_rtp_conv.c
+++ b/src/trau/trau_rtp_conv.c
@@ -98,6 +98,36 @@
//static const uint8_t c_bits_check_fr[] = { 0, 0, 0, 1, 0 };
//static const uint8_t c_bits_check_efr[] = { 1, 1, 0, 1, 0 };
+/*
+ * This little helper function modifies a marked-bad (BFI=1) GSM-FR payload
+ * so it would no longer classify as SID by the bit counting rules
+ * of GSM 06.31 section 6.1.1. It is needed at times when a BTS emits
+ * "BFI with no data", out-of-band SID bits (C13 & C14) are set to 0,
+ * but the stale bit pattern in the output buffer used by the BTS
+ * happens to be (usually invalid) SID.
+ *
+ * In order to reliably "break the SID", we need to set 16 bits in the
+ * SID field to values opposite of SID codeword, i.e., to 1 for GSM-FR.
+ * The SID field of GSM-FR includes the msb of every xMc pulse, and also
+ * the middle bit of most (but not all) xMc pulses. The pulses whose
+ * middle bit is not part of the SID field are the last 9 pulses of
+ * the last subframe. For simplicity, we produce the desired effect
+ * by setting the middle bit of pulses 0, 1, 2 and 3 in every subframe,
+ * for a total of 16 bits.
+ */
+static void make_fr_bfi_nonsid(uint8_t *fr_bytes)
+{
+ uint8_t *subf_bytes;
+ unsigned subn;
+
+ subf_bytes = fr_bytes + 5; /* skip signature and LARc bits */
+ for (subn = 0; subn < 4; subn++) {
+ subf_bytes[2] |= 0x24;
+ subf_bytes[3] |= 0x90;
+ subf_bytes += 7;
+ }
+}
+
/*! Generate the 33 bytes RTP payload for GSM-FR from a decoded TRAU frame.
* \param[out] out caller-provided output buffer
* \param[in] out_len length of out buffer in bytes
@@ -153,6 +183,36 @@
j++;
}
+ /*
+ * Many BTS models will emit the previous content of their internal
+ * buffer, perhaps corrupted in some peculiar way, when they need to
+ * emit "BFI with no data" because they received FACCH, or because
+ * they received nothing at all and no channel decoding attempt was
+ * made. When this situation occurs, the Rx DTX handler in the TRAU
+ * needs to be told "this is a regular BFI, not an invalid SID",
+ * and the BTS makes this indication by setting C12=1 (BFI),
+ * C13=0 and C14=0 (SID=0). However, the stale or corrupted frame
+ * payload bit content in the Abis output buffer will still sometimes
+ * indicate SID (usually invalid) by the bit counting rules of
+ * GSM 06.31 section 6.1.1! This situation creates a problem
+ * in the case of TW-TS-001 output: there are no out-of-band bits
+ * for C13 & C14, and when the remote transcoder or TFO transform
+ * calls osmo_fr_sid_classify() or its libgsmfr2 equivalent,
+ * it will detect an invalid SID (always invalid due to BFI=1)
+ * instead of intended-by-BTS "regular BFI". Solution: because
+ * there is no need to preserve all payload bits that are known
+ * to be bad or dummy, we can afford to corrupt some of them;
+ * as a workaround for the unintentional-SID problem, we "break"
+ * the SID field.
+ *
+ * Note that BFI=1 is a required condition for the logic below.
+ * Because standard RFC 3551 output does not support BFI,
+ * this logic applies only to TW-TS-001 output.
+ */
+ if (tf->c_bits[11] && !tf->c_bits[12] && !tf->c_bits[13] &&
+ osmo_fr_is_any_sid(out))
+ make_fr_bfi_nonsid(out);
+
return req_out_len;
}
@@ -356,6 +416,38 @@
}
}
+/*
+ * This little helper function modifies a marked-bad (BFI=1) GSM-EFR payload
+ * so it would no longer classify as SID by the bit counting rules
+ * of GSM 06.81 section 6.1.1. It is needed at times when a BTS emits
+ * "BFI with no data", out-of-band SID bits (C13 & C14) are set to 0,
+ * but the stale bit pattern in the output buffer used by the BTS
+ * happens to be (usually invalid) SID.
+ *
+ * In order to reliably "break the SID", we need to set 16 bits in the
+ * SID field to values opposite of SID codeword, i.e., to 0 for GSM-EFR.
+ * The SID field of GSM-EFR includes (in every subframe) some bits in
+ * LTP lag and LTP gain fields, and many bits in the fixed codebook
+ * excitation pulses portion. The reference decoder from ETSI makes use
+ * of the fixed codebook portion even in marked-bad frames, hence
+ * we prefer not to corrupt this portion - but we can still reliably
+ * break the SID by clearing some bits in LTP lag and gain fields.
+ * Let's clear the two lsbs of each LTP lag and LTP gain field,
+ * giving us the needed total of 16 bits.
+ */
+static void make_efr_bfi_nonsid(uint8_t *efr_bytes)
+{
+ /* subframe 0 */
+ efr_bytes[6] &= 0x99;
+ /* subframe 1 */
+ efr_bytes[12] &= 0xE6;
+ efr_bytes[13] &= 0x7F;
+ /* subframe 2 */
+ efr_bytes[19] &= 0x33;
+ /* subframe 3 */
+ efr_bytes[25] &= 0xCC;
+}
+
/*! Generate the 31 bytes RTP payload for GSM-EFR from a decoded TRAU frame.
* \param[out] out caller-provided output buffer
* \param[in] out_len length of out buffer in bytes
@@ -430,6 +522,36 @@
if (rc)
goto bad_frame;
+ /*
+ * Many BTS models will emit the previous content of their internal
+ * buffer, perhaps corrupted in some peculiar way, when they need to
+ * emit "BFI with no data" because they received FACCH, or because
+ * they received nothing at all and no channel decoding attempt was
+ * made. When this situation occurs, the Rx DTX handler in the TRAU
+ * needs to be told "this is a regular BFI, not an invalid SID",
+ * and the BTS makes this indication by setting C12=1 (BFI),
+ * C13=0 and C14=0 (SID=0). However, the stale or corrupted frame
+ * payload bit content in the Abis output buffer will still sometimes
+ * indicate SID (usually invalid) by the bit counting rules of
+ * GSM 06.81 section 6.1.1! This situation creates a problem
+ * in the case of TW-TS-001 output: there are no out-of-band bits
+ * for C13 & C14, and when the remote transcoder or TFO transform
+ * calls osmo_efr_sid_classify() or its libgsmefr equivalent,
+ * it will detect an invalid SID (always invalid due to BFI=1)
+ * instead of intended-by-BTS "regular BFI". Solution: because
+ * there is no need to preserve all payload bits that are known
+ * to be bad or dummy, we can afford to corrupt some of them;
+ * as a workaround for the unintentional-SID problem, we "break"
+ * the SID field.
+ *
+ * Note that BFI=1 is a required condition for the logic below.
+ * Because standard RFC 3551 output does not support BFI,
+ * this logic applies only to TW-TS-001 output.
+ */
+ if (tf->c_bits[11] && !tf->c_bits[12] && !tf->c_bits[13] &&
+ osmo_efr_is_any_sid(out))
+ make_efr_bfi_nonsid(out);
+
return req_out_len;
bad_frame:
diff --git a/tests/trau_conv/trau16_efr_twts001.ok b/tests/trau_conv/trau16_efr_twts001.ok
index 14dfe9d..761f03b 100644
--- a/tests/trau_conv/trau16_efr_twts001.ok
+++ b/tests/trau_conv/trau16_efr_twts001.ok
@@ -173,11 +173,11 @@
TW-TS-001 TEH octet: 0xE2
EFR frame:
LPC 15 202 265 142 28
- 339 0 15 15 15 15 12 2 2 2 2 4 19
- 60 15 15 15 15 15 12 4 5 5 0 0 19
- 144 10 7 15 15 15 14 0 0 1 0 0 19
- 63 15 15 12 15 15 12 0 2 2 2 0 19
- SID recompute: 1
+ 336 0 15 15 15 15 12 2 2 2 2 4 19
+ 60 12 15 15 15 15 12 4 5 5 0 0 19
+ 144 8 7 15 15 15 14 0 0 1 0 0 19
+ 60 12 15 12 15 15 12 0 2 2 2 0 19
+ SID recompute: 0
ID efr-dtx-dtmf.bin frame 0x53a51
0000ebead10eba68ada886f7fbfca494ca8cfffffe4ba04ac07ffffff0048491fffcffe0a485c2ff
TRAU frame type: EFR
@@ -197,8 +197,8 @@
TW-TS-001 TEH octet: 0xE2
EFR frame:
LPC 30 10 408 138 54
- 339 0 11 15 11 15 12 2 2 2 2 4 18
- 61 7 15 15 15 15 12 4 5 5 0 0 18
- 16 10 7 15 15 15 14 0 0 1 0 0 18
- 63 15 15 12 15 15 12 0 2 2 2 0 22
- SID recompute: 1
+ 336 0 11 15 11 15 12 2 2 2 2 4 18
+ 60 4 15 15 15 15 12 4 5 5 0 0 18
+ 16 8 7 15 15 15 14 0 0 1 0 0 18
+ 60 12 15 12 15 15 12 0 2 2 2 0 22
+ SID recompute: 0
diff --git a/tests/trau_conv/trau16_fr_twts001.ok b/tests/trau_conv/trau16_fr_twts001.ok
index a26439b..5cef573 100644
--- a/tests/trau_conv/trau16_fr_twts001.ok
+++ b/tests/trau_conv/trau16_fr_twts001.ok
@@ -149,8 +149,8 @@
TW-TS-001 TEH octet: 0xE2
FR frame:
LARc 59 36 12 24 13 4 3 0
- 64 1 0 49 3 2 1 0 0 0 0 0 0 0 0 0 0
- 96 3 0 34 0 1 7 3 1 0 0 0 0 0 0 0 0
- 64 1 1 10 0 0 0 0 1 1 1 0 0 0 0 0 0
- 0 0 0 33 5 1 1 0 0 0 0 1 0 0 0 2 2
- SID recompute: 1
+ 64 1 0 49 3 2 3 2 0 0 0 0 0 0 0 0 0
+ 96 3 0 34 2 3 7 3 1 0 0 0 0 0 0 0 0
+ 64 1 1 10 2 2 2 2 1 1 1 0 0 0 0 0 0
+ 0 0 0 33 7 3 3 2 0 0 0 1 0 0 0 2 2
+ SID recompute: 0
--
To view, visit https://gerrit.osmocom.org/c/libosmo-abis/+/38182?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: libosmo-abis
Gerrit-Branch: master
Gerrit-Change-Id: I2800301f7ac25537c4d7fb14acefd73a80590d7c
Gerrit-Change-Number: 38182
Gerrit-PatchSet: 1
Gerrit-Owner: falconia <falcon(a)freecalypso.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>
laforge has submitted this change. ( https://gerrit.osmocom.org/c/libosmo-abis/+/38184?usp=email )
Change subject: trau2rtp EFR unit test: add patterns with bad CRC
......................................................................
trau2rtp EFR unit test: add patterns with bad CRC
Unlike its FRv1 counterpart, the TRAU frame format for EFR includes
CRC, intended to catch transmission errors in terrestrial circuits.
Add some bad CRC patterns to the unit test for osmo_trau2rtp() EFR.
Change-Id: Iee69b6a1593bdc8d11904a5f2ad46cd0fc6052a9
---
M tests/trau_conv/trau16_efr.in
M tests/trau_conv/trau16_efr_std.ok
M tests/trau_conv/trau16_efr_twts001.ok
3 files changed, 26 insertions(+), 0 deletions(-)
Approvals:
Jenkins Builder: Verified
laforge: Looks good to me, approved
pespin: Looks good to me, but someone else must approve
diff --git a/tests/trau_conv/trau16_efr.in b/tests/trau_conv/trau16_efr.in
index b4bdb02..0f113aa 100644
--- a/tests/trau_conv/trau16_efr.in
+++ b/tests/trau_conv/trau16_efr.in
@@ -78,3 +78,11 @@
0000ebecca1b97f3b45086fffffc8000c68efffffe008046c07fffffe0008451fffcffe08004e2ff
ID perfect SID with BFI=1 SID=0
0000ebe8ca1b97f3b45086fffffc8000c68efffffe008046c07fffffe0008451fffcffe08004e2ff
+
+# The following frames were also constructed by hand. They have bad CRC
+# and thus test the handling of this case in osmo_trau2rtp().
+
+ID all Dn bits set to 0
+0000ebe88000800080008000800080008000800080008000800080008000800080008000800082ff
+ID all Dn bits set to 1
+0000ebe8fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeff
diff --git a/tests/trau_conv/trau16_efr_std.ok b/tests/trau_conv/trau16_efr_std.ok
index 8fff73a..110cd0e 100644
--- a/tests/trau_conv/trau16_efr_std.ok
+++ b/tests/trau_conv/trau16_efr_std.ok
@@ -116,3 +116,11 @@
0000ebe8ca1b97f3b45086fffffc8000c68efffffe008046c07fffffe0008451fffcffe08004e2ff
TRAU frame type: EFR
osmo_trau2rtp() result: 0
+ID all Dn bits set to 0
+0000ebe88000800080008000800080008000800080008000800080008000800080008000800082ff
+ TRAU frame type: EFR
+ osmo_trau2rtp() result: 0
+ID all Dn bits set to 1
+0000ebe8fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeff
+ TRAU frame type: EFR
+ osmo_trau2rtp() result: 0
diff --git a/tests/trau_conv/trau16_efr_twts001.ok b/tests/trau_conv/trau16_efr_twts001.ok
index 2b11d14..370d1ba 100644
--- a/tests/trau_conv/trau16_efr_twts001.ok
+++ b/tests/trau_conv/trau16_efr_twts001.ok
@@ -226,3 +226,13 @@
0 12 15 15 15 15 12 0 0 0 0 0 17
12 12 15 12 15 15 12 0 0 0 0 0 19
SID recompute: 0
+ID all Dn bits set to 0
+0000ebe88000800080008000800080008000800080008000800080008000800080008000800082ff
+ TRAU frame type: EFR
+ osmo_trau2rtp() result: 1
+ TW-TS-001 TEH octet: 0xE6
+ID all Dn bits set to 1
+0000ebe8fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeff
+ TRAU frame type: EFR
+ osmo_trau2rtp() result: 1
+ TW-TS-001 TEH octet: 0xE6
--
To view, visit https://gerrit.osmocom.org/c/libosmo-abis/+/38184?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: libosmo-abis
Gerrit-Branch: master
Gerrit-Change-Id: Iee69b6a1593bdc8d11904a5f2ad46cd0fc6052a9
Gerrit-Change-Number: 38184
Gerrit-PatchSet: 1
Gerrit-Owner: falconia <falcon(a)freecalypso.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>
Attention is currently required from: falconia.
laforge has posted comments on this change by falconia. ( https://gerrit.osmocom.org/c/libosmo-abis/+/38182?usp=email )
Change subject: trau2rtp TW-TS-001: handle corner case of BFI with stale SID bits
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit https://gerrit.osmocom.org/c/libosmo-abis/+/38182?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: libosmo-abis
Gerrit-Branch: master
Gerrit-Change-Id: I2800301f7ac25537c4d7fb14acefd73a80590d7c
Gerrit-Change-Number: 38182
Gerrit-PatchSet: 1
Gerrit-Owner: falconia <falcon(a)freecalypso.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: falconia <falcon(a)freecalypso.org>
Gerrit-Comment-Date: Tue, 17 Sep 2024 15:15:21 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
laforge has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/38190?usp=email )
Change subject: jobs/osmo-gsm-tester_gerrit: disable
......................................................................
jobs/osmo-gsm-tester_gerrit: disable
Disable it, as currently the osmo-gsm-tester jenkins nodes are offline
and starting such a job just leads to a job that is stuck in the jenkins
queue.
Change-Id: Idf5dd17ca24e9589ef9175de42b5102c79285a7d
---
M jobs/osmo-gsm-tester-runner.yml
1 file changed, 1 insertion(+), 0 deletions(-)
Approvals:
Jenkins Builder: Verified
laforge: Looks good to me, approved
pespin: Looks good to me, but someone else must approve
diff --git a/jobs/osmo-gsm-tester-runner.yml b/jobs/osmo-gsm-tester-runner.yml
index b45d537..5bc7140 100644
--- a/jobs/osmo-gsm-tester-runner.yml
+++ b/jobs/osmo-gsm-tester-runner.yml
@@ -154,6 +154,7 @@
# gerrit job
- job:
name: 'osmo-gsm-tester_gerrit'
+ disabled: true # osmo-gsm-tester nodes are currently offline
defaults: runner
scm:
- 'osmo-gsm-tester-gerrit'
--
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/38190?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: Idf5dd17ca24e9589ef9175de42b5102c79285a7d
Gerrit-Change-Number: 38190
Gerrit-PatchSet: 1
Gerrit-Owner: osmith <osmith(a)sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>
laforge has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/38188?usp=email )
Change subject: jobs/osmo-gsm-tester-runner: fix syntax errors
......................................................................
jobs/osmo-gsm-tester-runner: fix syntax errors
Fix the syntax of this file, so it is compatible with Jenkins Job
Builder 6.3.0.
* In shell blocks, {{ and }} needs to be used instead of { and } if the
string in brackets does not refer to a JJB variable.
* "${OSMO_GSM_TESTER_BRANCH}" would refer to a non-existing JJB var,
instead we want the shell variable here (that jenkins sets), so omit
the {}.
Change-Id: I5401d75ac40b8267b59443792079249f00b02111
---
M jobs/osmo-gsm-tester-runner.yml
1 file changed, 6 insertions(+), 6 deletions(-)
Approvals:
pespin: Looks good to me, but someone else must approve
Jenkins Builder: Verified
laforge: Looks good to me, approved
diff --git a/jobs/osmo-gsm-tester-runner.yml b/jobs/osmo-gsm-tester-runner.yml
index 4909fe7..259436c 100644
--- a/jobs/osmo-gsm-tester-runner.yml
+++ b/jobs/osmo-gsm-tester-runner.yml
@@ -34,7 +34,7 @@
- git:
url: https://gerrit.osmocom.org/osmo-gsm-tester
branches:
- - ${OSMO_GSM_TESTER_BRANCH}
+ - '$OSMO_GSM_TESTER_BRANCH'
wipe-workspace: false
skip-tag: true
basedir: osmo-gsm-tester
@@ -227,14 +227,14 @@
repo: osmo-gsm-tester_build-osmocom-bb
- shell: |
# Set a trap to fix workspace permissions / kill the docker container on exit
- clean_up() {
+ clean_up() {{
docker kill "osmo-gsm-tester-virtual" || true
docker run --rm \
-v "$WORKSPACE":/workspace \
debian:bullseye \
chmod -R a+rwX /workspace/
- }
- clean_up_trap() {
+ }}
+ clean_up_trap() {{
set +x
echo
echo "### Clean up ###"
@@ -243,7 +243,7 @@
trap - EXIT INT TERM 0
clean_up
- }
+ }}
trap clean_up_trap EXIT INT TERM 0
# Make sure no test results from a previous run remain
@@ -281,7 +281,7 @@
--name=osmo-gsm-tester-virtual \
--cap-add=sys_nice \
$USER/osmo-gsm-tester \
- /bin/bash -c 'LANG="en_US.utf8" LC_ALL="en_US.UTF-8" LC_LANG="en_US.UTF-8" PATH="$PWD/osmo-gsm-tester/src:${PATH}" ./osmo-gsm-tester/contrib/jenkins-run.sh'
+ /bin/bash -c 'LANG="en_US.utf8" LC_ALL="en_US.UTF-8" LC_LANG="en_US.UTF-8" PATH="$PWD/osmo-gsm-tester/src:${{PATH}}" ./osmo-gsm-tester/contrib/jenkins-run.sh'
publishers:
- archive:
artifacts: '*-run.tgz, *-bin.tgz'
--
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/38188?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: I5401d75ac40b8267b59443792079249f00b02111
Gerrit-Change-Number: 38188
Gerrit-PatchSet: 1
Gerrit-Owner: osmith <osmith(a)sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>