falconia has uploaded this change for review. ( https://gerrit.osmocom.org/c/libosmocore/+/37558?usp=email ) Change subject: codec: add osmo_hr_sid_classify() ...................................................................... codec: add osmo_hr_sid_classify() In order to support DTX, each frame coming out of the channel decoder for TCH/FS, TCH/HS and TCH/EFS needs to be classified as valid SID, invalid SID or non-SID speech as specified in GSM 06.31, 06.41 and 06.81, respectively. However, the case of TCH/HS (GSM 06.41) is more difficult than FR & EFR: ETSI provided an example implementation instead of a stipulation, and because they failed to document the BCI error flag they relied upon, ETSI's reference implementation of HR SID classification defied understanding for a long time. But now this mystery has been cracked: https://osmocom.org/projects/retro-gsm/wiki/HRv1_error_flags Add a function to libosmocodec that implements the same logic as ETSI's swSidDetection(), including support for BCI flag. This function is intended to be used in OsmoBTS (it can post-process the output of sysmoBTS PHY) when TW-TS-002 output is requested, and it can also be used in implementations of SDR-based GSM MS when feeding TCH/HS Rx to a proper speech decoder and Rx DTX handler for HRv1 codec. Related: OS#6036 Change-Id: I5f4eb65379646125b966cf182775b6e9348900bd --- M include/osmocom/codec/codec.h M src/codec/Makefile.am A src/codec/hr_sid_class.c 3 files changed, 211 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/58/37558/1 diff --git a/include/osmocom/codec/codec.h b/include/osmocom/codec/codec.h index c5981f8..7a23e7f 100644 --- a/include/osmocom/codec/codec.h +++ b/include/osmocom/codec/codec.h @@ -100,6 +100,9 @@ enum osmo_gsm631_sid_class osmo_fr_sid_classify(const uint8_t *rtp_payload); enum osmo_gsm631_sid_class osmo_efr_sid_classify(const uint8_t *rtp_payload); +enum osmo_gsm631_sid_class osmo_hr_sid_classify(const uint8_t *rtp_payload, + bool bci_flag, + bool *bfi_from_bci); /*! Check if given FR codec frame is any kind of SID, valid or invalid * \param[in] rtp_payload Buffer with RTP payload diff --git a/src/codec/Makefile.am b/src/codec/Makefile.am index bb01b9d..e9d58c1 100644 --- a/src/codec/Makefile.am +++ b/src/codec/Makefile.am @@ -18,6 +18,7 @@ gsm620.c \ gsm660.c \ gsm690.c \ + hr_sid_class.c \ ecu.c \ ecu_fr.c \ ecu_fr_old.c \ diff --git a/src/codec/hr_sid_class.c b/src/codec/hr_sid_class.c new file mode 100644 index 0000000..3454062 --- /dev/null +++ b/src/codec/hr_sid_class.c @@ -0,0 +1,178 @@ +/* + * This module implements osmo_hr_sid_classify() function - an independent + * reimplementation of the logic that was recommended (but not stipulated + * as normative) by ETSI for classifying received TCH/HS frames as + * valid SID, invalid SID or non-SID speech. + * + * Author: Mychaela N. Falconia &lt;falcon(a)freecalypso.org&gt;rg>, 2024 - however, + * Mother Mychaela's contributions are NOT subject to copyright. + * No rights reserved, all rights relinquished. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include <stdint.h> +#include <stdbool.h> + +#include <osmocom/codec/codec.h> + +/* + * This helper function takes a byte, counts how many bits are set to 1, + * and returns that number. + */ +static unsigned ones_in_byte(uint8_t byte) +{ + unsigned count = 0; + + for (; byte; byte >>= 1) { + if (byte & 1) + count++; + } + return count; +} + +/* + * This helper function takes two byte arrays of equal length (data and mask), + * applies the mask to the data, then counts how many bits are set to 1 + * under the mask, and returns that number. + */ +static unsigned count_ones_under_mask(const uint8_t *data, const uint8_t *mask, + unsigned nbytes) +{ + unsigned n, accum; + uint8_t and; + + accum = 0; + for (n = 0; n < nbytes; n++) { + and = *data++ & *mask++; + accum += ones_in_byte(and); + } + return accum; +} + +/* + * When a GSM-HR SID frame has been decoded correctly in voiced mode, + * the 79 bits of the SID field will be the last bits in the frame. + * In the packed format of TS 101 318, the bits of interest will be + * in the last 10 bytes. The following array is the mask to be applied. + */ +static const uint8_t sid_field_last10_mask[10] = + {0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + +/* + * When a GSM-HR SID frame has been incorrectly decoded in unvoiced mode + * (both mode bits got flipped to 0 by channel errors), the 79 bits + * of the SID field will be badly misordered all over the frame. + * However, they can still be counted for the purpose of SID detection. + * The following array is the mask to be applied to the whole frame + * (14 bytes) to locate the misordered SID field. + */ +static const uint8_t sid_field_misordered[14] = + {0x08, 0xEF, 0x1F, 0x3F, 0xF3, 0xFC, 0xA4, + 0xFF, 0xFA, 0x3F, 0xFF, 0x47, 0xFF, 0xEC}; + +/* + * In the channel coding scheme on TCH/HS, the HR codec frame of 112 bits + * is divided into 95 class 1 bits and 17 class 2 bits. In the packed + * format of TS 101 318, all 17 class 2 bits will always be in the last + * 4 bytes; however, the specific bits will be different depending on + * whether the frame was decoded in voiced or unvoiced mode. + * The following two arrays are masks to be applied to the last 4 bytes. + */ +static const uint8_t class2_mask_voiced[4] = {0x7F, 0x80, 0x3F, 0xE0}; +static const uint8_t class2_mask_unvoiced[4] = {0x07, 0x07, 0xFF, 0xE0}; + +/* + * osmo_hr_sid_classify() - this function is an independent reimplementation + * of the logic that was recommended (but not stipulated as normative) by ETSI + * for classifying received TCH/HS frames as valid SID, invalid SID or non-SID + * speech. ETSI's original version is swSidDetection() function in reid.c + * in GSM 06.06 source; the present version implements exactly the same + * logic (same inputs will produce same output), but differs in the following + * ways: + * + * - The frame of channel-decoded 112 payload bits was passed in the form + * of an array of 18 codec parameters in ETSI's version; the present version + * uses the packed format of TS 101 318 instead. + * + * - The C code implementation was written anew by Mother Mychaela; no code + * in this file has been copied directly from GSM 06.06 code drop. + * + * This function is meant to be used only in the same network element + * that performs GSM 05.03 channel decoding (OsmoBTS, new implementations + * of GSM MS), _*NOT*_ in programs or network elements that receive + * HRv1 codec frames from other elements via RTP or Abis-E1 etc! + * + * The BCI logic recommended by ETSI and implemented in practice by at least + * one vendor whose implementation has been reverse-engineered (TI Calypso) + * is included in this function. To understand this logic, please refer + * to this wiki description: + * + * https://osmocom.org/projects/retro-gsm/wiki/HRv1_error_flags + */ +enum osmo_gsm631_sid_class osmo_hr_sid_classify(const uint8_t *rtp_payload, + bool bci_flag, + bool *bfi_from_bci) +{ + uint8_t mode_bits = rtp_payload[4] & 0x30; + unsigned sid_field_ones, class1_ones, class2_ones; + unsigned sid_field_zeros, class1_zeros; + unsigned invalid_sid_threshold; + enum osmo_gsm631_sid_class sidc; + + if (mode_bits != 0) { /* decoded as voiced */ + sid_field_ones = count_ones_under_mask(rtp_payload + 4, + sid_field_last10_mask, 10); + class2_ones = count_ones_under_mask(rtp_payload + 10, + class2_mask_voiced, 4); + } else { /* decoded as unvoiced */ + sid_field_ones = count_ones_under_mask(rtp_payload, + sid_field_misordered, 14); + class2_ones = count_ones_under_mask(rtp_payload + 10, + class2_mask_unvoiced, 4); + } + class1_ones = sid_field_ones - class2_ones; + sid_field_zeros = 79 - sid_field_ones; + class1_zeros = 62 - class1_ones; + + /* frame classification logic recommended by ETSI */ + if (bci_flag) + invalid_sid_threshold = 16; + else + invalid_sid_threshold = 11; + + if (class1_zeros < 3) + sidc = OSMO_GSM631_SID_CLASS_VALID; + else if (sid_field_zeros < invalid_sid_threshold) + sidc = OSMO_GSM631_SID_CLASS_INVALID; + else + sidc = OSMO_GSM631_SID_CLASS_SPEECH; + + /* If the mode bits got corrupted and the frame was channel-decoded + * as unvoiced, it cannot be taken as valid SID because the bits + * that hold CN parameters have been misordered. Therefore, + * we have to turn it into invalid SID classification. + */ + if (mode_bits == 0 && sidc == OSMO_GSM631_SID_CLASS_VALID) + sidc = OSMO_GSM631_SID_CLASS_INVALID; + + /* ETSI's peculiar logic that "upgrades" BCI error flag to BFI + * (from lowest to highest error severity) when the decoded bit + * pattern matches a set criterion. We leave it up to applications + * whether they choose to apply this logic or not. If this logic + * is not wanted, pass NULL pointer as the last argument. + */ + if (bci_flag && bfi_from_bci && + sid_field_zeros >= 16 && sid_field_zeros <= 25) + *bfi_from_bci = true; + + return sidc; +} -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/37558?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: I5f4eb65379646125b966cf182775b6e9348900bd Gerrit-Change-Number: 37558 Gerrit-PatchSet: 1 Gerrit-Owner: falconia &lt;falcon(a)freecalypso.org&gt; Gerrit-MessageType: newchange