fixeria has posted comments on this change. ( https://gerrit.osmocom.org/c/libosmocore/+/35466?usp=email )
Change subject: core: osmo_tdef_fsm_inst_state_chg(): allow millisecond precision
......................................................................
Patch Set 2:
(1 comment)
File tests/tdef/tdef_test.ok:
https://gerrit.osmocom.org/c/libosmocore/+/35466/comment/f9b7deee_7aa14da5
PS2, Line 172: 0.001000 s remaining
Hmm, I would expect it to be `0.100000 s` given that we round-up to ms.
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/35466?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: I4c4ee89e7e32e86f74cd215f5cbfa44ace5426c1
Gerrit-Change-Number: 35466
Gerrit-PatchSet: 2
Gerrit-Owner: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Comment-Date: Fri, 29 Dec 2023 22:12:49 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Gerrit-MessageType: comment
Attention is currently required from: fixeria.
Hello Jenkins Builder,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/libosmocore/+/35466?usp=email
to look at the new patch set (#2).
The following approvals got outdated and were removed:
Verified-1 by Jenkins Builder
Change subject: core: osmo_tdef_fsm_inst_state_chg(): allow millisecond precision
......................................................................
core: osmo_tdef_fsm_inst_state_chg(): allow millisecond precision
This API predates commit 7b74551b9, which added support for millisecond
granularity to osmo_fsm. Let's do the same for the tdef FSM wrapper
API, allowing the millisecond precision without rounding-up to seconds.
Of course, this patch changes behavior of the existing API, but having
more precise state timeouts is not going to make the API user
experience worse.
The old behavior of using seconds is for kept for:
* OSMO_TDEF_CUSTOM -- still treated as if it was OSMO_TDEF_S.
* \param[in] default_timeout -- still expected to be in seconds.
Change-Id: I4c4ee89e7e32e86f74cd215f5cbfa44ace5426c1
Related: 7b74551b9 "fsm: Allow millisecond granularity in osmo_fsm built-in timer"
---
M include/osmocom/core/tdef.h
M src/core/tdef.c
M tests/tdef/tdef_test.ok
3 files changed, 53 insertions(+), 16 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/66/35466/2
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/35466?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: I4c4ee89e7e32e86f74cd215f5cbfa44ace5426c1
Gerrit-Change-Number: 35466
Gerrit-PatchSet: 2
Gerrit-Owner: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-MessageType: newpatchset
fixeria has uploaded this change for review. ( https://gerrit.osmocom.org/c/libosmocore/+/35466?usp=email )
Change subject: core: osmo_tdef_fsm_inst_state_chg(): allow millisecond precision
......................................................................
core: osmo_tdef_fsm_inst_state_chg(): allow millisecond precision
This API predates commit 7b74551b9, which added support for millisecond
granularity to osmo_fsm. Let's do the same for the tdef FSM wrapper
API, allowing the millisecond precision without rounding-up to seconds.
Of course, this patch changes behavior of the existing API, but having
more precise state timeouts is not going to make the API user
experience worse.
The old behavior of using seconds is for kept for:
* OSMO_TDEF_CUSTOM -- still treated as if it was OSMO_TDEF_S.
* \param[in] default_timeout -- still expected to be in seconds.
Change-Id: I4c4ee89e7e32e86f74cd215f5cbfa44ace5426c1
Related: 7b74551b9 "fsm: Allow millisecond granularity in osmo_fsm built-in timer"
---
M include/osmocom/core/tdef.h
M src/core/tdef.c
M tests/tdef/tdef_test.ok
3 files changed, 53 insertions(+), 16 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/66/35466/1
diff --git a/include/osmocom/core/tdef.h b/include/osmocom/core/tdef.h
index d9d2675..402d010 100644
--- a/include/osmocom/core/tdef.h
+++ b/include/osmocom/core/tdef.h
@@ -121,11 +121,13 @@
const struct osmo_tdef_state_timeout *osmo_tdef_get_state_timeout(uint32_t state,
const struct osmo_tdef_state_timeout *timeouts_array);
-/*! Call osmo_fsm_inst_state_chg() or osmo_fsm_inst_state_chg_keep_timer(), depending on the timeouts_array, tdefs and
- * default_timeout.
+/*! Call osmo_fsm_inst_state_chg[_ms]() or osmo_fsm_inst_state_chg_keep_timer[_ms](),
+ * depending on the timeouts_array, tdefs and default_timeout.
*
- * A T timer configured in sub-second precision is rounded up to the next full second. A timer in unit =
- * OSMO_TDEF_CUSTOM is applied as if the unit is in seconds (i.e. this macro does not make sense for custom units!).
+ * A timer defined with sub-millisecond precision (e.g OSMO_TDEF_US) is rounded up to the next full millisecond.
+ * A timer value defined in units higher than millisecond (e.g. OSMO_TDEF_S, OSMO_TDEF_M) is converted to milliseconds.
+ * A timer in unit = OSMO_TDEF_CUSTOM is applied as if the unit is in seconds (i.e. this macro does not make sense
+ * for custom units!).
*
* See osmo_tdef_get_state_timeout() and osmo_tdef_get().
*
@@ -153,9 +155,10 @@
* \param[in] state State number to transition to.
* \param[in] timeouts_array Array of struct osmo_tdef_state_timeout[32] to look up state in.
* \param[in] tdefs Array of struct osmo_tdef (last entry zero initialized) to look up T in.
- * \param[in] default_timeout If a T is set in timeouts_array, but no timeout value is configured for T, then use this
- * default timeout value as fallback, or pass -1 to abort the program.
- * \return Return value from osmo_fsm_inst_state_chg() or osmo_fsm_inst_state_chg_keep_timer().
+ * \param[in] default_timeout If a T is set in timeouts_array, but no timeout value is configured for T,
+ * then use this default timeout value (in seconds) as fallback,
+ * or pass a negative number to abort the program.
+ * \return Return value from osmo_fsm_inst_state_chg[_ms]() or osmo_fsm_inst_state_chg_keep_timer[_ms]().
*/
#define osmo_tdef_fsm_inst_state_chg(fi, state, timeouts_array, tdefs, default_timeout) \
_osmo_tdef_fsm_inst_state_chg(fi, state, timeouts_array, tdefs, default_timeout, \
diff --git a/src/core/tdef.c b/src/core/tdef.c
index abbe581..4d83ee1 100644
--- a/src/core/tdef.c
+++ b/src/core/tdef.c
@@ -337,26 +337,37 @@
const char *file, int line)
{
const struct osmo_tdef_state_timeout *t = osmo_tdef_get_state_timeout(state, timeouts_array);
- unsigned long val = 0;
+ unsigned long val_ms = 0;
/* No timeout defined for this state? */
if (!t)
return _osmo_fsm_inst_state_chg(fi, state, 0, 0, file, line);
- if (t->T)
- val = osmo_tdef_get(tdefs, t->T, OSMO_TDEF_S, default_timeout);
+ if (t->T) {
+ const struct osmo_tdef *tdef = osmo_tdef_get_entry((struct osmo_tdef*)tdefs, t->T);
+ if (tdef == NULL) {
+ /* emulate the old behavior: treat default_timeout as OSMO_TDEF_S */
+ OSMO_ASSERT(default_timeout >= 0);
+ val_ms = default_timeout * 1000;
+ } else {
+ val_ms = osmo_tdef_round(tdef->val, tdef->unit, OSMO_TDEF_MS);
+ /* emulate the old behavior: treat OSMO_TDEF_CUSTOM as OSMO_TDEF_S */
+ if (tdef->unit == OSMO_TDEF_CUSTOM)
+ val_ms *= 1000;
+ }
+ }
if (t->keep_timer) {
if (t->T)
- return _osmo_fsm_inst_state_chg_keep_or_start_timer(fi, state, val, t->T, file, line);
+ return _osmo_fsm_inst_state_chg_keep_or_start_timer_ms(fi, state, val_ms, t->T, file, line);
else
return _osmo_fsm_inst_state_chg_keep_timer(fi, state, file, line);
}
- /* val is always initialized here, because if t->keep_timer is false, t->T must be != 0.
+ /* val_ms is always initialized here, because if t->keep_timer is false, t->T must be != 0.
* Otherwise osmo_tdef_get_state_timeout() would have returned NULL. */
OSMO_ASSERT(t->T);
- return _osmo_fsm_inst_state_chg(fi, state, val, t->T, file, line);
+ return _osmo_fsm_inst_state_chg_ms(fi, state, val_ms, t->T, file, line);
}
const struct value_string osmo_tdef_unit_names[] = {
diff --git a/tests/tdef/tdef_test.ok b/tests/tdef/tdef_test.ok
index d934292..827b9ab 100644
--- a/tests/tdef/tdef_test.ok
+++ b/tests/tdef/tdef_test.ok
@@ -165,11 +165,11 @@
test_tdef_state_timeout()
state=A T=0, no timeout
--> A (configured as T1 100 s) rc=0; state=A T=1, 100.000000 s remaining
- --> B (configured as T2 100 ms) rc=0; state=B T=2, 1.000000 s remaining
+ --> B (configured as T2 100 ms) rc=0; state=B T=2, 0.100000 s remaining
--> C (configured as T3 50 m) rc=0; state=C T=3, 3000.000000 s remaining
--> D (configured as T4 100 custom-unit) rc=0; state=D T=4, 100.000000 s remaining
- --> E (configured as T-5 100 ms) rc=0; state=E T=-5, 1.000000 s remaining
- --> F (configured as T-6 100 us) rc=0; state=F T=-6, 1.000000 s remaining
+ --> E (configured as T-5 100 ms) rc=0; state=E T=-5, 0.100000 s remaining
+ --> F (configured as T-6 100 us) rc=0; state=F T=-6, 0.001000 s remaining
--> G (configured as T7 50 s) rc=0; state=G T=7, 50.000000 s remaining
--> H (configured as T8 300 s) rc=0; state=H T=8, 300.000000 s remaining
--> I (configured as T9 5 m) rc=0; state=I T=9, 300.000000 s remaining
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/35466?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: I4c4ee89e7e32e86f74cd215f5cbfa44ace5426c1
Gerrit-Change-Number: 35466
Gerrit-PatchSet: 1
Gerrit-Owner: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-MessageType: newchange
laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/35453?usp=email )
Change subject: support UCS-2 characters in EF.MMSUP, EF.ADN, EF.SPN, EF.PNN, EF.ECC
......................................................................
support UCS-2 characters in EF.MMSUP, EF.ADN, EF.SPN, EF.PNN, EF.ECC
Now that we have support for the UCS-2 encoding as per TS 102 221 Annex A,
we can start to make use of it from various file constructs.
As some specs say "Either 7-bit GSM or UCS-2" we also introduce
a related automatic GsmOrUcs2Adapter and GsmOrUcs2String class.
Change-Id: I4eb8aea0a13260a143e2c60fca73c3c4312fd3b2
---
M pySim/construct.py
M pySim/ts_31_102.py
M pySim/ts_51_011.py
3 files changed, 56 insertions(+), 4 deletions(-)
Approvals:
fixeria: Looks good to me, but someone else must approve
Jenkins Builder: Verified
laforge: Looks good to me, approved
diff --git a/pySim/construct.py b/pySim/construct.py
index 778a878..1ed3576 100644
--- a/pySim/construct.py
+++ b/pySim/construct.py
@@ -48,6 +48,29 @@
def _encode(self, obj, context, path):
return codecs.encode(obj, "utf-8")
+class GsmOrUcs2Adapter(Adapter):
+ """Try to encode into a GSM 03.38 string; if that fails, fall back to UCS-2 as described
+ in TS 102 221 Annex A."""
+ def _decode(self, obj, context, path):
+ # In case the string contains only 0xff bytes we interpret it as an empty string
+ if obj == b'\xff' * len(obj):
+ return ""
+ # one of the magic bytes of TS 102 221 Annex A
+ if obj[0] in [0x80, 0x81, 0x82]:
+ ad = Ucs2Adapter(GreedyBytes)
+ else:
+ ad = GsmString(GreedyBytes)
+ return ad._decode(obj, context, path)
+
+ def _encode(self, obj, context, path):
+ # first try GSM 03.38; then fall back to TS 102 221 Annex A UCS-2
+ try:
+ ad = GsmString(GreedyBytes)
+ return ad._encode(obj, context, path)
+ except:
+ ad = Ucs2Adapter(GreedyBytes)
+ return ad._encode(obj, context, path)
+
class Ucs2Adapter(Adapter):
"""convert a bytes() type that contains UCS2 encoded characters encoded as defined in TS 102 221
Annex A to normal python string representation (and back)."""
@@ -447,6 +470,20 @@
'''
return GsmStringAdapter(Rpad(Bytes(n), pattern=b'\xff'), codec='gsm03.38')
+def GsmOrUcs2String(n):
+ '''
+ GSM 03.38 or UCS-2 (TS 102 221 Annex A) encoded byte string of fixed length n.
+ Encoder appends padding bytes (b'\\xff') to maintain
+ length. Decoder removes those trailing bytes.
+
+ Exceptions are raised for invalid characters
+ and length excess.
+
+ Parameters:
+ n (Integer): Fixed length of the encoded byte string
+ '''
+ return GsmOrUcs2Adapter(Rpad(Bytes(n), pattern=b'\xff'))
+
class GreedyInteger(Construct):
"""A variable-length integer implementation, think of combining GrredyBytes with BytesInteger."""
def __init__(self, signed=False, swapped=False, minlen=0):
diff --git a/pySim/ts_31_102.py b/pySim/ts_31_102.py
index 16526c2..1a35cb7 100644
--- a/pySim/ts_31_102.py
+++ b/pySim/ts_31_102.py
@@ -529,7 +529,7 @@
cc_construct = BcdAdapter(Rpad(Bytes(3)))
category_construct = FlagsEnum(Byte, police=1, ambulance=2, fire_brigade=3, marine_guard=4,
mountain_rescue=5, manual_ecall=6, automatic_ecall=7)
- alpha_construct = GsmStringAdapter(Rpad(GreedyBytes))
+ alpha_construct = GsmOrUcs2Adapter(Rpad(GreedyBytes))
def __init__(self, fid='6fb7', sfid=0x01, name='EF.ECC',
desc='Emergency Call Codes'):
diff --git a/pySim/ts_51_011.py b/pySim/ts_51_011.py
index 422b35e..6523769 100644
--- a/pySim/ts_51_011.py
+++ b/pySim/ts_51_011.py
@@ -145,7 +145,7 @@
def __init__(self, fid='6f3a', sfid=None, name='EF.ADN', desc='Abbreviated Dialing Numbers', ext=1, **kwargs):
super().__init__(fid, sfid=sfid, name=name, desc=desc, rec_len=(14, 30), **kwargs)
ext_name = 'ext%u_record_id' % ext
- self._construct = Struct('alpha_id'/COptional(GsmStringAdapter(Rpad(Bytes(this._.total_len-14)), codec='ascii')),
+ self._construct = Struct('alpha_id'/COptional(GsmOrUcs2Adapter(Rpad(Bytes(this._.total_len-14)))),
'len_of_bcd'/Int8ub,
'ton_npi'/TonNpi,
'dialing_nr'/ExtendedBcdAdapter(BcdAdapter(Rpad(Bytes(10)))),
@@ -514,7 +514,7 @@
'hide_in_oplmn'/Flag,
'show_in_hplmn'/Flag,
# Bytes 2..17
- 'spn'/Bytewise(GsmString(16))
+ 'spn'/Bytewise(GsmOrUcs2String(16))
)
# TS 51.011 Section 10.3.13
@@ -929,7 +929,7 @@
# TS 51.011 Section 10.3.54
class EF_MMSUP(LinFixedEF):
class MMS_UserPref_ProfileName(BER_TLV_IE, tag=0x81):
- pass
+ _construct = GsmOrUcs2Adapter(GreedyBytes)
class MMS_UserPref_Info(BER_TLV_IE, tag=0x82):
pass
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/35453?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I4eb8aea0a13260a143e2c60fca73c3c4312fd3b2
Gerrit-Change-Number: 35453
Gerrit-PatchSet: 6
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: dexter <pmaier(a)sysmocom.de>
Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-MessageType: merged
laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/35452?usp=email )
Change subject: Implement convoluted encoding of UCS-2 as per TS 102 221 Annex A
......................................................................
Implement convoluted encoding of UCS-2 as per TS 102 221 Annex A
TS 102 221 Annex A defines three variants of encoding UCS-2 characters
into byte streams in files on UICC cards: One rather simplistic one, and
two variants for optimizing memory utilization on the card.
Let's impelement a construct "Ucs2Adapter" class for this.
Change-Id: Ic8bc8f71079faec1bf0e538dc0dfa21403869c6d
---
M pySim/construct.py
M tests/test_construct.py
2 files changed, 217 insertions(+), 0 deletions(-)
Approvals:
laforge: Looks good to me, approved
Jenkins Builder: Verified
fixeria: Looks good to me, but someone else must approve
diff --git a/pySim/construct.py b/pySim/construct.py
index f78adfe..778a878 100644
--- a/pySim/construct.py
+++ b/pySim/construct.py
@@ -48,6 +48,164 @@
def _encode(self, obj, context, path):
return codecs.encode(obj, "utf-8")
+class Ucs2Adapter(Adapter):
+ """convert a bytes() type that contains UCS2 encoded characters encoded as defined in TS 102 221
+ Annex A to normal python string representation (and back)."""
+ def _decode(self, obj, context, path):
+ # In case the string contains only 0xff bytes we interpret it as an empty string
+ if obj == b'\xff' * len(obj):
+ return ""
+ if obj[0] == 0x80:
+ # TS 102 221 Annex A Variant 1
+ return codecs.decode(obj[1:], 'utf_16_be')
+ elif obj[0] == 0x81:
+ # TS 102 221 Annex A Variant 2
+ out = ""
+ # second byte contains a value indicating the number of characters
+ num_of_chars = obj[1]
+ # the third byte contains an 8 bit number which defines bits 15 to 8 of a 16 bit base
+ # pointer, where bit 16 is set to zero, and bits 7 to 1 are also set to zero. These
+ # sixteen bits constitute a base pointer to a "half-page" in the UCS2 code space
+ base_ptr = obj[2] << 7
+ for ch in obj[3:3+num_of_chars]:
+ # if bit 8 of the byte is set to zero, the remaining 7 bits of the byte contain a
+ # GSM Default Alphabet character, whereas if bit 8 of the byte is set to one, then
+ # the remaining seven bits are an offset value added to the 16 bit base pointer
+ # defined earlier, and the resultant 16 bit value is a UCS2 code point
+ if ch & 0x80:
+ codepoint = (ch & 0x7f) + base_ptr
+ out += codecs.decode(codepoint.to_bytes(2, byteorder='big'), 'utf_16_be')
+ else:
+ out += codecs.decode(bytes([ch]), 'gsm03.38')
+ return out
+ elif obj[0] == 0x82:
+ # TS 102 221 Annex A Variant 3
+ out = ""
+ # second byte contains a value indicating the number of characters
+ num_of_chars = obj[1]
+ # third and fourth bytes contain a 16 bit number which defines the complete 16 bit base
+ # pointer to a half-page in the UCS2 code space, for use with some or all of the
+ # remaining bytes in the string
+ base_ptr = obj[2] << 8 | obj[3]
+ for ch in obj[4:4+num_of_chars]:
+ # if bit 8 of the byte is set to zero, the remaining 7 bits of the byte contain a
+ # GSM Default Alphabet character, whereas if bit 8 of the byte is set to one, the
+ # remaining seven bits are an offset value added to the base pointer defined in
+ # bytes three and four, and the resultant 16 bit value is a UCS2 code point, else: #
+ # GSM default alphabet
+ if ch & 0x80:
+ codepoint = (ch & 0x7f) + base_ptr
+ out += codecs.decode(codepoint.to_bytes(2, byteorder='big'), 'utf_16_be')
+ else:
+ out += codecs.decode(bytes([ch]), 'gsm03.38')
+ return out
+ else:
+ raise ValueError('First byte of TS 102 221 UCS-2 must be 0x80, 0x81 or 0x82')
+
+ def _encode(self, obj, context, path):
+ def encodable_in_gsm338(instr: str) -> bool:
+ """Determine if given input string is encode-ale in gsm03.38."""
+ try:
+ # TODO: figure out if/how we can constrain to default alphabet. The gsm0338
+ # library seems to include the spanish lock/shift table
+ codecs.encode(instr, 'gsm03.38')
+ except ValueError:
+ return False
+ return True
+
+ def codepoints_not_in_gsm338(instr: str) -> typing.List[int]:
+ """Return an integer list of UCS2 codepoints for all characters of 'inster'
+ which are not representable in the GSM 03.38 default alphabet."""
+ codepoint_list = []
+ for c in instr:
+ if encodable_in_gsm338(c):
+ continue
+ c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+ codepoint_list.append(c_codepoint)
+ return codepoint_list
+
+ def diff_between_min_and_max_of_list(inlst: typing.List) -> int:
+ return max(inlst) - min(inlst)
+
+ def encodable_in_variant2(instr: str) -> bool:
+ codepoint_prefix = None
+ for c in instr:
+ if encodable_in_gsm338(c):
+ continue
+ c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+ if c_codepoint >= 0x8000:
+ return False
+ c_prefix = c_codepoint >> 7
+ if codepoint_prefix is None:
+ codepoint_prefix = c_prefix
+ else:
+ if c_prefix != codepoint_prefix:
+ return False
+ return True
+
+ def encodable_in_variant3(instr: str) -> bool:
+ codepoint_list = codepoints_not_in_gsm338(instr)
+ # compute delta between max and min; check if it's encodable in 7 bits
+ if diff_between_min_and_max_of_list(codepoint_list) >= 0x80:
+ return False
+ return True
+
+ def _encode_variant1(instr: str) -> bytes:
+ """Encode according to TS 102 221 Annex A Variant 1"""
+ return b'\x80' + codecs.encode(obj, 'utf_16_be')
+
+ def _encode_variant2(instr: str) -> bytes:
+ """Encode according to TS 102 221 Annex A Variant 2"""
+ codepoint_prefix = None
+ # second byte contains a value indicating the number of characters
+ hdr = b'\x81' + len(instr).to_bytes(1, byteorder='big')
+ chars = b''
+ for c in instr:
+ try:
+ enc = codecs.encode(c, 'gsm03.38')
+ except ValueError:
+ c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+ c_prefix = c_codepoint >> 7
+ if codepoint_prefix is None:
+ codepoint_prefix = c_prefix
+ assert codepoint_prefix == c_prefix
+ enc = (0x80 + (c_codepoint & 0x7f)).to_bytes(1, byteorder='big')
+ chars += enc
+ if codepoint_prefix == None:
+ codepoint_prefix = 0
+ return hdr + codepoint_prefix.to_bytes(1, byteorder='big') + chars
+
+ def _encode_variant3(instr: str) -> bytes:
+ """Encode according to TS 102 221 Annex A Variant 3"""
+ # second byte contains a value indicating the number of characters
+ hdr = b'\x82' + len(instr).to_bytes(1, byteorder='big')
+ chars = b''
+ codepoint_list = codepoints_not_in_gsm338(instr)
+ codepoint_base = min(codepoint_list)
+ for c in instr:
+ try:
+ # if bit 8 of the byte is set to zero, the remaining 7 bits of the byte contain a GSM
+ # Default # Alphabet character
+ enc = codecs.encode(c, 'gsm03.38')
+ except ValueError:
+ # if bit 8 of the byte is set to one, the remaining seven bits are an offset
+ # value added to the base pointer defined in bytes three and four, and the
+ # resultant 16 bit value is a UCS2 code point
+ c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+ c_codepoint_delta = c_codepoint - codepoint_base
+ assert c_codepoint_delta < 0x80
+ enc = (0x80 + c_codepoint_delta).to_bytes(1, byteorder='big')
+ chars += enc
+ # third and fourth bytes contain a 16 bit number which defines the complete 16 bit base
+ # pointer to a half-page in the UCS2 code space
+ return hdr + codepoint_base.to_bytes(2, byteorder='big') + chars
+
+ if encodable_in_variant2(obj):
+ return _encode_variant2(obj)
+ elif encodable_in_variant3(obj):
+ return _encode_variant3(obj)
+ else:
+ return _encode_variant1(obj)
class BcdAdapter(Adapter):
"""convert a bytes() type to a string of BCD nibbles."""
diff --git a/tests/test_construct.py b/tests/test_construct.py
index f1bee5a..11822a8 100644
--- a/tests/test_construct.py
+++ b/tests/test_construct.py
@@ -33,5 +33,49 @@
self.assertEqual(filter_dict(inp), out)
+class TestUcs2Adapter(unittest.TestCase):
+ # the three examples from TS 102 221 Annex A
+ EXAMPLE1 = b'\x80\x00\x30\x00\x31\x00\x32\x00\x33'
+ EXAMPLE2 = b'\x81\x05\x13\x53\x95\xa6\xa6\xff\xff'
+ EXAMPLE3 = b'\x82\x05\x05\x30\x2d\x82\xd3\x2d\x31'
+ ad = Ucs2Adapter(GreedyBytes)
+
+ def test_example1_decode(self):
+ dec = self.ad._decode(self.EXAMPLE1, None, None)
+ self.assertEqual(dec, "0123")
+
+ def test_example2_decode(self):
+ dec = self.ad._decode(self.EXAMPLE2, None, None)
+ self.assertEqual(dec, "S\u0995\u09a6\u09a6\u09ff")
+
+ def test_example3_decode(self):
+ dec = self.ad._decode(self.EXAMPLE3, None, None)
+ self.assertEqual(dec, "-\u0532\u0583-1")
+
+ testdata = [
+ # variant 2 with only GSM alphabet characters
+ ( "mahlzeit", '8108006d61686c7a656974' ),
+ # variant 2 with mixed GSM alphabet + UCS2
+ ( "mahlzeit\u099523", '810b136d61686c7a656974953233' ),
+ # variant 3 due to codepoint exceeding 8 bit
+ ( "mahl\u8023zeit", '820980236d61686c807a656974' ),
+ # variant 1 as there is no common codepoint pointer / prefix
+ ( "\u3000\u2000\u1000", '80300020001000' ),
+ ]
+
+ def test_data_decode(self):
+ for string, encoded_hex in self.testdata:
+ encoded = h2b(encoded_hex)
+ dec = self.ad._decode(encoded, None, None)
+ self.assertEqual(dec, string)
+
+ def test_data_encode(self):
+ for string, encoded_hex in self.testdata:
+ encoded = h2b(encoded_hex)
+ re_enc = self.ad._encode(string, None, None)
+ self.assertEqual(encoded, re_enc)
+
+
+
if __name__ == "__main__":
unittest.main()
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/35452?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: Ic8bc8f71079faec1bf0e538dc0dfa21403869c6d
Gerrit-Change-Number: 35452
Gerrit-PatchSet: 6
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: dexter <pmaier(a)sysmocom.de>
Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-MessageType: merged