<p>Falkenber9 has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/pysim/+/24180">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">Add codecs for EF_SPN and GSM strings via construct<br><br>This will replace the hand-crafted codec for EF_SPN<br>by a struct definition using the construct library.<br>Old encoders are updated and kept for API compatibility<br>but are not used internally anymore.<br><br>New data structures:<br>* Rpad(Adapter): Right-padded bytestring (0xff, adjustable)<br>* GsmStringAdapter(Adapter): Codec for "SMS default 7-bit<br> coded alphabet as defined int TS 23.038" using<br> the gsm0338 library.<br>* GsmString(n): Convenient wrapper of both above<br><br>Adjustments:<br>* utils: update+deprecate old dec_spn(), enc_spn()<br>* remove refs to deprecated functions<br><br>Change-Id: Ia1d3a3835933bac0002b7c52511481dd8094b994<br>---<br>M pySim-read.py<br>M pySim/cards.py<br>M pySim/construct.py<br>M pySim/ts_51_011.py<br>M pySim/utils.py<br>M requirements.txt<br>M setup.py<br>7 files changed, 96 insertions(+), 27 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/80/24180/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/pySim-read.py b/pySim-read.py</span><br><span>index ebe0e29..9feef9d 100755</span><br><span>--- a/pySim-read.py</span><br><span>+++ b/pySim-read.py</span><br><span>@@ -36,7 +36,7 @@</span><br><span> from pySim.transport import init_reader, argparse_add_reader_args</span><br><span> from pySim.cards import card_detect, Card, UsimCard, IsimCard</span><br><span> from pySim.utils import h2b, swap_nibbles, rpad, dec_imsi, dec_iccid, dec_msisdn</span><br><span style="color: hsl(0, 100%, 40%);">-from pySim.utils import format_xplmn_w_act, dec_spn, dec_st, dec_addr_tlv</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.utils import format_xplmn_w_act, dec_st, dec_addr_tlv</span><br><span> from pySim.utils import h2s, format_ePDGSelection</span><br><span> </span><br><span> option_parser = argparse.ArgumentParser(prog='pySim-read',</span><br><span>diff --git a/pySim/cards.py b/pySim/cards.py</span><br><span>index 30857b3..dcba26c 100644</span><br><span>--- a/pySim/cards.py</span><br><span>+++ b/pySim/cards.py</span><br><span>@@ -25,7 +25,7 @@</span><br><span> from typing import Optional, Dict, Tuple</span><br><span> import abc</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-from pySim.ts_51_011 import EF, DF, EF_AD</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_51_011 import EF, DF, EF_AD, EF_SPN</span><br><span> from pySim.ts_31_102 import EF_USIM_ADF_map</span><br><span> from pySim.ts_31_103 import EF_ISIM_ADF_map</span><br><span> from pySim.utils import *</span><br><span>@@ -203,15 +203,24 @@</span><br><span> return sw</span><br><span> </span><br><span> def read_spn(self):</span><br><span style="color: hsl(0, 100%, 40%);">- (spn, sw) = self._scc.read_binary(EF['SPN'])</span><br><span style="color: hsl(120, 100%, 40%);">+ (content, sw) = self._scc.read_binary(EF['SPN'])</span><br><span> if sw == '9000':</span><br><span style="color: hsl(0, 100%, 40%);">- return (dec_spn(spn), sw)</span><br><span style="color: hsl(120, 100%, 40%);">+ abstract_data = EF_SPN().decode_hex(content)</span><br><span style="color: hsl(120, 100%, 40%);">+ show_in_hplmn = abstract_data['show_in_hplmn']</span><br><span style="color: hsl(120, 100%, 40%);">+ hide_in_oplmn = abstract_data['hide_in_oplmn']</span><br><span style="color: hsl(120, 100%, 40%);">+ name = abstract_data['spn']</span><br><span style="color: hsl(120, 100%, 40%);">+ return ((name, show_in_hplmn, hide_in_oplmn), sw)</span><br><span> else:</span><br><span> return (None, sw)</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- def update_spn(self, name, hplmn_disp=False, oplmn_disp=False):</span><br><span style="color: hsl(0, 100%, 40%);">- content = enc_spn(name, hplmn_disp, oplmn_disp)</span><br><span style="color: hsl(0, 100%, 40%);">- data, sw = self._scc.update_binary(EF['SPN'], rpad(content, 32))</span><br><span style="color: hsl(120, 100%, 40%);">+ def update_spn(self, name="", show_in_hplmn=False, hide_in_oplmn=False):</span><br><span style="color: hsl(120, 100%, 40%);">+ abstract_data = {</span><br><span style="color: hsl(120, 100%, 40%);">+ 'hide_in_oplmn' : hide_in_oplmn,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'show_in_hplmn' : show_in_hplmn,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'spn' : name,</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ content = EF_SPN().encode_hex(abstract_data)</span><br><span style="color: hsl(120, 100%, 40%);">+ data, sw = self._scc.update_binary(EF['SPN'], content)</span><br><span> return sw</span><br><span> </span><br><span> def read_binary(self, ef, length=None, offset=0):</span><br><span>@@ -915,8 +924,7 @@</span><br><span> </span><br><span> # set Service Provider Name</span><br><span> if p.get('name') is not None:</span><br><span style="color: hsl(0, 100%, 40%);">- content = enc_spn(p['name'], True, True)</span><br><span style="color: hsl(0, 100%, 40%);">- data, sw = self._scc.update_binary('6F46', rpad(content, 32))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.update_spn(p['name'], True, True)</span><br><span> </span><br><span> if p.get('acc') is not None:</span><br><span> self.update_acc(p['acc'])</span><br><span>@@ -1310,8 +1318,7 @@</span><br><span> </span><br><span> # set Service Provider Name</span><br><span> if p.get('name') is not None:</span><br><span style="color: hsl(0, 100%, 40%);">- content = enc_spn(p['name'], True, True)</span><br><span style="color: hsl(0, 100%, 40%);">- data, sw = self._scc.update_binary('6F46', rpad(content, 32))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.update_spn(p['name'], True, True)</span><br><span> </span><br><span> # write EF.IMSI</span><br><span> if p.get('imsi'):</span><br><span>diff --git a/pySim/construct.py b/pySim/construct.py</span><br><span>index b0f03b7..a903305 100644</span><br><span>--- a/pySim/construct.py</span><br><span>+++ b/pySim/construct.py</span><br><span>@@ -1,5 +1,6 @@</span><br><span> from construct import *</span><br><span> from pySim.utils import b2h, h2b, swap_nibbles</span><br><span style="color: hsl(120, 100%, 40%);">+import gsm0338</span><br><span> </span><br><span> """Utility code related to the integration of the 'construct' declarative parser."""</span><br><span> </span><br><span>@@ -33,6 +34,42 @@</span><br><span> def _encode(self, obj, context, path):</span><br><span> return h2b(swap_nibbles(obj))</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+class Rpad(Adapter):</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ Encoder appends padding bytes (b'\\xff') up to target size.</span><br><span style="color: hsl(120, 100%, 40%);">+ Decoder removes trailing padding bytes.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ Parameters:</span><br><span style="color: hsl(120, 100%, 40%);">+ subcon: Subconstruct as defined by construct library</span><br><span style="color: hsl(120, 100%, 40%);">+ pattern: set padding pattern (default: b'\\xff')</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, subcon, pattern=b'\xff'):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__(subcon)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.pattern = pattern</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _decode(self, obj, context, path):</span><br><span style="color: hsl(120, 100%, 40%);">+ return obj.rstrip(self.pattern)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode(self, obj, context, path):</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(obj) > self.sizeof():</span><br><span style="color: hsl(120, 100%, 40%);">+ raise SizeofError("Input ({}) exceeds target size ({})".format(len(obj), self.sizeof()))</span><br><span style="color: hsl(120, 100%, 40%);">+ return obj + self.pattern * (self.sizeof() - len(obj))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class GsmStringAdapter(Adapter):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Convert GSM 03.38 encoded bytes to a string."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, subcon, codec='gsm03.38', err='strict'):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__(subcon)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.codec = codec</span><br><span style="color: hsl(120, 100%, 40%);">+ self.err = err</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _decode(self, obj, context, path):</span><br><span style="color: hsl(120, 100%, 40%);">+ return obj.decode(self.codec)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode(self, obj, context, path):</span><br><span style="color: hsl(120, 100%, 40%);">+ return obj.encode(self.codec, self.err)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> def filter_dict(d, exclude_prefix='_'):</span><br><span> """filter the input dict to ensure no keys starting with 'exclude_prefix' remain."""</span><br><span> res = {}</span><br><span>@@ -88,3 +125,17 @@</span><br><span> n (Integer): Number of bytes (default: 1)</span><br><span> '''</span><br><span> return Default(Bytes(n), __RFU_VALUE)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def GsmString(n):</span><br><span style="color: hsl(120, 100%, 40%);">+ '''</span><br><span style="color: hsl(120, 100%, 40%);">+ GSM 03.38 encoded byte string of fixed length n.</span><br><span style="color: hsl(120, 100%, 40%);">+ Encoder appends padding bytes (b'\\xff') to maintain</span><br><span style="color: hsl(120, 100%, 40%);">+ length. Decoder removes those trailing bytes.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ Exceptions are raised for invalid characters</span><br><span style="color: hsl(120, 100%, 40%);">+ and length excess.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ Parameters:</span><br><span style="color: hsl(120, 100%, 40%);">+ n (Integer): Fixed length of the encoded byte string</span><br><span style="color: hsl(120, 100%, 40%);">+ '''</span><br><span style="color: hsl(120, 100%, 40%);">+ return GsmStringAdapter(Rpad(Bytes(n), pattern=b'\xff'), codec='gsm03.38')</span><br><span>diff --git a/pySim/ts_51_011.py b/pySim/ts_51_011.py</span><br><span>index dba0369..6ab07f0 100644</span><br><span>--- a/pySim/ts_51_011.py</span><br><span>+++ b/pySim/ts_51_011.py</span><br><span>@@ -323,7 +323,7 @@</span><br><span> from struct import pack, unpack</span><br><span> from construct import *</span><br><span> from construct import Optional as COptional</span><br><span style="color: hsl(0, 100%, 40%);">-from pySim.construct import HexAdapter, BcdAdapter, FlagRFU, ByteRFU, GreedyBytesRFU, BitsRFU, BytesRFU</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.construct import *</span><br><span> import enum</span><br><span> </span><br><span> from pySim.filesystem import *</span><br><span>@@ -519,11 +519,14 @@</span><br><span> class EF_SPN(TransparentEF):</span><br><span> def __init__(self, fid='6f46', sfid=None, name='EF.SPN', desc='Service Provider Name', size={17,17}):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size)</span><br><span style="color: hsl(0, 100%, 40%);">- def _decode_hex(self, raw_hex):</span><br><span style="color: hsl(0, 100%, 40%);">- return {'spn': dec_spn(raw_hex)}</span><br><span style="color: hsl(0, 100%, 40%);">- def _encode_hex(self, abstract):</span><br><span style="color: hsl(0, 100%, 40%);">- spn = abstract['spn']</span><br><span style="color: hsl(0, 100%, 40%);">- return enc_spn(spn[0], spn[1], spn[2])</span><br><span style="color: hsl(120, 100%, 40%);">+ self._construct = BitStruct(</span><br><span style="color: hsl(120, 100%, 40%);">+ # Byte 1</span><br><span style="color: hsl(120, 100%, 40%);">+ 'rfu'/BitsRFU(6),</span><br><span style="color: hsl(120, 100%, 40%);">+ 'hide_in_oplmn'/Flag,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'show_in_hplmn'/Flag,</span><br><span style="color: hsl(120, 100%, 40%);">+ # Bytes 2..17</span><br><span style="color: hsl(120, 100%, 40%);">+ 'spn'/Bytewise(GsmString(16))</span><br><span style="color: hsl(120, 100%, 40%);">+ )</span><br><span> </span><br><span> # TS 51.011 Section 10.3.13</span><br><span> class EF_CBMI(TransRecEF):</span><br><span>diff --git a/pySim/utils.py b/pySim/utils.py</span><br><span>index a3cd1b5..3a8ddac 100644</span><br><span>--- a/pySim/utils.py</span><br><span>+++ b/pySim/utils.py</span><br><span>@@ -248,17 +248,23 @@</span><br><span> return res</span><br><span> </span><br><span> def dec_spn(ef):</span><br><span style="color: hsl(0, 100%, 40%);">- byte1 = int(ef[0:2])</span><br><span style="color: hsl(0, 100%, 40%);">- hplmn_disp = (byte1&0x01 == 0x01)</span><br><span style="color: hsl(0, 100%, 40%);">- oplmn_disp = (byte1&0x02 == 0x02)</span><br><span style="color: hsl(0, 100%, 40%);">- name = h2s(ef[2:])</span><br><span style="color: hsl(0, 100%, 40%);">- return (name, hplmn_disp, oplmn_disp)</span><br><span style="color: hsl(120, 100%, 40%);">+ """Obsolete, kept for API compatibility"""</span><br><span style="color: hsl(120, 100%, 40%);">+ from ts_51_011 import EF_SPN</span><br><span style="color: hsl(120, 100%, 40%);">+ abstract_data = EF_SPN().decode_hex(ef)</span><br><span style="color: hsl(120, 100%, 40%);">+ show_in_hplmn = abstract_data['show_in_hplmn']</span><br><span style="color: hsl(120, 100%, 40%);">+ hide_in_oplmn = abstract_data['hide_in_oplmn']</span><br><span style="color: hsl(120, 100%, 40%);">+ name = abstract_data['spn']</span><br><span style="color: hsl(120, 100%, 40%);">+ return (name, show_in_hplmn, hide_in_oplmn)</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-def enc_spn(name:str, hplmn_disp=False, oplmn_disp=False):</span><br><span style="color: hsl(0, 100%, 40%);">- byte1 = 0x00</span><br><span style="color: hsl(0, 100%, 40%);">- if hplmn_disp: byte1 = byte1|0x01</span><br><span style="color: hsl(0, 100%, 40%);">- if oplmn_disp: byte1 = byte1|0x02</span><br><span style="color: hsl(0, 100%, 40%);">- return i2h([byte1])+s2h(name)</span><br><span style="color: hsl(120, 100%, 40%);">+def enc_spn(name:str, show_in_hplmn=False, hide_in_oplmn=False):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Obsolete, kept for API compatibility"""</span><br><span style="color: hsl(120, 100%, 40%);">+ from ts_51_011 import EF_SPN</span><br><span style="color: hsl(120, 100%, 40%);">+ abstract_data = {</span><br><span style="color: hsl(120, 100%, 40%);">+ 'hide_in_oplmn' : hide_in_oplmn,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'show_in_hplmn' : show_in_hplmn,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'spn' : name,</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ return EF_SPN().encode_hex(abstract_data)</span><br><span> </span><br><span> def hexstr_to_Nbytearr(s, nbytes):</span><br><span> return [s[i:i+(nbytes*2)] for i in range(0, len(s), (nbytes*2)) ]</span><br><span>diff --git a/requirements.txt b/requirements.txt</span><br><span>index 6e8336c..10e9667 100644</span><br><span>--- a/requirements.txt</span><br><span>+++ b/requirements.txt</span><br><span>@@ -5,3 +5,4 @@</span><br><span> jsonpath-ng</span><br><span> construct</span><br><span> bidict</span><br><span style="color: hsl(120, 100%, 40%);">+gsm0338</span><br><span>diff --git a/setup.py b/setup.py</span><br><span>index 08608dc..2ef498f 100644</span><br><span>--- a/setup.py</span><br><span>+++ b/setup.py</span><br><span>@@ -16,6 +16,7 @@</span><br><span> "jsonpath-ng",</span><br><span> "construct >= 2.9",</span><br><span> "bidict",</span><br><span style="color: hsl(120, 100%, 40%);">+ "gsm0338",</span><br><span> ],</span><br><span> scripts=[</span><br><span> 'pySim-prog.py',</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/pysim/+/24180">change 24180</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/pysim/+/24180"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: pysim </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: Ia1d3a3835933bac0002b7c52511481dd8094b994 </div>
<div style="display:none"> Gerrit-Change-Number: 24180 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Falkenber9 <robert.falkenberg@tu-dortmund.de> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>