<p>dexter has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/pysim/+/24012">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">filesystem: add unit tests for encoder/decoder methods<br><br>Some files have a custom _encode... and _decode... metod. Those methods<br>can be detected automatically and tested with a test vector that is<br>directly defined in the respective file class.<br><br>Change-Id: I02d884547f4982e0b8ed7ef21b8cda75237942e2<br>Related: OS#4963<br>---<br>M pySim/filesystem.py<br>M pySim/ts_102_221.py<br>M pySim/ts_31_102.py<br>M pySim/ts_31_103.py<br>M pySim/ts_51_011.py<br>A tests/test_files.py<br>6 files changed, 126 insertions(+), 1 deletion(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/12/24012/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/pySim/filesystem.py b/pySim/filesystem.py</span><br><span>index dec521e..f0c985c 100644</span><br><span>--- a/pySim/filesystem.py</span><br><span>+++ b/pySim/filesystem.py</span><br><span>@@ -34,7 +34,7 @@</span><br><span> </span><br><span> from typing import cast, Optional, Iterable, List, Any, Dict, Tuple</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-from pySim.utils import sw_match, h2b, b2h, is_hex</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.utils import sw_match, h2b, b2h, is_hex, JsonEncoder</span><br><span> from pySim.construct import filter_dict</span><br><span> from pySim.exceptions import *</span><br><span> from pySim.jsonpath import js_path_find, js_path_modify</span><br><span>@@ -378,6 +378,58 @@</span><br><span> sels.update({x.name:x for x in self.parent.children.values() if x != self})</span><br><span> return sels</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ def test_encode_decode(self, verbose=False):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Unit-test to verify the encoder and decoder function of a certain</span><br><span style="color: hsl(120, 100%, 40%);">+ file. To run this, simply define a _encode_decode_testvector[] testvector</span><br><span style="color: hsl(120, 100%, 40%);">+ list inside the specific file implementation. The list shall contain at</span><br><span style="color: hsl(120, 100%, 40%);">+ least one dictionary with at least one set of abstract data.</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Find out which encoder/decoder functions are present</span><br><span style="color: hsl(120, 100%, 40%);">+ has_dec_rec = hasattr(self.__class__, '_decode_record_hex') and callable(getattr(self.__class__, '_decode_record_hex'))</span><br><span style="color: hsl(120, 100%, 40%);">+ has_enc_rec = hasattr(self.__class__, '_encode_record_hex') and callable(getattr(self.__class__, '_encode_record_hex'))</span><br><span style="color: hsl(120, 100%, 40%);">+ has_dec = hasattr(self.__class__, '_decode_hex') and callable(getattr(self.__class__, '_decode_hex'))</span><br><span style="color: hsl(120, 100%, 40%);">+ has_enc = hasattr(self.__class__, '_encode_hex') and callable(getattr(self.__class__, '_encode_hex'))</span><br><span style="color: hsl(120, 100%, 40%);">+ has_testvec = hasattr(self.__class__, '_encode_decode_testvector')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Check if a decoder, encoder and a testvector is present</span><br><span style="color: hsl(120, 100%, 40%);">+ if has_dec_rec and has_enc_rec:</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Testing %s (record oriented)" % (self.name))</span><br><span style="color: hsl(120, 100%, 40%);">+ encode = self._encode_record_hex</span><br><span style="color: hsl(120, 100%, 40%);">+ decode = self._decode_record_hex</span><br><span style="color: hsl(120, 100%, 40%);">+ if not has_testvec:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" Cannot test: Please define _encode_decode_testvector[] in %s" % (self.name))</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+ elif has_dec and has_enc:</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Testing %s (transparent)" % (self.name))</span><br><span style="color: hsl(120, 100%, 40%);">+ encode = self._encode_hex</span><br><span style="color: hsl(120, 100%, 40%);">+ decode = self._decode_hex</span><br><span style="color: hsl(120, 100%, 40%);">+ if not has_testvec:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" Cannot test: Please define _encode_decode_testvector[] in %s" % (self.name))</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Encode+Decode the test data and make sure the end result matches the</span><br><span style="color: hsl(120, 100%, 40%);">+ # input</span><br><span style="color: hsl(120, 100%, 40%);">+ for testvec_json in self.__class__._encode_decode_testvector:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" testvec_json: " + str(testvec_json))</span><br><span style="color: hsl(120, 100%, 40%);">+ testvec = json.loads(testvec_json)</span><br><span style="color: hsl(120, 100%, 40%);">+ if verbose:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" testvec: " + str(testvec))</span><br><span style="color: hsl(120, 100%, 40%);">+ encoded = encode(testvec)</span><br><span style="color: hsl(120, 100%, 40%);">+ if verbose:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" encoded: " + str(encoded))</span><br><span style="color: hsl(120, 100%, 40%);">+ decoded = decode(encoded)</span><br><span style="color: hsl(120, 100%, 40%);">+ if verbose:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" decoded: " + str(decoded))</span><br><span style="color: hsl(120, 100%, 40%);">+ decoded_json = json.dumps(decoded, cls=JsonEncoder)</span><br><span style="color: hsl(120, 100%, 40%);">+ if verbose:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" decoded_json: " + str(decoded_json))</span><br><span style="color: hsl(120, 100%, 40%);">+ if testvec_json != decoded_json:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise ValueError("The encoded end result (decoded_json) does not match the original input (testvec_json)")</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" Ok.")</span><br><span> </span><br><span> class TransparentEF(CardEF):</span><br><span> """Transparent EF (Entry File) in the smart card filesystem.</span><br><span>diff --git a/pySim/ts_102_221.py b/pySim/ts_102_221.py</span><br><span>index 88a36a1..ab4ecf6 100644</span><br><span>--- a/pySim/ts_102_221.py</span><br><span>+++ b/pySim/ts_102_221.py</span><br><span>@@ -189,6 +189,7 @@</span><br><span> </span><br><span> # TS 102 221 Section 13.2</span><br><span> class EF_ICCID(TransparentEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"iccid": "8988211000000433188"}']</span><br><span> def __init__(self, fid='2fe2', sfid=0x02, name='EF.ICCID', desc='ICC Identification'):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, size={10,10})</span><br><span> </span><br><span>diff --git a/pySim/ts_31_102.py b/pySim/ts_31_102.py</span><br><span>index de729be..e7f5fbb 100644</span><br><span>--- a/pySim/ts_31_102.py</span><br><span>+++ b/pySim/ts_31_102.py</span><br><span>@@ -291,6 +291,7 @@</span><br><span> </span><br><span> # TS 31.102 4.4.11.8</span><br><span> class EF_SUCI_Calc_Info(TransparentEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"prot_scheme_id_list": [{"priority": 0, "identifier": 2, "key_index": 1}, {"priority": 1, "identifier": 1, "key_index": 2}, {"priority": 2, "identifier": 0, "key_index": 0}], "hnet_pubkey_list": [{"hnet_pubkey_identifier": 27, "hnet_pubkey": "0272da71976234ce833a6907425867b82e074d44ef907dfb4b3e21c1c2256ebcd1"}, {"hnet_pubkey_identifier": 30, "hnet_pubkey": "5a8d38864820197c3394b92613b20b91633cbd897119273bf8e4a6f4eec0a650"}]}']</span><br><span> def __init__(self, fid="4f07", sfid=0x07, name='EF.SUCI_Calc_Info', size={2, None},</span><br><span> desc='SUCI Calc Info'):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size)</span><br><span>diff --git a/pySim/ts_31_103.py b/pySim/ts_31_103.py</span><br><span>index 1009ba3..bac7a7d 100644</span><br><span>--- a/pySim/ts_31_103.py</span><br><span>+++ b/pySim/ts_31_103.py</span><br><span>@@ -117,6 +117,8 @@</span><br><span> </span><br><span> # TS 31.103 Section 4.2.8</span><br><span> class EF_PCSCF(LinFixedEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"addr": "192.168.100.110", "addr_type": "01"}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"addr": "hello,world", "addr_type": "00"}']</span><br><span> def __init__(self, fid='6f09', sfid=None, name='EF.P-CSCF', desc='P-CSCF Address'):</span><br><span> super().__init__(fid=fid, sfid=sfid, name=name, desc=desc)</span><br><span> def _decode_record_hex(self, raw_hex):</span><br><span>diff --git a/pySim/ts_51_011.py b/pySim/ts_51_011.py</span><br><span>index 8e4428c..06f4c0e 100644</span><br><span>--- a/pySim/ts_51_011.py</span><br><span>+++ b/pySim/ts_51_011.py</span><br><span>@@ -376,6 +376,8 @@</span><br><span> </span><br><span> # TS 51.011 Section 10.5.5</span><br><span> class EF_MSISDN(LinFixedEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"msisdn": [1, 1, "+4916012345678"]}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"msisdn": [1, 3, "123456"]}']</span><br><span> def __init__(self, fid='6f40', sfid=None, name='EF.MSISDN', desc='MSISDN'):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, rec_len={15, 34})</span><br><span> def _decode_record_hex(self, raw_hex_data):</span><br><span>@@ -465,6 +467,8 @@</span><br><span> </span><br><span> # TS 51.011 Section 10.3.2</span><br><span> class EF_IMSI(TransparentEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"imsi": "001010000000102"}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"imsi": "123456789012345"}']</span><br><span> def __init__(self, fid='6f07', sfid=None, name='EF.IMSI', desc='IMSI', size={9,9}):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size)</span><br><span> def _decode_hex(self, raw_hex):</span><br><span>@@ -474,6 +478,9 @@</span><br><span> </span><br><span> # TS 51.011 Section 10.3.4</span><br><span> class EF_PLMNsel(TransRecEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"mcc": "123", "mnc": "45"}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "001", "mnc": "01"}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "456", "mnc": "123"}']</span><br><span> def __init__(self, fid='6f30', sfid=None, name='EF.PLMNsel', desc='PLMN selector',</span><br><span> size={24,None}, rec_len=3):</span><br><span> super().__init__(fid, name=name, sfid=sfid, desc=desc, size=size, rec_len=rec_len)</span><br><span>@@ -517,6 +524,9 @@</span><br><span> </span><br><span> # TS 51.011 Section 10.3.11</span><br><span> class EF_SPN(TransparentEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"spn": ["testme", false, false]}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"spn": ["", true, false]}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"spn": ["hello world", false, true]}']</span><br><span> def __init__(self, fid='6f46', sfid=None, name='EF.SPN', desc='Service Provider Name', size={17,17}):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size)</span><br><span> def _decode_hex(self, raw_hex):</span><br><span>@@ -638,6 +648,10 @@</span><br><span> 'corporate'/BcdAdapter(Bytes(4)))</span><br><span> # TS 51.011 Section 10.3.30</span><br><span> class EF_CNL(TransRecEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"mcc": "", "mnc": "", "network_subset": 255, "service_provider_id": 255, "corporate_id": 255}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "001", "mnc": "01", "network_subset": 23, "service_provider_id": 42, "corporate_id": 5}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "123", "mnc": "567", "network_subset": 255, "service_provider_id": 255, "corporate_id": 255}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "262", "mnc": "12", "network_subset": 111, "service_provider_id": 0, "corporate_id": 0}']</span><br><span> def __init__(self, fid='6f32', sfid=None, name='EF.CNL', size={6,None}, rec_len=6,</span><br><span> desc='Co-operative Network List'):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size, rec_len=rec_len)</span><br><span>@@ -676,6 +690,10 @@</span><br><span> </span><br><span> # TS 51.011 Section 10.3.35..37</span><br><span> class EF_xPLMNwAcT(TransRecEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ _encode_decode_testvector = ['{"mcc": "001", "mnc": "01", "act": ["UTRAN", "E-UTRAN", "GSM", "GSM COMPACT", "cdma2000 HRPD", "cdma2000 1xRTT"]}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "001", "mnc": "01", "act": ["UTRAN", "E-UTRAN WB-S1", "GSM", "GSM COMPACT", "cdma2000 HRPD", "cdma2000 1xRTT"]}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "001", "mnc": "01", "act": ["UTRAN", "E-UTRAN NB-S1", "GSM", "GSM COMPACT", "cdma2000 HRPD", "cdma2000 1xRTT"]}',</span><br><span style="color: hsl(120, 100%, 40%);">+ '{"mcc": "001", "mnc": "01", "act": ["UTRAN", "E-UTRAN WB-S1", "E-UTRAN NB-S1", "GSM", "GSM COMPACT", "cdma2000 HRPD", "cdma2000 1xRTT"]}']</span><br><span> def __init__(self, fid, sfid=None, name=None, desc=None, size={40,None}, rec_len=5):</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size, rec_len=rec_len)</span><br><span> def _decode_record_hex(self, in_hex):</span><br><span>diff --git a/tests/test_files.py b/tests/test_files.py</span><br><span>new file mode 100755</span><br><span>index 0000000..a855afa</span><br><span>--- /dev/null</span><br><span>+++ b/tests/test_files.py</span><br><span>@@ -0,0 +1,51 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import unittest</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim import utils</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_31_102 import EF_SUCI_Calc_Info</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from typing import List</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import json</span><br><span style="color: hsl(120, 100%, 40%);">+import gc</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_51_011 import EF, DF, EF_SST_map</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_31_102 import EF_UST_map, EF_USIM_ADF_map</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_31_103 import EF_IST_map, EF_ISIM_ADF_map</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.filesystem import CardEF, CardMF, CardDF, CardADF</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_51_011 import CardProfileSIM, DF_TELECOM, DF_GSM</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_102_221 import CardProfileUICC</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_31_102 import CardApplicationUSIM</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.ts_31_103 import CardApplicationISIM</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Check if a file (specified by its name) exists in a given list with files</span><br><span style="color: hsl(120, 100%, 40%);">+def file_in_list(file_list, name):</span><br><span style="color: hsl(120, 100%, 40%);">+ for f in file_list:</span><br><span style="color: hsl(120, 100%, 40%);">+ if f.name == name:</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class DecTestCase(unittest.TestCase):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def testFileContentDecoderEncoder(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # Create files in memory</span><br><span style="color: hsl(120, 100%, 40%);">+ profile = CardProfileUICC()</span><br><span style="color: hsl(120, 100%, 40%);">+ profile.add_application(CardApplicationUSIM)</span><br><span style="color: hsl(120, 100%, 40%);">+ profile.add_application(CardApplicationISIM)</span><br><span style="color: hsl(120, 100%, 40%);">+ df_telecom = DF_TELECOM()</span><br><span style="color: hsl(120, 100%, 40%);">+ df_gsm = DF_GSM()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Collect one sample of each EF from memory</span><br><span style="color: hsl(120, 100%, 40%);">+ test_candidates = []</span><br><span style="color: hsl(120, 100%, 40%);">+ for obj in gc.get_objects():</span><br><span style="color: hsl(120, 100%, 40%);">+ if isinstance(obj, CardEF):</span><br><span style="color: hsl(120, 100%, 40%);">+ if not file_in_list(test_candidates, obj.name):</span><br><span style="color: hsl(120, 100%, 40%);">+ test_candidates.append(obj)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Execute integrated encoder/decoder unit tests</span><br><span style="color: hsl(120, 100%, 40%);">+ for obj in test_candidates:</span><br><span style="color: hsl(120, 100%, 40%);">+ obj.test_encode_decode()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+if __name__ == "__main__":</span><br><span style="color: hsl(120, 100%, 40%);">+ unittest.main()</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/pysim/+/24012">change 24012</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/pysim/+/24012"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: pysim </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I02d884547f4982e0b8ed7ef21b8cda75237942e2 </div>
<div style="display:none"> Gerrit-Change-Number: 24012 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: dexter <pmaier@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>