<p>dexter has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/pysim/+/23483">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">commands, ts_102_221: replace pytlv with uttlv<br><br>pytlv requires to tell it all expected tags in a TLV string that is<br>going to be parsed. This often causes problems since TLV is by design<br>extensible, so newer specifications may add additional tags which our<br>code then would not tolerate. uttlv does not have such a limitation, pre<br>defining the tags in advance is optional here. Lets use uttlv from now<br>on.<br><br>Change-Id: I971b818836aff2fbeb6769f819d496cee4bfa7cf<br>Related: OS#4963<br>---<br>M README.md<br>M contrib/jenkins.sh<br>M pySim/commands.py<br>M pySim/ts_102_221.py<br>M requirements.txt<br>5 files changed, 60 insertions(+), 45 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/83/23483/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/README.md b/README.md</span><br><span>index 028a84a..8cce9ac 100644</span><br><span>--- a/README.md</span><br><span>+++ b/README.md</span><br><span>@@ -36,13 +36,13 @@</span><br><span> </span><br><span> - pyscard</span><br><span> - serial</span><br><span style="color: hsl(0, 100%, 40%);">-- pytlv (for specific card types)</span><br><span style="color: hsl(120, 100%, 40%);">+- uttlv (for specific card types)</span><br><span> - cmd2 (for pySim-shell.py)</span><br><span> </span><br><span> Example for Debian:</span><br><span> </span><br><span> apt-get install python3-pyscard python3-serial python3-cmd2 python3-pip python3-yaml</span><br><span style="color: hsl(0, 100%, 40%);">- pip3 install pytlv</span><br><span style="color: hsl(120, 100%, 40%);">+ pip3 install uttlv</span><br><span> </span><br><span> Alternatively, everything can be installed using pip:</span><br><span> </span><br><span>diff --git a/contrib/jenkins.sh b/contrib/jenkins.sh</span><br><span>index e286047..db5a75e 100755</span><br><span>--- a/contrib/jenkins.sh</span><br><span>+++ b/contrib/jenkins.sh</span><br><span>@@ -11,7 +11,7 @@</span><br><span> </span><br><span> virtualenv -p python3 venv --system-site-packages</span><br><span> . venv/bin/activate</span><br><span style="color: hsl(0, 100%, 40%);">-pip install pytlv</span><br><span style="color: hsl(120, 100%, 40%);">+pip install uttlv</span><br><span> pip install pyyaml</span><br><span> pip install cmd2</span><br><span> </span><br><span>diff --git a/pySim/commands.py b/pySim/commands.py</span><br><span>index 9aed588..ef355e7 100644</span><br><span>--- a/pySim/commands.py</span><br><span>+++ b/pySim/commands.py</span><br><span>@@ -21,7 +21,7 @@</span><br><span> # along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span> #</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-from pySim.utils import rpad, b2h</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.utils import rpad, b2h, h2b</span><br><span> </span><br><span> class SimCardCommands(object):</span><br><span> def __init__(self, transport):</span><br><span>@@ -33,11 +33,10 @@</span><br><span> def __parse_fcp(self, fcp):</span><br><span> # see also: ETSI TS 102 221, chapter 11.1.1.3.1 Response for MF,</span><br><span> # DF or ADF</span><br><span style="color: hsl(0, 100%, 40%);">- from pytlv.TLV import TLV</span><br><span style="color: hsl(0, 100%, 40%);">- tlvparser = TLV(['82', '83', '84', 'a5', '8a', '8b', '8c', '80', 'ab', 'c6', '81', '88'])</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- # pytlv is case sensitive!</span><br><span style="color: hsl(0, 100%, 40%);">- fcp = fcp.lower()</span><br><span style="color: hsl(120, 100%, 40%);">+ from uttlv import TLV</span><br><span style="color: hsl(120, 100%, 40%);">+ tlvparser = TLV()</span><br><span style="color: hsl(120, 100%, 40%);">+ tlvparser.tag_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ tlvparser.len_size = 1</span><br><span> </span><br><span> if fcp[0:2] != '62':</span><br><span> raise ValueError('Tag of the FCP template does not match, expected 62 but got %s'%fcp[0:2])</span><br><span>@@ -57,7 +56,8 @@</span><br><span> </span><br><span> # Skip FCP tag and length</span><br><span> tlv = fcp[skip:]</span><br><span style="color: hsl(0, 100%, 40%);">- return tlvparser.parse(tlv)</span><br><span style="color: hsl(120, 100%, 40%);">+ tlvparser.parse_array(bytes(h2b(tlv)))</span><br><span style="color: hsl(120, 100%, 40%);">+ return tlvparser</span><br><span> </span><br><span> # Tell the length of a record by the card response</span><br><span> # USIMs respond with an FCP template, which is different</span><br><span>@@ -67,7 +67,7 @@</span><br><span> def __record_len(self, r):</span><br><span> if self.sel_ctrl == "0004":</span><br><span> tlv_parsed = self.__parse_fcp(r[-1])</span><br><span style="color: hsl(0, 100%, 40%);">- file_descriptor = tlv_parsed['82']</span><br><span style="color: hsl(120, 100%, 40%);">+ file_descriptor = b2h(tlv_parsed[0x82])</span><br><span> # See also ETSI TS 102 221, chapter 11.1.1.4.3 File Descriptor</span><br><span> return int(file_descriptor[4:8], 16)</span><br><span> else:</span><br><span>@@ -78,7 +78,7 @@</span><br><span> def __len(self, r):</span><br><span> if self.sel_ctrl == "0004":</span><br><span> tlv_parsed = self.__parse_fcp(r[-1])</span><br><span style="color: hsl(0, 100%, 40%);">- return int(tlv_parsed['80'], 16)</span><br><span style="color: hsl(120, 100%, 40%);">+ return int(b2h(tlv_parsed[0x80]), 16)</span><br><span> else:</span><br><span> return int(r[-1][4:8], 16)</span><br><span> </span><br><span>diff --git a/pySim/ts_102_221.py b/pySim/ts_102_221.py</span><br><span>index 256a697..a8d67a1 100644</span><br><span>--- a/pySim/ts_102_221.py</span><br><span>+++ b/pySim/ts_102_221.py</span><br><span>@@ -17,7 +17,7 @@</span><br><span> along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span> """</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-from pytlv.TLV import *</span><br><span style="color: hsl(120, 100%, 40%);">+from uttlv import TLV</span><br><span> from struct import pack, unpack</span><br><span> from pySim.utils import *</span><br><span> from pySim.filesystem import *</span><br><span>@@ -122,18 +122,6 @@</span><br><span> '83': lambda x: int(x, 16),</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-# pytlv unfortunately doesn't have a setting using which we can make it</span><br><span style="color: hsl(0, 100%, 40%);">-# accept unknown tags. It also doesn't raise a specific exception type but</span><br><span style="color: hsl(0, 100%, 40%);">-# just the generic ValueError, so we cannot ignore those either. Instead,</span><br><span style="color: hsl(0, 100%, 40%);">-# we insert a dict entry for every possible proprietary tag permitted</span><br><span style="color: hsl(0, 100%, 40%);">-def fixup_fcp_proprietary_tlv_map(tlv_map):</span><br><span style="color: hsl(0, 100%, 40%);">- if 'D0' in tlv_map:</span><br><span style="color: hsl(0, 100%, 40%);">- return</span><br><span style="color: hsl(0, 100%, 40%);">- for i in range(0xd0, 0xff):</span><br><span style="color: hsl(0, 100%, 40%);">- i_hex = i2h([i]).upper()</span><br><span style="color: hsl(0, 100%, 40%);">- tlv_map[i_hex] = 'proprietary_' + i_hex</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> def tlv_key_replace(inmap, indata):</span><br><span> def newkey(inmap, key):</span><br><span> if key in inmap:</span><br><span>@@ -153,22 +141,39 @@</span><br><span> </span><br><span> # ETSI TS 102 221 Section 11.1.1.3</span><br><span> def decode_select_response(resp_hex):</span><br><span style="color: hsl(0, 100%, 40%);">- fixup_fcp_proprietary_tlv_map(FCP_Proprietary_TLV_MAP)</span><br><span style="color: hsl(0, 100%, 40%);">- resp_hex = resp_hex.upper()</span><br><span> # outer layer</span><br><span style="color: hsl(0, 100%, 40%);">- fcp_base_tlv = TLV(['62'])</span><br><span style="color: hsl(0, 100%, 40%);">- fcp_base = fcp_base_tlv.parse(resp_hex)</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_base_tlv = TLV()</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_base_tlv.tag_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_base_tlv.len_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_base_tlv.parse_array(bytes(h2b(resp_hex)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # actual FCP</span><br><span style="color: hsl(0, 100%, 40%);">- fcp_tlv = TLV(FCP_TLV_MAP)</span><br><span style="color: hsl(0, 100%, 40%);">- fcp = fcp_tlv.parse(fcp_base['62'])</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_tlv = TLV()</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_tlv.tag_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_tlv.len_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_tlv.parse_array(fcp_base_tlv[0x62])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # further decode the proprietary information</span><br><span style="color: hsl(0, 100%, 40%);">- if fcp['A5']:</span><br><span style="color: hsl(0, 100%, 40%);">- prop_tlv = TLV(FCP_Proprietary_TLV_MAP)</span><br><span style="color: hsl(0, 100%, 40%);">- prop = prop_tlv.parse(fcp['A5'])</span><br><span style="color: hsl(0, 100%, 40%);">- fcp['A5'] = tlv_val_interpret(FCP_prorietary_interpreter_map, prop)</span><br><span style="color: hsl(0, 100%, 40%);">- fcp['A5'] = tlv_key_replace(FCP_Proprietary_TLV_MAP, fcp['A5'])</span><br><span style="color: hsl(120, 100%, 40%);">+ if fcp_tlv[0xA5]:</span><br><span style="color: hsl(120, 100%, 40%);">+ prop_tlv = TLV()</span><br><span style="color: hsl(120, 100%, 40%);">+ prop_tlv.tag_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ prop_tlv.len_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ prop_tlv.parse_array(fcp_tlv[0xA5])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp = fcp_tlv._items</span><br><span style="color: hsl(120, 100%, 40%);">+ prop = prop_tlv._items</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_str = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ for key in fcp:</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_str.update({("%02X" % key):b2h(fcp[key])})</span><br><span style="color: hsl(120, 100%, 40%);">+ prop_str = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ for key in prop:</span><br><span style="color: hsl(120, 100%, 40%);">+ prop_str.update({("%02X" % key):b2h(prop[key])})</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_str['A5'] = tlv_val_interpret(FCP_prorietary_interpreter_map, prop_str)</span><br><span style="color: hsl(120, 100%, 40%);">+ fcp_str['A5'] = tlv_key_replace(FCP_Proprietary_TLV_MAP, fcp_str['A5'])</span><br><span> # finally make sure we get human-readable keys in the output dict</span><br><span style="color: hsl(0, 100%, 40%);">- r = tlv_val_interpret(FCP_interpreter_map, fcp)</span><br><span style="color: hsl(120, 100%, 40%);">+ r = tlv_val_interpret(FCP_interpreter_map, fcp_str)</span><br><span> return tlv_key_replace(FCP_TLV_MAP, r)</span><br><span> </span><br><span> </span><br><span>@@ -178,14 +183,24 @@</span><br><span> super().__init__(fid, sfid=sfid, name=name, desc=desc, rec_len={5,54})</span><br><span> </span><br><span> def _decode_record_hex(self, raw_hex_data):</span><br><span style="color: hsl(0, 100%, 40%);">- raw_hex_data = raw_hex_data.upper()</span><br><span style="color: hsl(0, 100%, 40%);">- atempl_base_tlv = TLV(['61'])</span><br><span style="color: hsl(0, 100%, 40%);">- atempl_base = atempl_base_tlv.parse(raw_hex_data)</span><br><span style="color: hsl(0, 100%, 40%);">- atempl_TLV_MAP = {'4F': 'aid_value', 50:'label'}</span><br><span style="color: hsl(0, 100%, 40%);">- atempl_tlv = TLV(atempl_TLV_MAP)</span><br><span style="color: hsl(0, 100%, 40%);">- atempl = atempl_tlv.parse(atempl_base['61'])</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_base_tlv = TLV()</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_base_tlv.tag_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_base_tlv.len_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_base_tlv.parse_array(bytes(h2b(raw_hex_data)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_tlv = TLV()</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_tlv.tag_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_tlv.len_size = 1</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_tlv.parse_array(atempl_base_tlv[0x61])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl = atempl_tlv._items</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_str = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ for key in atempl:</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_str.update({("%02X" % key):b2h(atempl[key])})</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # FIXME: "All other Dos are according to ISO/IEC 7816-4"</span><br><span style="color: hsl(0, 100%, 40%);">- return tlv_key_replace(atempl_TLV_MAP, atempl)</span><br><span style="color: hsl(120, 100%, 40%);">+ atempl_TLV_MAP = {'4F': 'aid_value', 50:'label'}</span><br><span style="color: hsl(120, 100%, 40%);">+ return tlv_key_replace(atempl_TLV_MAP, atempl_str)</span><br><span> </span><br><span> # TS 102 221 Section 13.2</span><br><span> class EF_ICCID(TransparentEF):</span><br><span>diff --git a/requirements.txt b/requirements.txt</span><br><span>index 978a3db..fb900b0 100644</span><br><span>--- a/requirements.txt</span><br><span>+++ b/requirements.txt</span><br><span>@@ -1,4 +1,4 @@</span><br><span> pyscard</span><br><span> serial</span><br><span style="color: hsl(0, 100%, 40%);">-pytlv</span><br><span style="color: hsl(120, 100%, 40%);">+uttlv</span><br><span> cmd2</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/pysim/+/23483">change 23483</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/pysim/+/23483"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: pysim </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I971b818836aff2fbeb6769f819d496cee4bfa7cf </div>
<div style="display:none"> Gerrit-Change-Number: 23483 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: dexter <pmaier@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>