<p>laforge has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/pysim/+/24451">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">Introduce new object-oriented TLV parser/decoder/encoder<br><br>This introduces a new TLV library that heavily builds upon python object<br>oriented concepts. Contrary to classic TLV parsers it doesn't focus on<br>the structure of Tag, Length and binary Value only, but it supports<br>actual decoding/interpretation of the value part into some kind of JSON<br>serializable dict. The latter can be achieved by imperative<br>encode/decode methods, or by using our existing declarative 'construct'<br>based approach.<br><br>The TLV library supports both BER-TLV and COMPREHENSION-TLV for both<br>nested and non-nested TLV definitions.<br><br>As an example we include TLV definitions for a number of CAT (Card<br>Application Toolkit) IEs.<br><br>Change-Id: I7fc1699443bc9d8a4e7cdd2687af9af7cc03c30e<br>---<br>M docs/library.rst<br>A pySim/cat.py<br>A pySim/tlv.py<br>3 files changed, 617 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/51/24451/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/docs/library.rst b/docs/library.rst</span><br><span>index f531ec5..6ecd55a 100644</span><br><span>--- a/docs/library.rst</span><br><span>+++ b/docs/library.rst</span><br><span>@@ -80,6 +80,11 @@</span><br><span> .. automodule:: pySim.construct</span><br><span> :members:</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+pySim TLV utilities</span><br><span style="color: hsl(120, 100%, 40%);">+-------------------</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+.. automodule:: pySim.tlv</span><br><span style="color: hsl(120, 100%, 40%);">+ :members:</span><br><span> </span><br><span> pySim utility functions</span><br><span> -----------------------</span><br><span>diff --git a/pySim/cat.py b/pySim/cat.py</span><br><span>new file mode 100644</span><br><span>index 0000000..cd98a81</span><br><span>--- /dev/null</span><br><span>+++ b/pySim/cat.py</span><br><span>@@ -0,0 +1,222 @@</span><br><span style="color: hsl(120, 100%, 40%);">+"""Code related to the Card Application Toolkit (CAT) as described in</span><br><span style="color: hsl(120, 100%, 40%);">+mainly) ETSI TS 102 223, ETSI TS 101 220 and 3GPP TS 31.111."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# (C) 2021 by Harald Welte <laforge@osmocom.org></span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is free software: you can redistribute it and/or modify</span><br><span style="color: hsl(120, 100%, 40%);">+# it under the terms of the GNU General Public License as published by</span><br><span style="color: hsl(120, 100%, 40%);">+# the Free Software Foundation, either version 2 of the License, or</span><br><span style="color: hsl(120, 100%, 40%);">+# (at your option) any later version.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is distributed in the hope that it will be useful,</span><br><span style="color: hsl(120, 100%, 40%);">+# but WITHOUT ANY WARRANTY; without even the implied warranty of</span><br><span style="color: hsl(120, 100%, 40%);">+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span><br><span style="color: hsl(120, 100%, 40%);">+# GNU General Public License for more details.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# You should have received a copy of the GNU General Public License</span><br><span style="color: hsl(120, 100%, 40%);">+# along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.tlv import *</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.construct import *</span><br><span style="color: hsl(120, 100%, 40%);">+from construct import *</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# Tag values as per TS 101 220 Table 7.23</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.1</span><br><span style="color: hsl(120, 100%, 40%);">+class Address(COMPR_TLV_IE, tag=0x06):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('ton_npi'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'call_number'/BcdAdapter(Bytes(this._.total_len-1)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.2</span><br><span style="color: hsl(120, 100%, 40%);">+class AlphaIdentifier(COMPR_TLV_IE, tag=0x05):</span><br><span style="color: hsl(120, 100%, 40%);">+ # FIXME: like EF.ADN</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.3</span><br><span style="color: hsl(120, 100%, 40%);">+class Subaddress(COMPR_TLV_IE, tag=0x08):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.4</span><br><span style="color: hsl(120, 100%, 40%);">+class CapabilityConfigParams(COMPR_TLV_IE, tag=0x07):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 31.111 Section 8.5</span><br><span style="color: hsl(120, 100%, 40%);">+class CBSPage(COMPR_TLV_IE, tag=0x0C):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.6</span><br><span style="color: hsl(120, 100%, 40%);">+class CommandDetails(COMPR_TLV_IE, tag=0x01):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('command_number'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'type_of_command'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'command_qualifier'/Int8ub)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.7</span><br><span style="color: hsl(120, 100%, 40%);">+class DeviceIdentities(COMPR_TLV_IE, tag=0x02):</span><br><span style="color: hsl(120, 100%, 40%);">+ DEV_IDS = bidict({</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01: 'keypad',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x02: 'display',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x03: 'earpiece',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x10: 'addl_card_reader_0',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x11: 'addl_card_reader_1',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x12: 'addl_card_reader_2',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x13: 'addl_card_reader_3',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x14: 'addl_card_reader_4',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x15: 'addl_card_reader_5',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x16: 'addl_card_reader_6',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x17: 'addl_card_reader_7',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x21: 'channel_1',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x22: 'channel_2',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x23: 'channel_3',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x24: 'channel_4',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x25: 'channel_5',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x26: 'channel_6',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x27: 'channel_7',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x31: 'ecat_client_1',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x32: 'ecat_client_2',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x33: 'ecat_client_3',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x34: 'ecat_client_4',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x35: 'ecat_client_5',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x36: 'ecat_client_6',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x37: 'ecat_client_7',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x38: 'ecat_client_8',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x39: 'ecat_client_9',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x3a: 'ecat_client_a',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x3b: 'ecat_client_b',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x3c: 'ecat_client_c',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x3d: 'ecat_client_d',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x3e: 'ecat_client_e',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x3f: 'ecat_client_f',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x81: 'uicc',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x82: 'terminal',</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x83: 'network',</span><br><span style="color: hsl(120, 100%, 40%);">+ })</span><br><span style="color: hsl(120, 100%, 40%);">+ def _from_bytes(self, do:bytes):</span><br><span style="color: hsl(120, 100%, 40%);">+ return {'source_dev_id': self.DEV_IDS[do[0]], 'dest_dev_id': self.DEV_IDS[do[1]]}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _to_bytes(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ src = self.DEV_IDS.inverse[self.decoded['source_dev_id']]</span><br><span style="color: hsl(120, 100%, 40%);">+ dst = self.DEV_IDS.inverse[self.decoded['dest_dev_id']]</span><br><span style="color: hsl(120, 100%, 40%);">+ return bytes([src, dst])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.8</span><br><span style="color: hsl(120, 100%, 40%);">+class Duration(COMPR_TLV_IE, tag=0x04):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('time_unit'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'time_interval'/Int8ub)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.9</span><br><span style="color: hsl(120, 100%, 40%);">+class Item(COMPR_TLV_IE, tag=0x0f):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('identifier'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'text_string'/GsmStringAdapter(GreedyBytes))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.10</span><br><span style="color: hsl(120, 100%, 40%);">+class ItemIdentifier(COMPR_TLV_IE, tag=0x10):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('identifier'/Int8ub)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.11</span><br><span style="color: hsl(120, 100%, 40%);">+class ResponseLength(COMPR_TLV_IE, tag=0x11):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('minimum_length'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'maximum_length'/Int8ub)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.12</span><br><span style="color: hsl(120, 100%, 40%);">+class Result(COMPR_TLV_IE, tag=0x03):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('general_result'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'additional_information'/HexAdapter(GreedyBytes))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.13 + TS 31.111 Section 8.13</span><br><span style="color: hsl(120, 100%, 40%);">+class SMS_TPDU(COMPR_TLV_IE, tag=0x0B):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.15</span><br><span style="color: hsl(120, 100%, 40%);">+class TextString(COMPR_TLV_IE, tag=0x0d):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('dcs'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'text_string'/HexAdapter(GreedyBytes))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 102 223 Section 8.16</span><br><span style="color: hsl(120, 100%, 40%);">+class Tone(COMPR_TLV_IE, tag=0x0e):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('tone'/Int8ub)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 31 111 Section 8.17</span><br><span style="color: hsl(120, 100%, 40%);">+class USSDString(COMPR_TLV_IE, tag=0x0a):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = Struct('dcs'/Int8ub,</span><br><span style="color: hsl(120, 100%, 40%);">+ 'ussd_string'/HexAdapter(GreedyBytes))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 101 220 Table 7.17</span><br><span style="color: hsl(120, 100%, 40%);">+class ProactiveCommand(BER_TLV_IE, tag=0xD0):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 101 220 Table 7.17 + 31.111 7.1.1.2</span><br><span style="color: hsl(120, 100%, 40%);">+class SMSPPDownload(BER_TLV_IE, tag=0xD1,</span><br><span style="color: hsl(120, 100%, 40%);">+ nested=[DeviceIdentities, Address, SMS_TPDU]):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# TS 101 220 Table 7.17 + 31.111 7.1.1.3</span><br><span style="color: hsl(120, 100%, 40%);">+class SMSCBDownload(BER_TLV_IE, tag=0xD2,</span><br><span style="color: hsl(120, 100%, 40%);">+ nested=[DeviceIdentities, CBSPage]):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class USSDDownload(BER_TLV_IE, tag=0xD9,</span><br><span style="color: hsl(120, 100%, 40%);">+ nested=[DeviceIdentities, USSDString]):</span><br><span style="color: hsl(120, 100%, 40%);">+ pass</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+term_prof_bits = {</span><br><span style="color: hsl(120, 100%, 40%);">+ 1: 'Profile download',</span><br><span style="color: hsl(120, 100%, 40%);">+ 2: 'SMS-PP data doanload',</span><br><span style="color: hsl(120, 100%, 40%);">+ 3: 'Cell Broadcast data download',</span><br><span style="color: hsl(120, 100%, 40%);">+ 4: 'Menu selection',</span><br><span style="color: hsl(120, 100%, 40%);">+ 5: 'SMS-PP data download',</span><br><span style="color: hsl(120, 100%, 40%);">+ 6: 'Timer expiration',</span><br><span style="color: hsl(120, 100%, 40%);">+ 7: 'USSD string DO support in CC by USIM',</span><br><span style="color: hsl(120, 100%, 40%);">+ 8: 'Call Control by NAA',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ 9: 'Command result',</span><br><span style="color: hsl(120, 100%, 40%);">+ 10: 'Call Controll by NAA',</span><br><span style="color: hsl(120, 100%, 40%);">+ 11: 'Call Control by NAA',</span><br><span style="color: hsl(120, 100%, 40%);">+ 12: 'MO short message control support',</span><br><span style="color: hsl(120, 100%, 40%);">+ 13: 'Call Control by NAA',</span><br><span style="color: hsl(120, 100%, 40%);">+ 14: 'UCS2 Entry supported',</span><br><span style="color: hsl(120, 100%, 40%);">+ 15: 'UCS2 Display supported',</span><br><span style="color: hsl(120, 100%, 40%);">+ 16: 'Display Text',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ 17: 'Proactive UICC: DISPLAY TEXT',</span><br><span style="color: hsl(120, 100%, 40%);">+ 18: 'Proactive UICC: GET INKEY',</span><br><span style="color: hsl(120, 100%, 40%);">+ 19: 'Proactive UICC: GET INPUT',</span><br><span style="color: hsl(120, 100%, 40%);">+ 20: 'Proactive UICC: MORE TIME',</span><br><span style="color: hsl(120, 100%, 40%);">+ 21: 'Proactive UICC: PLAY TONE',</span><br><span style="color: hsl(120, 100%, 40%);">+ 22: 'Proactive UICC: POLL INTERVAL',</span><br><span style="color: hsl(120, 100%, 40%);">+ 23: 'Proactive UICC: POLLING OFF',</span><br><span style="color: hsl(120, 100%, 40%);">+ 24: 'Proactive UICC: REFRESH',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ 25: 'Proactive UICC: SELECT ITEM',</span><br><span style="color: hsl(120, 100%, 40%);">+ 26: 'Proactive UICC: SEND SHORT MESSAGE with 3GPP-SMS-TPDU',</span><br><span style="color: hsl(120, 100%, 40%);">+ 27: 'Proactive UICC: SEND SS',</span><br><span style="color: hsl(120, 100%, 40%);">+ 28: 'Proactive UICC: SEND USSD',</span><br><span style="color: hsl(120, 100%, 40%);">+ 29: 'Proactive UICC: SET UP CALL',</span><br><span style="color: hsl(120, 100%, 40%);">+ 30: 'Proactive UICC: SET UP MENU',</span><br><span style="color: hsl(120, 100%, 40%);">+ 31: 'Proactive UICC: PROVIDE LOCAL INFORMATION (MCC, MNC, LAC, Cell ID & IMEI)',</span><br><span style="color: hsl(120, 100%, 40%);">+ 32: 'Proactive UICC: PROVIDE LOCAL INFORMATION (NMR)',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ 33: 'Proactive UICC: SET UP EVENT LIST',</span><br><span style="color: hsl(120, 100%, 40%);">+ 34: 'Event: MT call',</span><br><span style="color: hsl(120, 100%, 40%);">+ 35: 'Event: Call connected',</span><br><span style="color: hsl(120, 100%, 40%);">+ 36: 'Event: Call disconnected',</span><br><span style="color: hsl(120, 100%, 40%);">+ 37: 'Event: Location status',</span><br><span style="color: hsl(120, 100%, 40%);">+ 38: 'Event: User activity',</span><br><span style="color: hsl(120, 100%, 40%);">+ 39: 'Event: Idle screen available',</span><br><span style="color: hsl(120, 100%, 40%);">+ 40: 'Event: Card reader status',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ 41: 'Event: Language selection',</span><br><span style="color: hsl(120, 100%, 40%);">+ 42: 'Event: Browser Termination',</span><br><span style="color: hsl(120, 100%, 40%);">+ 43: 'Event: Data aailable',</span><br><span style="color: hsl(120, 100%, 40%);">+ 44: 'Event: Channel status',</span><br><span style="color: hsl(120, 100%, 40%);">+ 45: 'Event: Access Technology Change',</span><br><span style="color: hsl(120, 100%, 40%);">+ 46: 'Event: Display parameters changed',</span><br><span style="color: hsl(120, 100%, 40%);">+ 47: 'Event: Local Connection',</span><br><span style="color: hsl(120, 100%, 40%);">+ 48: 'Event: Network Search Mode Change',</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # FIXME: remainder</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span>diff --git a/pySim/tlv.py b/pySim/tlv.py</span><br><span>new file mode 100644</span><br><span>index 0000000..3bb1828</span><br><span>--- /dev/null</span><br><span>+++ b/pySim/tlv.py</span><br><span>@@ -0,0 +1,390 @@</span><br><span style="color: hsl(120, 100%, 40%);">+"""object-oriented TLV parser/encoder library."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# (C) 2021 by Harald Welte <laforge@osmocom.org></span><br><span style="color: hsl(120, 100%, 40%);">+# All Rights Reserved</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is free software: you can redistribute it and/or modify</span><br><span style="color: hsl(120, 100%, 40%);">+# it under the terms of the GNU General Public License as published by</span><br><span style="color: hsl(120, 100%, 40%);">+# the Free Software Foundation, either version 2 of the License, or</span><br><span style="color: hsl(120, 100%, 40%);">+# (at your option) any later version.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# This program is distributed in the hope that it will be useful,</span><br><span style="color: hsl(120, 100%, 40%);">+# but WITHOUT ANY WARRANTY; without even the implied warranty of</span><br><span style="color: hsl(120, 100%, 40%);">+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span><br><span style="color: hsl(120, 100%, 40%);">+# GNU General Public License for more details.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# You should have received a copy of the GNU General Public License</span><br><span style="color: hsl(120, 100%, 40%);">+# along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from typing import Optional, List, Dict, Any, Tuple</span><br><span style="color: hsl(120, 100%, 40%);">+from bidict import bidict</span><br><span style="color: hsl(120, 100%, 40%);">+from construct import *</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.utils import bertlv_encode_len, bertlv_parse_len, bertlv_encode_tag, bertlv_parse_tag</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.utils import comprehensiontlv_encode_tag, comprehensiontlv_parse_tag</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.utils import bertlv_parse_one, comprehensiontlv_parse_one</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.utils import bertlv_parse_tag_raw, comprehensiontlv_parse_tag_raw</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.construct import parse_construct, LV, HexAdapter, BcdAdapter, BitsRFU, GsmStringAdapter</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.exceptions import *</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import inspect</span><br><span style="color: hsl(120, 100%, 40%);">+import abc</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class TlvMeta(abc.ABCMeta):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Metaclass which we use to set some class variables at the time of defining a subclass.</span><br><span style="color: hsl(120, 100%, 40%);">+ This allows us to create subclasses for each TLV/IE type, where the class represents fixed</span><br><span style="color: hsl(120, 100%, 40%);">+ parameters like the tag/type and instances of it represent the actual TLV data."""</span><br><span style="color: hsl(120, 100%, 40%);">+ def __new__(metacls, name, bases, namespace, **kwargs):</span><br><span style="color: hsl(120, 100%, 40%);">+ #print("TlvMeta_new_(metacls=%s, name=%s, bases=%s, namespace=%s, kwargs=%s)" % (metacls, name, bases, namespace, kwargs))</span><br><span style="color: hsl(120, 100%, 40%);">+ x = super().__new__(metacls, name, bases, namespace)</span><br><span style="color: hsl(120, 100%, 40%);">+ # this becomes a _class_ variable, not an instance variable</span><br><span style="color: hsl(120, 100%, 40%);">+ x.tag = namespace.get('tag', kwargs.get('tag', None))</span><br><span style="color: hsl(120, 100%, 40%);">+ x.desc = namespace.get('desc', kwargs.get('desc', None))</span><br><span style="color: hsl(120, 100%, 40%);">+ nested = namespace.get('nested', kwargs.get('nested', None))</span><br><span style="color: hsl(120, 100%, 40%);">+ if nested is None or inspect.isclass(nested) and issubclass(nested, TLV_IE_Collection):</span><br><span style="color: hsl(120, 100%, 40%);">+ # caller has specified TLV_IE_Collection sub-class, we can directly reference it</span><br><span style="color: hsl(120, 100%, 40%);">+ x.nested_collection_cls = nested</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ # caller passed list of other TLV classes that might possibly appear within us,</span><br><span style="color: hsl(120, 100%, 40%);">+ # build a dynamically-created TLV_IE_Collection sub-class and reference it</span><br><span style="color: hsl(120, 100%, 40%);">+ name = 'auto_collection_%s' % (name)</span><br><span style="color: hsl(120, 100%, 40%);">+ cls = type(name, (TLV_IE_Collection,), {'nested': nested})</span><br><span style="color: hsl(120, 100%, 40%);">+ x.nested_collection_cls = cls</span><br><span style="color: hsl(120, 100%, 40%);">+ return x</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class TlvCollectionMeta(abc.ABCMeta):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Metaclass which we use to set some class variables at the time of defining a subclass.</span><br><span style="color: hsl(120, 100%, 40%);">+ This allows us to create subclasses for each Collection type, where the class represents fixed</span><br><span style="color: hsl(120, 100%, 40%);">+ parameters like the nested IE classes and instances of it represent the actual TLV data."""</span><br><span style="color: hsl(120, 100%, 40%);">+ def __new__(metacls, name, bases, namespace, **kwargs):</span><br><span style="color: hsl(120, 100%, 40%);">+ #print("TlvCollectionMeta_new_(metacls=%s, name=%s, bases=%s, namespace=%s, kwargs=%s)" % (metacls, name, bases, namespace, kwargs))</span><br><span style="color: hsl(120, 100%, 40%);">+ x = super().__new__(metacls, name, bases, namespace)</span><br><span style="color: hsl(120, 100%, 40%);">+ # this becomes a _class_ variable, not an instance variable</span><br><span style="color: hsl(120, 100%, 40%);">+ x.possible_nested = namespace.get('nested', kwargs.get('nested', None))</span><br><span style="color: hsl(120, 100%, 40%);">+ return x</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class Transcodable(abc.ABC):</span><br><span style="color: hsl(120, 100%, 40%);">+ _construct = None</span><br><span style="color: hsl(120, 100%, 40%);">+ """Base class for something that can be encoded + encoded. Decoding and Encoding happens either</span><br><span style="color: hsl(120, 100%, 40%);">+ * via a 'construct' object stored in a derived class' _construct variable, or</span><br><span style="color: hsl(120, 100%, 40%);">+ * via a 'construct' object stored in an instance _construct variable, or</span><br><span style="color: hsl(120, 100%, 40%);">+ * via a derived class' _{to,from}_bytes() methods."""</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.encoded = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self.decoded = None</span><br><span style="color: hsl(120, 100%, 40%);">+ self._construct = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_bytes(self) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Convert from internal representation to binary bytes. Store the binary result</span><br><span style="color: hsl(120, 100%, 40%);">+ in the internal state and return it."""</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._construct:</span><br><span style="color: hsl(120, 100%, 40%);">+ do = self._construct.build(self.decoded, total_len=None)</span><br><span style="color: hsl(120, 100%, 40%);">+ elif self.__class__._construct:</span><br><span style="color: hsl(120, 100%, 40%);">+ do = self.__class__._construct.build(self.decoded, total_len=None)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ do = self._to_bytes()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.encoded = do</span><br><span style="color: hsl(120, 100%, 40%);">+ return do</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # not an abstractmethod, as it is only required if no _construct exists</span><br><span style="color: hsl(120, 100%, 40%);">+ def _to_bytes(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ raise NotImplementedError</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def from_bytes(self, do:bytes):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Convert from binary bytes to internal representation. Store the decoded result</span><br><span style="color: hsl(120, 100%, 40%);">+ in the internal state and return it."""</span><br><span style="color: hsl(120, 100%, 40%);">+ self.encoded = do</span><br><span style="color: hsl(120, 100%, 40%);">+ if self._construct:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.decoded = parse_construct(self._construct, do)</span><br><span style="color: hsl(120, 100%, 40%);">+ elif self.__class__._construct:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.decoded = parse_construct(self.__class__._construct, do)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.decoded = self._from_bytes(do)</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.decoded</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # not an abstractmethod, as it is only required if no _construct exists</span><br><span style="color: hsl(120, 100%, 40%);">+ def _from_bytes(self, do:bytes):</span><br><span style="color: hsl(120, 100%, 40%);">+ raise NotImplementedError</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class IE(Transcodable, metaclass=TlvMeta):</span><br><span style="color: hsl(120, 100%, 40%);">+ # we specify the metaclass so any downstream subclasses will automatically use it</span><br><span style="color: hsl(120, 100%, 40%);">+ """Base class for various Information Elements. We understand the notion of a hierarchy</span><br><span style="color: hsl(120, 100%, 40%);">+ of IEs on top of the Transcodable class."""</span><br><span style="color: hsl(120, 100%, 40%);">+ # this is overridden by the TlvMeta metaclass, if it is used to create subclasses</span><br><span style="color: hsl(120, 100%, 40%);">+ nested_collection_cls = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, **kwargs):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.nested_collection = None</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.nested_collection_cls:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.nested_collection = self.nested_collection_cls()</span><br><span style="color: hsl(120, 100%, 40%);">+ # if we are a constructed IE, [ordered] list of actual child-IE instances</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = kwargs.get('children', [])</span><br><span style="color: hsl(120, 100%, 40%);">+ self.decoded = kwargs.get('decoded', None)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __repr__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Return a string representing the [nested] IE data (for print)."""</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(self.children):</span><br><span style="color: hsl(120, 100%, 40%);">+ member_strs = [repr(x) for x in self.children]</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s(%s)' % (type(self).__name__, ','.join(member_strs))</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s(%s)' % (type(self).__name__, self.decoded)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_dict(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Return a JSON-serializable dict representing the [nested] IE data."""</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(self.children):</span><br><span style="color: hsl(120, 100%, 40%);">+ v = [x.to_dict() for x in self.children]</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ v = self.decoded</span><br><span style="color: hsl(120, 100%, 40%);">+ return {type(self).__name__: v}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def from_dict(self, decoded:dict):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Set the IE internal decoded representation to data from the argument.</span><br><span style="color: hsl(120, 100%, 40%);">+ If this is a nested IE, the child IE instance list is re-created."""</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.nested_collection:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = self.nested_collection.from_dict(decoded)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = []</span><br><span style="color: hsl(120, 100%, 40%);">+ self.decoded = decoded</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def is_constructed(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Is this IE constructed by further nested IEs?"""</span><br><span style="color: hsl(120, 100%, 40%);">+ if len(self.children):</span><br><span style="color: hsl(120, 100%, 40%);">+ return True</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ return False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @abc.abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_ie(self) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Convert the internal representation to entire IE including IE header."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_bytes(self) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Convert the internal representation _of the value part_ to binary bytes."""</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.is_constructed():</span><br><span style="color: hsl(120, 100%, 40%);">+ # concatenate the encoded IE of all children to form the value part</span><br><span style="color: hsl(120, 100%, 40%);">+ out = b''</span><br><span style="color: hsl(120, 100%, 40%);">+ for c in self.children:</span><br><span style="color: hsl(120, 100%, 40%);">+ out += c.to_ie()</span><br><span style="color: hsl(120, 100%, 40%);">+ return out</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ return super().to_bytes()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def from_bytes(self, do:bytes):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Parse _the value part_ from binary bytes to internal representation."""</span><br><span style="color: hsl(120, 100%, 40%);">+ if self.nested_collection:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = self.nested_collection.from_bytes(do)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = []</span><br><span style="color: hsl(120, 100%, 40%);">+ return super().from_bytes(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class TLV_IE(IE):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Abstract base class for various TLV type Information Elements."""</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, **kwargs):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__(**kwargs)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _compute_tag(self) -> int:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Compute the tag (sometimes the tag encodes part of the value)."""</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.tag</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ @abc.abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _parse_tag_raw(cls, do:bytes) -> Tuple[int, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Obtain the raw TAG at the start of the bytes provided by the user."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ @abc.abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _parse_len(cls, do:bytes) -> Tuple[int, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Obtain the length encoded at the start of the bytes provided by the user."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @abc.abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode_tag(self) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Encode the tag part. Must be provided by derived (TLV format specific) class."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @abc.abstractmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode_len(self, val:bytes) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Encode the length part assuming a certain binary value. Must be provided by</span><br><span style="color: hsl(120, 100%, 40%);">+ derived (TLV format specific) class."""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_ie(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.to_tlv()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_tlv(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Convert the internal representation to binary TLV bytes."""</span><br><span style="color: hsl(120, 100%, 40%);">+ val = self.to_bytes()</span><br><span style="color: hsl(120, 100%, 40%);">+ return self._encode_tag() + self._encode_len(val) + val</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def from_tlv(self, do:bytes):</span><br><span style="color: hsl(120, 100%, 40%);">+ (rawtag, remainder) = self.__class__._parse_tag_raw(do)</span><br><span style="color: hsl(120, 100%, 40%);">+ if rawtag:</span><br><span style="color: hsl(120, 100%, 40%);">+ if rawtag != self.tag:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise ValueError("%s: Encountered tag %s doesn't match our supported tag %s" %</span><br><span style="color: hsl(120, 100%, 40%);">+ (self, rawtag, self.tag))</span><br><span style="color: hsl(120, 100%, 40%);">+ (length, remainder) = self.__class__._parse_len(remainder)</span><br><span style="color: hsl(120, 100%, 40%);">+ value = remainder[:length]</span><br><span style="color: hsl(120, 100%, 40%);">+ remainder = remainder[length:]</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ value = do</span><br><span style="color: hsl(120, 100%, 40%);">+ remainder = b''</span><br><span style="color: hsl(120, 100%, 40%);">+ dec = self.from_bytes(value)</span><br><span style="color: hsl(120, 100%, 40%);">+ return dec, remainder</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class BER_TLV_IE(TLV_IE):</span><br><span style="color: hsl(120, 100%, 40%);">+ """TLV_IE formatted as ASN.1 BER described in ITU-T X.690 8.1.2."""</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, **kwargs):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__(**kwargs)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _decode_tag(cls, do:bytes) -> Tuple[dict, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ return bertlv_parse_tag(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _parse_tag_raw(cls, do:bytes) -> Tuple[int, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ return bertlv_parse_tag_raw(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _parse_len(cls, do:bytes) -> Tuple[int, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ return bertlv_parse_len(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode_tag(self) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ return bertlv_encode_tag(self._compute_tag())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode_len(self, val:bytes) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ return bertlv_encode_len(len(val))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class COMPR_TLV_IE(TLV_IE):</span><br><span style="color: hsl(120, 100%, 40%);">+ """TLV_IE formated as COMPREHENSION-TLV as described in ETSI TS 101 220."""</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, **kwargs):</span><br><span style="color: hsl(120, 100%, 40%);">+ super().__init__(**kwargs)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.comprehension = False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _decode_tag(cls, do:bytes) -> Tuple[dict, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ return comprehensiontlv_parse_tag(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _parse_tag_raw(cls, do:bytes) -> Tuple[int, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ return comprehensiontlv_parse_tag_raw(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @classmethod</span><br><span style="color: hsl(120, 100%, 40%);">+ def _parse_len(cls, do:bytes) -> Tuple[int, bytes]:</span><br><span style="color: hsl(120, 100%, 40%);">+ return bertlv_parse_len(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode_tag(self) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ return comprehensiontlv_encode_tag(self._compute_tag())</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _encode_len(self, val:bytes) -> bytes:</span><br><span style="color: hsl(120, 100%, 40%);">+ return bertlv_encode_len(len(val))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class TLV_IE_Collection(metaclass=TlvCollectionMeta):</span><br><span style="color: hsl(120, 100%, 40%);">+ # we specify the metaclass so any downstream subclasses will automatically use it</span><br><span style="color: hsl(120, 100%, 40%);">+ """A TLV_IE_Collection consists of multiple TLV_IE classes identified by their tags.</span><br><span style="color: hsl(120, 100%, 40%);">+ A given encoded DO may contain any of them in any order, and may contain multiple instances</span><br><span style="color: hsl(120, 100%, 40%);">+ of each DO."""</span><br><span style="color: hsl(120, 100%, 40%);">+ # this is overridden by the TlvCollectionMeta metaclass, if it is used to create subclasses</span><br><span style="color: hsl(120, 100%, 40%);">+ possible_nested = []</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, desc=None, **kwargs):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.desc = desc</span><br><span style="color: hsl(120, 100%, 40%);">+ #print("possible_nested: ", self.possible_nested)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.members = kwargs.get('nested', self.possible_nested)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.members_by_tag = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ self.members_by_name = {}</span><br><span style="color: hsl(120, 100%, 40%);">+ self.members_by_tag = { m.tag:m for m in self.members }</span><br><span style="color: hsl(120, 100%, 40%);">+ self.members_by_name = { m.__name__:m for m in self.members }</span><br><span style="color: hsl(120, 100%, 40%);">+ # if we are a constructed IE, [ordered] list of actual child-IE instances</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = kwargs.get('children', [])</span><br><span style="color: hsl(120, 100%, 40%);">+ self.encoded = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __str__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ member_strs = [str(x) for x in self.members]</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s(%s)' % (type(self).__name__, ','.join(member_strs))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __repr__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ member_strs = [repr(x) for x in self.members]</span><br><span style="color: hsl(120, 100%, 40%);">+ return '%s(%s)' % (self.__class__, ','.join(member_strs))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def __add__(self, other):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Extending TLV_IE_Collections with other TLV_IE_Collections or TLV_IEs."""</span><br><span style="color: hsl(120, 100%, 40%);">+ if isinstance(other, DataObjectCollection):</span><br><span style="color: hsl(120, 100%, 40%);">+ # adding one collection to another</span><br><span style="color: hsl(120, 100%, 40%);">+ members = self.members + other.members</span><br><span style="color: hsl(120, 100%, 40%);">+ return DataObjectCollection(self.name, self.desc, members)</span><br><span style="color: hsl(120, 100%, 40%);">+ elif inspect.isclass(other) and issubclass(other, TLV_IE):</span><br><span style="color: hsl(120, 100%, 40%);">+ # adding a member to a collection</span><br><span style="color: hsl(120, 100%, 40%);">+ return TLV_IE_Collection(self.desc, self.members + [other])</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise TypeError</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def from_bytes(self, binary:bytes) -> List[TLV_IE]:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Create a list of TLV_IEs from the collection based on binary input data.</span><br><span style="color: hsl(120, 100%, 40%);">+ Args:</span><br><span style="color: hsl(120, 100%, 40%);">+ binary : binary bytes of encoded data</span><br><span style="color: hsl(120, 100%, 40%);">+ Returns:</span><br><span style="color: hsl(120, 100%, 40%);">+ list of instances of TLV_IE sub-classes containing parsed data</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ self.encoded = binary</span><br><span style="color: hsl(120, 100%, 40%);">+ # list of instances of TLV_IE collection member classes appearing in the data</span><br><span style="color: hsl(120, 100%, 40%);">+ res = []</span><br><span style="color: hsl(120, 100%, 40%);">+ remainder = binary</span><br><span style="color: hsl(120, 100%, 40%);">+ first = next(iter(self.members_by_tag.values()))</span><br><span style="color: hsl(120, 100%, 40%);">+ # iterate until no binary trailer is left</span><br><span style="color: hsl(120, 100%, 40%);">+ while len(remainder):</span><br><span style="color: hsl(120, 100%, 40%);">+ # obtain the tag at the start of the remainder</span><br><span style="color: hsl(120, 100%, 40%);">+ tag, r = first._parse_tag_raw(remainder)</span><br><span style="color: hsl(120, 100%, 40%);">+ if tag in self.members_by_tag:</span><br><span style="color: hsl(120, 100%, 40%);">+ cls = self.members_by_tag[tag]</span><br><span style="color: hsl(120, 100%, 40%);">+ # create an instance and parse accordingly</span><br><span style="color: hsl(120, 100%, 40%);">+ inst = cls()</span><br><span style="color: hsl(120, 100%, 40%);">+ dec, remainder = inst.from_tlv(remainder)</span><br><span style="color: hsl(120, 100%, 40%);">+ res.append(inst)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ # unknown tag; create the related class on-the-fly using the same base class</span><br><span style="color: hsl(120, 100%, 40%);">+ name = 'unknown_%s_%X' % (first.__base__.__name__, tag)</span><br><span style="color: hsl(120, 100%, 40%);">+ cls = type(name, (first.__base__,), {'tag':tag, 'possible_nested':[],</span><br><span style="color: hsl(120, 100%, 40%);">+ 'nested_collection_cls':None})</span><br><span style="color: hsl(120, 100%, 40%);">+ cls._from_bytes = lambda s, a : {'raw': a.hex()}</span><br><span style="color: hsl(120, 100%, 40%);">+ cls._to_bytes = lambda s: bytes.fromhex(s.decoded['raw'])</span><br><span style="color: hsl(120, 100%, 40%);">+ # create an instance and parse accordingly</span><br><span style="color: hsl(120, 100%, 40%);">+ inst = cls()</span><br><span style="color: hsl(120, 100%, 40%);">+ dec, remainder = inst.from_tlv(remainder)</span><br><span style="color: hsl(120, 100%, 40%);">+ res.append(inst)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = res</span><br><span style="color: hsl(120, 100%, 40%);">+ return res</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def from_dict(self, decoded:List[dict]) -> List[TLV_IE]:</span><br><span style="color: hsl(120, 100%, 40%);">+ """Create a list of TLV_IE instances from the collection based on an array</span><br><span style="color: hsl(120, 100%, 40%);">+ of dicts, where they key indicates the name of the TLV_IE subclass to use."""</span><br><span style="color: hsl(120, 100%, 40%);">+ # list of instances of TLV_IE collection member classes appearing in the data</span><br><span style="color: hsl(120, 100%, 40%);">+ res = []</span><br><span style="color: hsl(120, 100%, 40%);">+ for i in decoded:</span><br><span style="color: hsl(120, 100%, 40%);">+ for k in i.keys():</span><br><span style="color: hsl(120, 100%, 40%);">+ if k in self.members_by_name:</span><br><span style="color: hsl(120, 100%, 40%);">+ cls = self.members_by_name[k]</span><br><span style="color: hsl(120, 100%, 40%);">+ inst = cls(decoded=i[k])</span><br><span style="color: hsl(120, 100%, 40%);">+ res.append(inst)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise ValueErorr('%s: Unknown TLV Class %s in %s; expected %s' %</span><br><span style="color: hsl(120, 100%, 40%);">+ (self, i[0], decoded, self.members_by_name.keys()))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.children = res</span><br><span style="color: hsl(120, 100%, 40%);">+ return res</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_dict(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return [x.to_dict() for x in self.children]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_bytes(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ out = b''</span><br><span style="color: hsl(120, 100%, 40%);">+ for c in self.children:</span><br><span style="color: hsl(120, 100%, 40%);">+ out += c.to_tlv()</span><br><span style="color: hsl(120, 100%, 40%);">+ return out</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def from_tlv(self, do):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.from_bytes(do)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def to_tlv(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ return self.to_bytes()</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/pysim/+/24451">change 24451</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/pysim/+/24451"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: pysim </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I7fc1699443bc9d8a4e7cdd2687af9af7cc03c30e </div>
<div style="display:none"> Gerrit-Change-Number: 24451 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: laforge <laforge@osmocom.org> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>