<p>laforge has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/pysim/+/23661">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">pySim-shell: JSONpath support for updating files/records<br><br>Change-Id: Iad09b3d878b8b58ad34cb549c80f8a6eb3149faa<br>---<br>M docs/shell.rst<br>M pySim/filesystem.py<br>A pySim/jsonpath.py<br>M requirements.txt<br>M setup.py<br>5 files changed, 79 insertions(+), 2 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/61/23661/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/docs/shell.rst b/docs/shell.rst</span><br><span>index 2edebbf..3cfc849 100644</span><br><span>--- a/docs/shell.rst</span><br><span>+++ b/docs/shell.rst</span><br><span>@@ -226,6 +226,33 @@</span><br><span> :module: pySim.filesystem</span><br><span> :func: TransparentEF.ShellCommands.upd_bin_dec_parser</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+In normal operation, update_binary_decoded needs a JSON document representing the entire file contents as</span><br><span style="color: hsl(120, 100%, 40%);">+input. This can be inconvenient if you want to keep 99% of the content but just toggle one specific</span><br><span style="color: hsl(120, 100%, 40%);">+parameter. That's where the JSONpath support comes in handy: You can specify a JSONpath to an element</span><br><span style="color: hsl(120, 100%, 40%);">+inside the document as well as a new value for tat field:</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+Th below example demonstrates this by modifying the ofm field within EF.AD:</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+::</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded</span><br><span style="color: hsl(120, 100%, 40%);">+ {</span><br><span style="color: hsl(120, 100%, 40%);">+ "ms_operation_mode": "normal",</span><br><span style="color: hsl(120, 100%, 40%);">+ "specific_facilities": {</span><br><span style="color: hsl(120, 100%, 40%);">+ "ofm": true</span><br><span style="color: hsl(120, 100%, 40%);">+ },</span><br><span style="color: hsl(120, 100%, 40%);">+ "len_of_mnc_in_imsi": 2</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ pySIM-shell (MF/ADF.USIM/EF.AD)> update_binary_decoded --json-path specific_facilities.ofm false</span><br><span style="color: hsl(120, 100%, 40%);">+ pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded</span><br><span style="color: hsl(120, 100%, 40%);">+ {</span><br><span style="color: hsl(120, 100%, 40%);">+ "ms_operation_mode": "normal",</span><br><span style="color: hsl(120, 100%, 40%);">+ "specific_facilities": {</span><br><span style="color: hsl(120, 100%, 40%);">+ "ofm": false</span><br><span style="color: hsl(120, 100%, 40%);">+ },</span><br><span style="color: hsl(120, 100%, 40%);">+ "len_of_mnc_in_imsi": 2</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> </span><br><span> </span><br><span> cmd2 settable parameters</span><br><span>diff --git a/pySim/filesystem.py b/pySim/filesystem.py</span><br><span>index a65a764..f8443f3 100644</span><br><span>--- a/pySim/filesystem.py</span><br><span>+++ b/pySim/filesystem.py</span><br><span>@@ -35,6 +35,7 @@</span><br><span> </span><br><span> from pySim.utils import sw_match, h2b, b2h, is_hex</span><br><span> from pySim.exceptions import *</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.jsonpath import js_path_find, js_path_modify</span><br><span> </span><br><span> class CardFile(object):</span><br><span> """Base class for all objects in the smart card filesystem.</span><br><span>@@ -418,10 +419,16 @@</span><br><span> </span><br><span> upd_bin_dec_parser = argparse.ArgumentParser()</span><br><span> upd_bin_dec_parser.add_argument('data', help='Abstract data (JSON format) to write')</span><br><span style="color: hsl(120, 100%, 40%);">+ upd_bin_dec_parser.add_argument('--json-path', type=str,</span><br><span style="color: hsl(120, 100%, 40%);">+ help='JSON path to modify specific element of file only')</span><br><span> @cmd2.with_argparser(upd_bin_dec_parser)</span><br><span> def do_update_binary_decoded(self, opts):</span><br><span> """Encode + Update (Write) data of a transparent EF"""</span><br><span style="color: hsl(0, 100%, 40%);">- data_json = json.loads(opts.data)</span><br><span style="color: hsl(120, 100%, 40%);">+ if opts.json_path:</span><br><span style="color: hsl(120, 100%, 40%);">+ (data_json, sw) = self._cmd.rs.read_binary_dec()</span><br><span style="color: hsl(120, 100%, 40%);">+ js_path_modify(data_json, opts.json_path, json.loads(opts.data))</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ data_json = json.loads(opts.data)</span><br><span> (data, sw) = self._cmd.rs.update_binary_dec(data_json)</span><br><span> if data:</span><br><span> self._cmd.poutput_json(data)</span><br><span>@@ -574,10 +581,17 @@</span><br><span> upd_rec_dec_parser = argparse.ArgumentParser()</span><br><span> upd_rec_dec_parser.add_argument('record_nr', type=int, help='Number of record to be read')</span><br><span> upd_rec_dec_parser.add_argument('data', help='Data bytes (hex format) to write')</span><br><span style="color: hsl(120, 100%, 40%);">+ upd_rec_dec_parser.add_argument('--json-path', type=str,</span><br><span style="color: hsl(120, 100%, 40%);">+ help='JSON path to modify specific element of record only')</span><br><span> @cmd2.with_argparser(upd_rec_dec_parser)</span><br><span> def do_update_record_decoded(self, opts):</span><br><span> """Encode + Update (write) data to a record-oriented EF"""</span><br><span style="color: hsl(0, 100%, 40%);">- (data, sw) = self._cmd.rs.update_record_dec(opts.record_nr, opts.data)</span><br><span style="color: hsl(120, 100%, 40%);">+ if opts.json_path:</span><br><span style="color: hsl(120, 100%, 40%);">+ (data_json, sw) = self._cmd.rs.read_record_dec(opts.record_nr)</span><br><span style="color: hsl(120, 100%, 40%);">+ js_path_modify(data_json, opts.json_path, json.loads(opts.data))</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ data_json = json.loads(opts.data)</span><br><span style="color: hsl(120, 100%, 40%);">+ (data, sw) = self._cmd.rs.update_record_dec(opts.record_nr, data_json)</span><br><span> if data:</span><br><span> self._cmd.poutput(data)</span><br><span> </span><br><span>diff --git a/pySim/jsonpath.py b/pySim/jsonpath.py</span><br><span>new file mode 100644</span><br><span>index 0000000..012d1d9</span><br><span>--- /dev/null</span><br><span>+++ b/pySim/jsonpath.py</span><br><span>@@ -0,0 +1,34 @@</span><br><span style="color: hsl(120, 100%, 40%);">+import json</span><br><span style="color: hsl(120, 100%, 40%);">+import pprint</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from jsonpath_ng import jsonpath, parse</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+"""JSONpath utliity functions as needed within pysim.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+As pySim-sell has the ability to represent SIM files as JSON strings,</span><br><span style="color: hsl(120, 100%, 40%);">+adding JSONpath allows us to conveniently modify individual sub-fields</span><br><span style="color: hsl(120, 100%, 40%);">+of a file or record in its JSON representation.</span><br><span style="color: hsl(120, 100%, 40%);">+"""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def js_path_find(js_dict, js_path):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Find/Match a JSON path within a givne JSON-serializable dict.</span><br><span style="color: hsl(120, 100%, 40%);">+ Args:</span><br><span style="color: hsl(120, 100%, 40%);">+ js_dict : JSON-serializable dict to operate on</span><br><span style="color: hsl(120, 100%, 40%);">+ js_path : JSONpath string</span><br><span style="color: hsl(120, 100%, 40%);">+ Returns: Result of the JSONpath expression</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ jsonpath_expr = parse(js_path)</span><br><span style="color: hsl(120, 100%, 40%);">+ return jsonpath_expr.find(js_dict)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def js_path_modify(js_dict, js_path, new_val):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Find/Match a JSON path within a givne JSON-serializable dict.</span><br><span style="color: hsl(120, 100%, 40%);">+ Args:</span><br><span style="color: hsl(120, 100%, 40%);">+ js_dict : JSON-serializable dict to operate on</span><br><span style="color: hsl(120, 100%, 40%);">+ js_path : JSONpath string</span><br><span style="color: hsl(120, 100%, 40%);">+ new_val : New value for field in js_dict at js_path</span><br><span style="color: hsl(120, 100%, 40%);">+ """</span><br><span style="color: hsl(120, 100%, 40%);">+ jsonpath_expr = parse(js_path)</span><br><span style="color: hsl(120, 100%, 40%);">+ jsonpath_expr.find(js_dict)</span><br><span style="color: hsl(120, 100%, 40%);">+ jsonpath_expr.update(js_dict, new_val)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span>diff --git a/requirements.txt b/requirements.txt</span><br><span>index 978a3db..f203ed1 100644</span><br><span>--- a/requirements.txt</span><br><span>+++ b/requirements.txt</span><br><span>@@ -2,3 +2,4 @@</span><br><span> serial</span><br><span> pytlv</span><br><span> cmd2</span><br><span style="color: hsl(120, 100%, 40%);">+jsonpath-ng</span><br><span>diff --git a/setup.py b/setup.py</span><br><span>index d9f742c..a02e327 100644</span><br><span>--- a/setup.py</span><br><span>+++ b/setup.py</span><br><span>@@ -13,6 +13,7 @@</span><br><span> "serial",</span><br><span> "pytlv",</span><br><span> "cmd2"</span><br><span style="color: hsl(120, 100%, 40%);">+ "jsonpath-ng"</span><br><span> ],</span><br><span> scripts=[</span><br><span> 'pySim-prog.py',</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/pysim/+/23661">change 23661</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/pysim/+/23661"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: pysim </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: Iad09b3d878b8b58ad34cb549c80f8a6eb3149faa </div>
<div style="display:none"> Gerrit-Change-Number: 23661 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: laforge <laforge@osmocom.org> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>