<p>dexter has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/pysim/+/25552">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">pySim-shell: add bulk provisioning support<br><br>There are scenarios where multiple cards need to get the same change.<br>Lets add a new command that takes a script as parameter and executes the<br>secript in a loop on multiple cards while prompting the user to change<br>the card before starting the next cycle.<br><br>Change-Id: I9e4926675c5a497a22fc6a4fefdd388fe18a2b2d<br>Related: SYS#5617<br>---<br>M pySim-shell.py<br>1 file changed, 182 insertions(+), 4 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/52/25552/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/pySim-shell.py b/pySim-shell.py</span><br><span>index 6a49080..1ca5cb2 100755</span><br><span>--- a/pySim-shell.py</span><br><span>+++ b/pySim-shell.py</span><br><span>@@ -30,6 +30,7 @@</span><br><span> import os</span><br><span> import sys</span><br><span> from pathlib import Path</span><br><span style="color: hsl(120, 100%, 40%);">+from io import StringIO</span><br><span> </span><br><span> from pySim.ts_51_011 import EF, DF, EF_SST_map</span><br><span> from pySim.ts_31_102 import EF_UST_map, EF_USIM_ADF_map</span><br><span>@@ -41,7 +42,7 @@</span><br><span> from pySim.cards import card_detect, SimCard</span><br><span> from pySim.utils import h2b, swap_nibbles, rpad, b2h, h2s, JsonEncoder, bertlv_parse_one</span><br><span> from pySim.utils import dec_st, sanitize_pin_adm, tabulate_str_list, is_hex, boxed_heading_str</span><br><span style="color: hsl(0, 100%, 40%);">-from pySim.card_handler import CardHandler</span><br><span style="color: hsl(120, 100%, 40%);">+from pySim.card_handler import CardHandler, CardHandlerAuto</span><br><span> </span><br><span> from pySim.filesystem import CardMF, RuntimeState, CardDF, CardADF</span><br><span> from pySim.ts_51_011 import CardProfileSIM, DF_TELECOM, DF_GSM</span><br><span>@@ -92,7 +93,7 @@</span><br><span> </span><br><span> class PysimApp(cmd2.Cmd):</span><br><span> CUSTOM_CATEGORY = 'pySim Commands'</span><br><span style="color: hsl(0, 100%, 40%);">- def __init__(self, card, rs, script = None):</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self, card, rs, sl, ch, script = None):</span><br><span> basic_commands = []</span><br><span> super().__init__(persistent_history_file='~/.pysim_shell_history', allow_cli_args=False,</span><br><span> use_ipython=True, auto_load_commands=False, startup_script=script)</span><br><span>@@ -102,6 +103,8 @@</span><br><span> self.conserve_write = True</span><br><span> self.json_pretty_print = True</span><br><span> self.apdu_trace = False</span><br><span style="color: hsl(120, 100%, 40%);">+ self.sl = sl</span><br><span style="color: hsl(120, 100%, 40%);">+ self.ch = ch</span><br><span> </span><br><span> self.add_settable(cmd2.Settable('numeric_path', bool, 'Print File IDs instead of names',</span><br><span> onchange_cb=self._onchange_numeric_path))</span><br><span>@@ -122,6 +125,8 @@</span><br><span> and commands to enable card operations.</span><br><span> """</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ rc = False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> # Unequip everything from pySim-shell that would not work in unequipped state</span><br><span> if self.rs:</span><br><span> self.rs.unregister_cmds(self)</span><br><span>@@ -146,10 +151,12 @@</span><br><span> onchange_cb=self._onchange_conserve_write))</span><br><span> self.iccid, sw = self.card.read_iccid()</span><br><span> rs.select('MF', self)</span><br><span style="color: hsl(120, 100%, 40%);">+ rc = True</span><br><span> else:</span><br><span> self.poutput("pySim-shell not equipped!")</span><br><span> </span><br><span> self.update_prompt()</span><br><span style="color: hsl(120, 100%, 40%);">+ return rc</span><br><span> </span><br><span> def poutput_json(self, data, force_no_pretty = False):</span><br><span> """like cmd2.poutput() but for a JSON serializable dict."""</span><br><span>@@ -197,6 +204,170 @@</span><br><span> rs, card = init_card(sl);</span><br><span> self.equip(card, rs)</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ class InterceptStderr(list):</span><br><span style="color: hsl(120, 100%, 40%);">+ def __init__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self._stderr_backup = sys.stderr</span><br><span style="color: hsl(120, 100%, 40%);">+ def __enter__(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self._stringio_stderr = StringIO()</span><br><span style="color: hsl(120, 100%, 40%);">+ sys.stderr = self._stringio_stderr</span><br><span style="color: hsl(120, 100%, 40%);">+ return self</span><br><span style="color: hsl(120, 100%, 40%);">+ def __exit__(self, *args):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.stderr = self._stringio_stderr.getvalue().strip()</span><br><span style="color: hsl(120, 100%, 40%);">+ del self._stringio_stderr</span><br><span style="color: hsl(120, 100%, 40%);">+ sys.stderr = self._stderr_backup</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _show_failure_sign(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" +-------------+")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ### +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" +-------------+")</span><br><span style="color: hsl(120, 100%, 40%);">+ print("")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _show_success_sign(self):</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" +-------------+")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + # ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## # +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" + ## +")</span><br><span style="color: hsl(120, 100%, 40%);">+ print(" +-------------+")</span><br><span style="color: hsl(120, 100%, 40%);">+ print("")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def _process_card(self, first, script_path):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Eearly phase of card initialzation (this part may fail with an exception)</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ rs, card = init_card(self.sl)</span><br><span style="color: hsl(120, 100%, 40%);">+ rc = self.equip(card, rs)</span><br><span style="color: hsl(120, 100%, 40%);">+ except:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("Card initialization failed with an execption:")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+ traceback.print_exc()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ return -1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Actual card processing step. This part should never fail with an exception since the cmd2</span><br><span style="color: hsl(120, 100%, 40%);">+ # do_run_script method will catch any exception that might occur during script execution.</span><br><span style="color: hsl(120, 100%, 40%);">+ if rc:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("Transcript stdout:")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+ with self.InterceptStderr() as logged:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.do_run_script(script_path)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("Transcript stderr:")</span><br><span style="color: hsl(120, 100%, 40%);">+ if logged.stderr:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput(logged.stderr)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("(none)")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Check for exceptions</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ if "EXCEPTION of type" not in logged.stderr:</span><br><span style="color: hsl(120, 100%, 40%);">+ return 0</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ return -1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ bulk_script_parser = argparse.ArgumentParser()</span><br><span style="color: hsl(120, 100%, 40%);">+ bulk_script_parser.add_argument('script_path', help="path to the script file")</span><br><span style="color: hsl(120, 100%, 40%);">+ bulk_script_parser.add_argument('--halt_on_error', help='stop card handling if an exeption occurs',</span><br><span style="color: hsl(120, 100%, 40%);">+ action='store_true')</span><br><span style="color: hsl(120, 100%, 40%);">+ bulk_script_parser.add_argument('--tries', type=int, default=2,</span><br><span style="color: hsl(120, 100%, 40%);">+ help='how many tries before trying the next card')</span><br><span style="color: hsl(120, 100%, 40%);">+ bulk_script_parser.add_argument('--on_stop_action', type=str, default=None,</span><br><span style="color: hsl(120, 100%, 40%);">+ help='commandline to execute when card handling has stopped')</span><br><span style="color: hsl(120, 100%, 40%);">+ bulk_script_parser.add_argument('--pre_card_action', type=str, default=None,</span><br><span style="color: hsl(120, 100%, 40%);">+ help='commandline to execute before actually talking to the card')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ @cmd2.with_argparser(bulk_script_parser)</span><br><span style="color: hsl(120, 100%, 40%);">+ @cmd2.with_category(CUSTOM_CATEGORY)</span><br><span style="color: hsl(120, 100%, 40%);">+ def do_bulk_script(self, opts):</span><br><span style="color: hsl(120, 100%, 40%);">+ """Run script on multiple cards (bulk provisioning)"""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Make sure that the script file exists and that it is readable.</span><br><span style="color: hsl(120, 100%, 40%);">+ if not os.access(opts.script_path, os.R_OK):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("Invalid script file!")</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ success_count = 0</span><br><span style="color: hsl(120, 100%, 40%);">+ fail_count = 0</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ first = True</span><br><span style="color: hsl(120, 100%, 40%);">+ while 1:</span><br><span style="color: hsl(120, 100%, 40%);">+ # TODO: Count consecutive failures, if more than N consecutive failures occur, then stop.</span><br><span style="color: hsl(120, 100%, 40%);">+ # The ratinale is: There may be a problem with the device, we do want to prevent that</span><br><span style="color: hsl(120, 100%, 40%);">+ # all remaining cards are fired to the error bin. This is only relevant for situations</span><br><span style="color: hsl(120, 100%, 40%);">+ # with large stacks, probably we do not need this feature right now.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ # In case of failure, try multiple times.</span><br><span style="color: hsl(120, 100%, 40%);">+ for i in range(opts.tries):</span><br><span style="color: hsl(120, 100%, 40%);">+ # fetch card into reader bay</span><br><span style="color: hsl(120, 100%, 40%);">+ ch.get(first)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # if necessary execute an action before we start processing the card</span><br><span style="color: hsl(120, 100%, 40%);">+ if(opts.pre_card_action):</span><br><span style="color: hsl(120, 100%, 40%);">+ os.system(opts.pre_card_action)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # process the card</span><br><span style="color: hsl(120, 100%, 40%);">+ rc = self._process_card(first, opts.script_path)</span><br><span style="color: hsl(120, 100%, 40%);">+ if rc == 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ success_count = success_count + 1</span><br><span style="color: hsl(120, 100%, 40%);">+ self._show_success_sign()</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Statistics: success :%i, failure: %i" % (success_count, fail_count))</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ fail_count = fail_count + 1</span><br><span style="color: hsl(120, 100%, 40%);">+ self._show_failure_sign()</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Statistics: success :%i, failure: %i" % (success_count, fail_count))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Depending on success or failure, the card goes either in the "error" bin or in the</span><br><span style="color: hsl(120, 100%, 40%);">+ # "done" bin.</span><br><span style="color: hsl(120, 100%, 40%);">+ if rc < 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ ch.error()</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ ch.done()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # In most cases it is possible to proceed with the next card, but the</span><br><span style="color: hsl(120, 100%, 40%);">+ # user may decide to halt immediately when an error occurs</span><br><span style="color: hsl(120, 100%, 40%);">+ if opts.halt_on_error and rc < 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ except (KeyboardInterrupt):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("Terminated by user!")</span><br><span style="color: hsl(120, 100%, 40%);">+ return;</span><br><span style="color: hsl(120, 100%, 40%);">+ except (SystemExit):</span><br><span style="color: hsl(120, 100%, 40%);">+ # When all cards are processed the card handler device will throw a SystemExit</span><br><span style="color: hsl(120, 100%, 40%);">+ # execption. Also Errors that are not recoverable (cards stuck etc.) will end up here.</span><br><span style="color: hsl(120, 100%, 40%);">+ # The user has the option to execute some action to make aware that the card handler</span><br><span style="color: hsl(120, 100%, 40%);">+ # needs service.</span><br><span style="color: hsl(120, 100%, 40%);">+ if(opts.on_stop_action):</span><br><span style="color: hsl(120, 100%, 40%);">+ os.system(opts.on_stop_action)</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+ except:</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("Card handling failed with an execption:")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+ traceback.print_exc()</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("---------------------8<---------------------")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.poutput("")</span><br><span style="color: hsl(120, 100%, 40%);">+ fail_count = fail_count + 1</span><br><span style="color: hsl(120, 100%, 40%);">+ self._show_failure_sign()</span><br><span style="color: hsl(120, 100%, 40%);">+ print("Statistics: success :%i, failure: %i" % (success_count, fail_count))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ first = False</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> echo_parser = argparse.ArgumentParser()</span><br><span> echo_parser.add_argument('string', help="string to echo on the shell")</span><br><span> </span><br><span>@@ -548,6 +719,8 @@</span><br><span> global_group.add_argument('--script', metavar='PATH', default=None,</span><br><span> help='script with pySim-shell commands to be executed automatically at start-up')</span><br><span> global_group.add_argument('--csv', metavar='FILE', default=None, help='Read card data from CSV file')</span><br><span style="color: hsl(120, 100%, 40%);">+global_group.add_argument("--card_handler", dest="card_handler_config", metavar="FILE",</span><br><span style="color: hsl(120, 100%, 40%);">+ help="Use automatic card handling machine")</span><br><span> </span><br><span> adm_group = global_group.add_mutually_exclusive_group()</span><br><span> adm_group.add_argument('-a', '--pin-adm', metavar='PIN_ADM1', dest='pin_adm', default=None,</span><br><span>@@ -583,13 +756,18 @@</span><br><span> # Create command layer</span><br><span> scc = SimCardCommands(transport=sl)</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ # Create a card handler (for bulk provisioning)</span><br><span style="color: hsl(120, 100%, 40%);">+ if opts.card_handler_config:</span><br><span style="color: hsl(120, 100%, 40%);">+ ch = CardHandlerAuto(None, opts.card_handler_config)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ ch = CardHandler(sl)</span><br><span> </span><br><span> # Detect and initalize the card in the reader. This may fail when there</span><br><span> # is no card in the reader or the card is unresponsive. PysimApp is</span><br><span> # able to tolerate and recover from that.</span><br><span> try:</span><br><span> rs, card = init_card(sl)</span><br><span style="color: hsl(0, 100%, 40%);">- app = PysimApp(card, rs, opts.script)</span><br><span style="color: hsl(120, 100%, 40%);">+ app = PysimApp(card, rs, sl, ch, opts.script)</span><br><span> except:</span><br><span> print("Card initalization failed with an execption:")</span><br><span> print("---------------------8<---------------------")</span><br><span>@@ -599,7 +777,7 @@</span><br><span> print(" it should also be noted that some readers may behave strangely when no card")</span><br><span> print(" is inserted.)")</span><br><span> print("")</span><br><span style="color: hsl(0, 100%, 40%);">- app = PysimApp(None, None, opts.script)</span><br><span style="color: hsl(120, 100%, 40%);">+ app = PysimApp(None, None, sl, ch, opts.script)</span><br><span> </span><br><span> # If the user supplies an ADM PIN at via commandline args authenticate</span><br><span> # immediately so that the user does not have to use the shell commands</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/pysim/+/25552">change 25552</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/pysim/+/25552"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: pysim </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I9e4926675c5a497a22fc6a4fefdd388fe18a2b2d </div>
<div style="display:none"> Gerrit-Change-Number: 25552 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: dexter <pmaier@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>