November 2025 - gerrit-log - lists.osmocom.org

[S] Change in pysim[master]: pySim-shell: add command to manually query the Card Key Provider
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41467?usp=email ) Change subject: pySim-shell: add command to manually query the Card Key Provider ...................................................................... pySim-shell: add command to manually query the Card Key Provider The Card Key Provider is a built in mechanism of pySim-shell which allows the user to read key material from a CSV file in order to avoid having to lookup and enter the key material himself. The lookup normally done by the pySim-shell commands automatically. However, in some cases it may also be useful to be able to query the CSV file manually in order to get certain fields displayed. Such a command is in particular helpful to check and diagnose the CSV data source. Related: SYS#7725 Change-Id: I76e0f883572a029bdca65a5a6b3eef306db1c221 --- M pySim-shell.py 1 file changed, 19 insertions(+), 1 deletion(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved diff --git a/pySim-shell.py b/pySim-shell.py index c629656..878aea0 100755 --- a/pySim-shell.py +++ b/pySim-shell.py @@ -69,7 +69,8 @@ from pySim.gsm_r import DF_EIRENE from pySim.cat import ProactiveCommand -from pySim.card_key_provider import CardKeyProviderCsv, card_key_provider_register, card_key_provider_get_field +from pySim.card_key_provider import CardKeyProviderCsv +from pySim.card_key_provider import card_key_provider_register, card_key_provider_get_field, card_key_provider_get from pySim.app import init_card @@ -498,6 +499,23 @@ """Echo (print) a string on the console""" self.poutput(' '.join(opts.STRING)) + query_card_key_parser = argparse.ArgumentParser() + query_card_key_parser.add_argument('FIELDS', help="fields to query", type=str, nargs='+') + query_card_key_parser.add_argument('--key', help='lookup key (typically \'ICCID\' or \'EID\')', + type=str, required=True) + query_card_key_parser.add_argument('--value', help='lookup key match value (e.g \'8988211000000123456\')', + type=str, required=True) + @cmd2.with_argparser(query_card_key_parser) + @cmd2.with_category(CUSTOM_CATEGORY) + def do_query_card_key(self, opts): + """Manually query the Card Key Provider""" + result = card_key_provider_get(opts.FIELDS, opts.key, opts.value) + self.poutput("Result:") + if result == {}: + self.poutput(" (none)") + for k in result.keys(): + self.poutput(" %s: %s" % (str(k), str(result.get(k)))) + @cmd2.with_category(CUSTOM_CATEGORY) def do_version(self, opts): """Print the pySim software version.""" -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41467?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I76e0f883572a029bdca65a5a6b3eef306db1c221 Gerrit-Change-Number: 41467 Gerrit-PatchSet: 5 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[S] Change in pysim[master]: pySim-shell: re-organize Card Key Provider related options
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41468?usp=email ) Change subject: pySim-shell: re-organize Card Key Provider related options ...................................................................... pySim-shell: re-organize Card Key Provider related options As we plan to support other formats as data source for the Card Key Provider soon, the more commandline options may be added and it makes sense to group the Card Key Provider options in a dedicated group. Let's also rename the option "--csv-column-key" to just "--column-key". The column encryption is a generic concept and not CSV format specific. (let's silently keep the "--csv-column-key" argument so maintain backward compatibility) Related: SYS#7725 Change-Id: I5093f8383551f8c9b84342ca6674c1ebdbbfc19c --- M pySim-shell.py 1 file changed, 14 insertions(+), 12 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved diff --git a/pySim-shell.py b/pySim-shell.py index 878aea0..a8b15d5 100755 --- a/pySim-shell.py +++ b/pySim-shell.py @@ -1127,10 +1127,6 @@ global_group = option_parser.add_argument_group('General Options') global_group.add_argument('--script', metavar='PATH', default=None, help='script with pySim-shell commands to be executed automatically at start-up') -global_group.add_argument('--csv', metavar='FILE', - default=None, help='Read card data from CSV file') -global_group.add_argument('--csv-column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append', - help='per-CSV-column AES transport key') global_group.add_argument("--card_handler", dest="card_handler_config", metavar="FILE", help="Use automatic card handling machine") global_group.add_argument("--noprompt", help="Run in non interactive mode", @@ -1140,6 +1136,15 @@ global_group.add_argument("--verbose", help="Enable verbose logging", action='store_true', default=False) +card_key_group = option_parser.add_argument_group('Card Key Provider Options') +card_key_group.add_argument('--csv', metavar='FILE', + default=str(Path.home()) + "/.osmocom/pysim/card_data.csv", + help='Read card data from CSV file') +card_key_group.add_argument('--csv-column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append', + help=argparse.SUPPRESS, dest='column_key') +card_key_group.add_argument('--column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append', + help='per-column AES transport key', dest='column_key') + adm_group = global_group.add_mutually_exclusive_group() adm_group.add_argument('-a', '--pin-adm', metavar='PIN_ADM1', dest='pin_adm', default=None, help='ADM PIN used for provisioning (overwrites default)') @@ -1168,15 +1173,12 @@ # Register csv-file as card data provider, either from specified CSV # or from CSV file in home directory - csv_column_keys = {} - for par in opts.csv_column_key: + column_keys = {} + for par in opts.column_key: name, key = par.split(':') - csv_column_keys[name] = key - csv_default = str(Path.home()) + "/.osmocom/pysim/card_data.csv" - if opts.csv: - card_key_provider_register(CardKeyProviderCsv(opts.csv, csv_column_keys)) - if os.path.isfile(csv_default): - card_key_provider_register(CardKeyProviderCsv(csv_default, csv_column_keys)) + column_keys[name] = key + if os.path.isfile(opts.csv): + card_key_provider_register(CardKeyProviderCsv(opts.csv, column_keys)) # Init card reader driver sl = init_reader(opts, proactive_handler = Proact()) -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41468?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I5093f8383551f8c9b84342ca6674c1ebdbbfc19c Gerrit-Change-Number: 41468 Gerrit-PatchSet: 6 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[S] Change in pysim[master]: card_key_provider: remove method _verify_get_data from base class
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41480?usp=email ) Change subject: card_key_provider: remove method _verify_get_data from base class ...................................................................... card_key_provider: remove method _verify_get_data from base class The method _verify_get_data was intended to be used to verify the user input before it further processed but ended up to be a simple check that only checks the name of the key column very basically. Unfortunately it is difficult to generalize the check code as the concrete implementation of those checks is highly format dependent. With the advent of eUICCs, we now have two data formats with different lookup keys, so a static list with valid lookup keys is also no longer up to the task. After all it makes not much sense to keep this method, so let's remove it. (From the technical perspective, the key column is not limitied to any specif field. In theory it would even be possible to use the KI as lookup key as well, even though it would not make sense in practice) Related: SYS#7725 Change-Id: Ibf5745fb8a4f927397adff33900731524715d6a9 --- M pySim/card_key_provider.py 1 file changed, 0 insertions(+), 22 deletions(-) Approvals: laforge: Looks good to me, approved Jenkins Builder: Verified diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index 0827b70..cdd871f 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -132,26 +132,6 @@ class CardKeyProvider(abc.ABC): """Base class, not containing any concrete implementation.""" - VALID_KEY_FIELD_NAMES = ['ICCID', 'EID', 'IMSI' ] - - # check input parameters, but do nothing concrete yet - def _verify_get_data(self, fields: List[str] = [], key: str = 'ICCID', value: str = "") -> Dict[str, str]: - """Verify multiple fields for identified card. - - Args: - fields : list of valid field names such as 'ADM1', 'PIN1', ... which are to be obtained - key : look-up key to identify card data, such as 'ICCID' - value : value for look-up key to identify card data - Returns: - dictionary of {field, value} strings for each requested field from 'fields' - """ - - if key not in self.VALID_KEY_FIELD_NAMES: - raise ValueError("Key field name '%s' is not a valid field name, valid field names are: %s" % - (key, str(self.VALID_KEY_FIELD_NAMES))) - - return {} - def get_field(self, field: str, key: str = 'ICCID', value: str = "") -> Optional[str]: """get a single field from CSV file using a specified key/value pair""" fields = [field] @@ -186,8 +166,6 @@ self.crypt = CardKeyFieldCryptor(transport_keys) def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]: - super()._verify_get_data(fields, key, value) - self.csv_file.seek(0) cr = csv.DictReader(self.csv_file) if not cr: -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41480?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Ibf5745fb8a4f927397adff33900731524715d6a9 Gerrit-Change-Number: 41480 Gerrit-PatchSet: 2 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[XS] Change in pysim[master]: pySim-shell: use log level INFO by default
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41479?usp=email ) Change subject: pySim-shell: use log level INFO by default ...................................................................... pySim-shell: use log level INFO by default The default log level of the PySimLogger is DEBUG by default. This is to ensure that all messages are printed in an unconfigured setup. However in pySim-Shell we care about configuring the logger, so let's set the debug log level to INFO in startup. This will allow us to turn debug messages on and off using the verbose switch. Change-Id: I89315f830ce1cc2d573887de4f4cf4e19d17543b Related: SYS#7725 --- M pySim-shell.py 1 file changed, 4 insertions(+), 1 deletion(-) Approvals: laforge: Looks good to me, approved Jenkins Builder: Verified diff --git a/pySim-shell.py b/pySim-shell.py index 8b0b58e..c629656 100755 --- a/pySim-shell.py +++ b/pySim-shell.py @@ -229,7 +229,7 @@ if new == True: PySimLogger.set_level(logging.DEBUG) else: - PySimLogger.set_level() + PySimLogger.set_level(logging.INFO) class Cmd2ApduTracer(ApduTracer): def __init__(self, cmd2_app): @@ -1144,6 +1144,9 @@ if (opts.verbose): PySimLogger.set_verbose(True) PySimLogger.set_level(logging.DEBUG) + else: + PySimLogger.set_verbose(False) + PySimLogger.set_level(logging.INFO) # Register csv-file as card data provider, either from specified CSV # or from CSV file in home directory -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41479?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I89315f830ce1cc2d573887de4f4cf4e19d17543b Gerrit-Change-Number: 41479 Gerrit-PatchSet: 2 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[S] Change in pysim[master]: card_key_provider: use case-insensitive field names
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41464?usp=email ) Change subject: card_key_provider: use case-insensitive field names ...................................................................... card_key_provider: use case-insensitive field names It is common in CSV files that the columns have uppercase names, so we have adopted this scheme when we started using the card_key_provider. This also means that the API of the card_key_provider_get() and card_key_provider_get_field() function now implicitly requires uppercase field names like 'ICCID', 'ADM1', etc. Unfortunately this may be unreliable, so let's convert the field names to uppercase as soon as we receive them. This makes the API case-insensitive and gives us the assurance that all field names we ever work with are in uppercase. Related: SYS#7725 Change-Id: I9d80752587e2ccff0963c10abd5a2f42f5868d79 --- M pySim/card_key_provider.py M tests/unittests/test_card_key_provider.py 2 files changed, 13 insertions(+), 0 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index 561418f..837249d 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -231,6 +231,8 @@ Returns: dictionary of {field, value} strings for each requested field from 'fields' """ + key = key.upper() + fields = [f.upper() for f in fields] for p in provider_list: if not isinstance(p, CardKeyProvider): raise ValueError( @@ -252,6 +254,8 @@ Returns: dictionary of {field, value} strings for the requested field """ + key = key.upper() + field = field.upper() for p in provider_list: if not isinstance(p, CardKeyProvider): raise ValueError( diff --git a/tests/unittests/test_card_key_provider.py b/tests/unittests/test_card_key_provider.py index bbe76e8..121998c 100644 --- a/tests/unittests/test_card_key_provider.py +++ b/tests/unittests/test_card_key_provider.py @@ -66,6 +66,11 @@ "KIC2","KIC3","KID1","KID2","KID3","KIK1","KIK2","KIK3","OPC"], "ICCID", t.get('ICCID')) self.assertEqual(result, t.get('EXPECTED')) + result = card_key_provider_get(["PIN1","puk1","PIN2","PUK2","KI","adm1","ADM2","KIC1", + "KIC2","kic3","KID1","KID2","KID3","kik1","KIK2","KIK3","OPC"], + "iccid", t.get('ICCID')) + self.assertEqual(result, t.get('EXPECTED')) + def test_card_key_provider_get_field(self): test_data = [{'EXPECTED' : "3eb8567fa0b4b1e63bcab13bff5f2702", 'ICCID' :"8988211000000000001"}, @@ -75,6 +80,10 @@ for t in test_data: result = card_key_provider_get_field("KIC1", "ICCID", t.get('ICCID')) self.assertEqual(result, t.get('EXPECTED')) + for t in test_data: + result = card_key_provider_get_field("kic1", "iccid", t.get('ICCID')) + self.assertEqual(result, t.get('EXPECTED')) + class TestCardKeyFieldCryptor(unittest.TestCase): -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41464?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I9d80752587e2ccff0963c10abd5a2f42f5868d79 Gerrit-Change-Number: 41464 Gerrit-PatchSet: 4 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[S] Change in pysim[master]: card_key_provider: rename parameter filename to csv_filename
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41465?usp=email ) Change subject: card_key_provider: rename parameter filename to csv_filename ...................................................................... card_key_provider: rename parameter filename to csv_filename let's rename the parameter filename to csv_filename to make it more clear to what kind of file this parameter refers. Change-Id: Id5b7c61b5e72fb205e30d2787855b2c276840a7b --- M pySim/card_key_provider.py 1 file changed, 7 insertions(+), 7 deletions(-) Approvals: laforge: Looks good to me, approved Jenkins Builder: Verified diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index 837249d..66e96e9 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -176,16 +176,16 @@ csv_file = None filename = None - def __init__(self, filename: str, transport_keys: dict): + def __init__(self, csv_filename: str, transport_keys: dict): """ Args: - filename : file name (path) of CSV file containing card-individual key/data + csv_filename : file name (path) of CSV file containing card-individual key/data transport_keys : (see class CardKeyFieldCryptor) """ - self.csv_file = open(filename, 'r') + self.csv_file = open(csv_filename, 'r') if not self.csv_file: - raise RuntimeError("Could not open CSV file '%s'" % filename) - self.filename = filename + raise RuntimeError("Could not open CSV file '%s'" % csv_filename) + self.csv_filename = csv_filename self.crypt = CardKeyFieldCryptor(transport_keys) def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]: @@ -194,7 +194,7 @@ self.csv_file.seek(0) cr = csv.DictReader(self.csv_file) if not cr: - raise RuntimeError("Could not open DictReader for CSV-File '%s'" % self.filename) + raise RuntimeError("Could not open DictReader for CSV-File '%s'" % self.csv_filename) cr.fieldnames = [field.upper() for field in cr.fieldnames] rc = {} @@ -204,7 +204,7 @@ if f in row: rc.update({f: self.crypt.decrypt_field(f, row[f])}) else: - raise RuntimeError("CSV-File '%s' lacks column '%s'" % (self.filename, f)) + raise RuntimeError("CSV-File '%s' lacks column '%s'" % (self.csv_filename, f)) return rc -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41465?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Id5b7c61b5e72fb205e30d2787855b2c276840a7b Gerrit-Change-Number: 41465 Gerrit-PatchSet: 4 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[M] Change in pysim[master]: card_key_provider: separate and refactor CSV column encryption
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41450?usp=email ) Change subject: card_key_provider: separate and refactor CSV column encryption ...................................................................... card_key_provider: separate and refactor CSV column encryption The CardKeyProviderCsv class implements a column decryption scheme where columns are protected using a transport key. The CSV files are enrcypted using contrib/csv-encrypt-columns.py. The current implementation has two main problems: - The decryption code in CardKeyProviderCsv is not specific to CSV files. It could be re-used in other formats, for example to decrypt columns (fields) red from a database. So let's split the decryption code in a separate class. - The encryption code in csv-encrypt-columns.py accesses methods and properties in CardKeyProviderCsv. Also having the coresponding encryption code somewhere out of tree may be confusing. Let's improve the design and put encryption and decryption functions in a single class. Let's also make sure the encryption/decryption is covered by unittests. Related: SYS#7725 Change-Id: I180457d4938f526d227c81020e4e03c6b3a57dab --- M contrib/csv-encrypt-columns.py M pySim/card_key_provider.py M tests/unittests/test_card_key_provider.py 3 files changed, 165 insertions(+), 60 deletions(-) Approvals: laforge: Looks good to me, approved Jenkins Builder: Verified diff --git a/contrib/csv-encrypt-columns.py b/contrib/csv-encrypt-columns.py index 49340a2..9967005 100755 --- a/contrib/csv-encrypt-columns.py +++ b/contrib/csv-encrypt-columns.py @@ -24,20 +24,12 @@ from Cryptodome.Cipher import AES from osmocom.utils import h2b, b2h, Hexstr -from pySim.card_key_provider import CardKeyProviderCsv +from pySim.card_key_provider import CardKeyFieldCryptor -def dict_keys_to_upper(d: dict) -> dict: - return {k.upper():v for k,v in d.items()} - -class CsvColumnEncryptor: +class CsvColumnEncryptor(CardKeyFieldCryptor): def __init__(self, filename: str, transport_keys: dict): self.filename = filename - self.transport_keys = dict_keys_to_upper(transport_keys) - - def encrypt_col(self, colname:str, value: str) -> Hexstr: - key = self.transport_keys[colname] - cipher = AES.new(h2b(key), AES.MODE_CBC, CardKeyProviderCsv.IV) - return b2h(cipher.encrypt(h2b(value))) + self.crypt = CardKeyFieldCryptor(transport_keys) def encrypt(self) -> None: with open(self.filename, 'r') as infile: @@ -49,9 +41,8 @@ cw.writeheader() for row in cr: - for key_colname in self.transport_keys: - if key_colname in row: - row[key_colname] = self.encrypt_col(key_colname, row[key_colname]) + for fieldname in cr.fieldnames: + row[fieldname] = self.crypt.encrypt_field(fieldname, row[fieldname]) cw.writerow(row) if __name__ == "__main__": @@ -71,9 +62,5 @@ print("You must specify at least one key!") sys.exit(1) - csv_column_keys = CardKeyProviderCsv.process_transport_keys(csv_column_keys) - for name, key in csv_column_keys.items(): - print("Encrypting column %s using AES key %s" % (name, key)) - cce = CsvColumnEncryptor(opts.CSVFILE, csv_column_keys) cce.encrypt() diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index 9f9dc70..561418f 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -10,7 +10,7 @@ operation with pySim-shell. """ -# (C) 2021-2024 by Sysmocom s.f.m.c. GmbH +# (C) 2021-2025 by Sysmocom s.f.m.c. GmbH # All Rights Reserved # # Author: Philipp Maier, Harald Welte @@ -31,23 +31,104 @@ from typing import List, Dict, Optional from Cryptodome.Cipher import AES from osmocom.utils import h2b, b2h +from pySim.log import PySimLogger import abc import csv +import logging + +log = PySimLogger.get("CARDKEY") card_key_providers = [] # type: List['CardKeyProvider'] -# well-known groups of columns relate to a given functionality. This avoids having -# to specify the same transport key N number of times, if the same key is used for multiple -# fields of one group, like KIC+KID+KID of one SD. -CRYPT_GROUPS = { - 'UICC_SCP02': ['UICC_SCP02_KIC1', 'UICC_SCP02_KID1', 'UICC_SCP02_KIK1'], - 'UICC_SCP03': ['UICC_SCP03_KIC1', 'UICC_SCP03_KID1', 'UICC_SCP03_KIK1'], - 'SCP03_ISDR': ['SCP03_ENC_ISDR', 'SCP03_MAC_ISDR', 'SCP03_DEK_ISDR'], - 'SCP03_ISDA': ['SCP03_ENC_ISDR', 'SCP03_MAC_ISDA', 'SCP03_DEK_ISDA'], - 'SCP03_ECASD': ['SCP03_ENC_ECASD', 'SCP03_MAC_ECASD', 'SCP03_DEK_ECASD'], +class CardKeyFieldCryptor: + """ + A Card key field encryption class that may be used by Card key provider implementations to add support for + a column-based encryption to protect sensitive material (cryptographic key material, ADM keys, etc.). + The sensitive material is encrypted using a "key-encryption key", occasionally also known as "transport key" + before it is stored into a file or database (see also GSMA FS.28). The "transport key" is then used to decrypt + the key material on demand. + """ + + # well-known groups of columns relate to a given functionality. This avoids having + # to specify the same transport key N number of times, if the same key is used for multiple + # fields of one group, like KIC+KID+KID of one SD. + __CRYPT_GROUPS = { + 'UICC_SCP02': ['UICC_SCP02_KIC1', 'UICC_SCP02_KID1', 'UICC_SCP02_KIK1'], + 'UICC_SCP03': ['UICC_SCP03_KIC1', 'UICC_SCP03_KID1', 'UICC_SCP03_KIK1'], + 'SCP03_ISDR': ['SCP03_ENC_ISDR', 'SCP03_MAC_ISDR', 'SCP03_DEK_ISDR'], + 'SCP03_ISDA': ['SCP03_ENC_ISDR', 'SCP03_MAC_ISDA', 'SCP03_DEK_ISDA'], + 'SCP03_ECASD': ['SCP03_ENC_ECASD', 'SCP03_MAC_ECASD', 'SCP03_DEK_ECASD'], } + __IV = b'\x23' * 16 + + @staticmethod + def __dict_keys_to_upper(d: dict) -> dict: + return {k.upper():v for k,v in d.items()} + + @staticmethod + def __process_transport_keys(transport_keys: dict, crypt_groups: dict): + """Apply a single transport key to multiple fields/columns, if the name is a group.""" + new_dict = {} + for name, key in transport_keys.items(): + if name in crypt_groups: + for field in crypt_groups[name]: + new_dict[field] = key + else: + new_dict[name] = key + return new_dict + + def __init__(self, transport_keys: dict): + """ + Create new field encryptor/decryptor object and set transport keys, usually one for each column. In some cases + it is also possible to use a single key for multiple columns (see also __CRYPT_GROUPS) + + Args: + transport_keys : a dict indexed by field name, whose values are hex-encoded AES keys for the + respective field (column) of the CSV. This is done so that different fields + (columns) can use different transport keys, which is strongly recommended by + GSMA FS.28 + """ + self.transport_keys = self.__process_transport_keys(self.__dict_keys_to_upper(transport_keys), + self.__CRYPT_GROUPS) + for name, key in self.transport_keys.items(): + log.debug("Encrypting/decrypting field %s using AES key %s" % (name, key)) + + def decrypt_field(self, field_name: str, encrypted_val: str) -> str: + """ + Decrypt a single field. The decryption is only applied if we have a transport key is known under the provided + field name, otherwise the field is treated as plaintext and passed through as it is. + + Args: + field_name : name of the field to decrypt (used to identify which key to use) + encrypted_val : encrypted field value + + Returns: + plaintext field value + """ + if not field_name.upper() in self.transport_keys: + return encrypted_val + cipher = AES.new(h2b(self.transport_keys[field_name.upper()]), AES.MODE_CBC, self.__IV) + return b2h(cipher.decrypt(h2b(encrypted_val))) + + def encrypt_field(self, field_name: str, plaintext_val: str) -> str: + """ + Encrypt a single field. The encryption is only applied if we have a transport key is known under the provided + field name, otherwise the field is treated as non sensitive and passed through as it is. + + Args: + field_name : name of the field to decrypt (used to identify which key to use) + encrypted_val : encrypted field value + + Returns: + plaintext field value + """ + if not field_name.upper() in self.transport_keys: + return plaintext_val + cipher = AES.new(h2b(self.transport_keys[field_name.upper()]), AES.MODE_CBC, self.__IV) + return b2h(cipher.encrypt(h2b(plaintext_val))) + class CardKeyProvider(abc.ABC): """Base class, not containing any concrete implementation.""" @@ -89,13 +170,9 @@ dictionary of {field, value} strings for each requested field from 'fields' """ - class CardKeyProviderCsv(CardKeyProvider): - """Card key provider implementation that allows to query against a specified CSV file. - Supports column-based encryption as it is generally a bad idea to store cryptographic key material in - plaintext. Instead, the key material should be encrypted by a "key-encryption key", occasionally also - known as "transport key" (see GSMA FS.28).""" - IV = b'\x23' * 16 + """Card key provider implementation that allows to query against a specified CSV file.""" + csv_file = None filename = None @@ -103,35 +180,13 @@ """ Args: filename : file name (path) of CSV file containing card-individual key/data - transport_keys : a dict indexed by field name, whose values are hex-encoded AES keys for the - respective field (column) of the CSV. This is done so that different fields - (columns) can use different transport keys, which is strongly recommended by - GSMA FS.28 + transport_keys : (see class CardKeyFieldCryptor) """ self.csv_file = open(filename, 'r') if not self.csv_file: raise RuntimeError("Could not open CSV file '%s'" % filename) self.filename = filename - self.transport_keys = self.process_transport_keys(transport_keys) - - @staticmethod - def process_transport_keys(transport_keys: dict): - """Apply a single transport key to multiple fields/columns, if the name is a group.""" - new_dict = {} - for name, key in transport_keys.items(): - if name in CRYPT_GROUPS: - for field in CRYPT_GROUPS[name]: - new_dict[field] = key - else: - new_dict[name] = key - return new_dict - - def _decrypt_field(self, field_name: str, encrypted_val: str) -> str: - """decrypt a single field, if we have a transport key for the field of that name.""" - if not field_name in self.transport_keys: - return encrypted_val - cipher = AES.new(h2b(self.transport_keys[field_name]), AES.MODE_CBC, self.IV) - return b2h(cipher.decrypt(h2b(encrypted_val))) + self.crypt = CardKeyFieldCryptor(transport_keys) def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]: super()._verify_get_data(fields, key, value) @@ -147,7 +202,7 @@ if row[key] == value: for f in fields: if f in row: - rc.update({f: self._decrypt_field(f, row[f])}) + rc.update({f: self.crypt.decrypt_field(f, row[f])}) else: raise RuntimeError("CSV-File '%s' lacks column '%s'" % (self.filename, f)) return rc diff --git a/tests/unittests/test_card_key_provider.py b/tests/unittests/test_card_key_provider.py index 03148e2..bbe76e8 100644 --- a/tests/unittests/test_card_key_provider.py +++ b/tests/unittests/test_card_key_provider.py @@ -4,7 +4,7 @@ import os from pySim.card_key_provider import * -class TestCardKeyProvider(unittest.TestCase): +class TestCardKeyProviderCsv(unittest.TestCase): def __init__(self, *args, **kwargs): column_keys = {"KI" : "000424252525535532532A0B0C0D0E0F", @@ -76,5 +76,68 @@ result = card_key_provider_get_field("KIC1", "ICCID", t.get('ICCID')) self.assertEqual(result, t.get('EXPECTED')) +class TestCardKeyFieldCryptor(unittest.TestCase): + + def __init__(self, *args, **kwargs): + transport_keys = {"KI" : "000424252525535532532A0B0C0D0E0F", + "OPC" : "000102030405065545645645645D0E0F", + "KIC1" : "06410203546406456456450B0C0D0E0F", + "UICC_SCP03" : "00040267840507667609045645645E0F"} + self.crypt = CardKeyFieldCryptor(transport_keys) + super().__init__(*args, **kwargs) + + def test_encrypt_field(self): + test_data = [{'EXPECTED' : "0b1e1e56cd62645aeb4c2d72a7c98f27", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "OPC"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "NOCRYPT"}, + {'EXPECTED' : "00248276d2734f108f9761e2f98e2a9d", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "UICC_SCP03_KIC1"}, + {'EXPECTED' : "00248276d2734f108f9761e2f98e2a9d", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "UICC_SCP03_KID1"}, + {'EXPECTED' : "00248276d2734f108f9761e2f98e2a9d", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "UICC_SCP03_KIK1"}, + {'EXPECTED' : "0b1e1e56cd62645aeb4c2d72a7c98f27", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "opc"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "nocrypt"}, + {'EXPECTED' : "00248276d2734f108f9761e2f98e2a9d", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "uicc_scp03_kic1"}, + {'EXPECTED' : "00248276d2734f108f9761e2f98e2a9d", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "uicc_scp03_kid1"}, + {'EXPECTED' : "00248276d2734f108f9761e2f98e2a9d", + 'PLAINTEXT_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "uicc_scp03_kik1"}] + + for t in test_data: + result = self.crypt.encrypt_field(t.get('FIELDNAME'), t.get('PLAINTEXT_VAL')) + self.assertEqual(result, t.get('EXPECTED')) + + def test_decrypt_field(self): + test_data = [{'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "0b1e1e56cd62645aeb4c2d72a7c98f27", 'FIELDNAME' : "OPC"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "NOCRYPT"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "00248276d2734f108f9761e2f98e2a9d", 'FIELDNAME' : "UICC_SCP03_KIC1"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "00248276d2734f108f9761e2f98e2a9d", 'FIELDNAME' : "UICC_SCP03_KID1"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "00248276d2734f108f9761e2f98e2a9d", 'FIELDNAME' : "UICC_SCP03_KIK1"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "0b1e1e56cd62645aeb4c2d72a7c98f27", 'FIELDNAME' : "opc"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "000102030405060708090a0b0c0d0e0f", 'FIELDNAME' : "nocrypt"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "00248276d2734f108f9761e2f98e2a9d", 'FIELDNAME' : "uicc_scp03_kic1"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "00248276d2734f108f9761e2f98e2a9d", 'FIELDNAME' : "uicc_scp03_kid1"}, + {'EXPECTED' : "000102030405060708090a0b0c0d0e0f", + 'ENCRYPTED_VAL' : "00248276d2734f108f9761e2f98e2a9d", 'FIELDNAME' : "uicc_scp03_kik1"}] + + for t in test_data: + result = self.crypt.decrypt_field(t.get('FIELDNAME'), t.get('ENCRYPTED_VAL')) + self.assertEqual(result, t.get('EXPECTED')) + + if __name__ == "__main__": unittest.main() -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41450?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I180457d4938f526d227c81020e4e03c6b3a57dab Gerrit-Change-Number: 41450 Gerrit-PatchSet: 8 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[XS] Change in pysim[master]: card_key_provider: remove unnecessary class property definitions
by laforge 26 Nov '25

26 Nov '25

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/41466?usp=email ) Change subject: card_key_provider: remove unnecessary class property definitions ...................................................................... card_key_provider: remove unnecessary class property definitions The two properties csv_file and csv_filename are defined by the constructor anyway, let's remove the declaration in the class body because it is not needed. Change-Id: Ibbe8e17b03a4ba0041c0e9990a5e9614388d9c03 --- M pySim/card_key_provider.py 1 file changed, 0 insertions(+), 3 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index 66e96e9..0827b70 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -173,9 +173,6 @@ class CardKeyProviderCsv(CardKeyProvider): """Card key provider implementation that allows to query against a specified CSV file.""" - csv_file = None - filename = None - def __init__(self, csv_filename: str, transport_keys: dict): """ Args: -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41466?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Ibbe8e17b03a4ba0041c0e9990a5e9614388d9c03 Gerrit-Change-Number: 41466 Gerrit-PatchSet: 4 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 0

[S] Change in pysim[master]: card_key_provider: remove method _verify_get_data from base class
by laforge 26 Nov '25

26 Nov '25

Attention is currently required from: dexter. laforge has posted comments on this change by dexter. ( https://gerrit.osmocom.org/c/pysim/+/41480?usp=email ) Change subject: card_key_provider: remove method _verify_get_data from base class ...................................................................... Patch Set 2: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41480?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Ibf5745fb8a4f927397adff33900731524715d6a9 Gerrit-Change-Number: 41480 Gerrit-PatchSet: 2 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: dexter <pmaier(a)sysmocom.de> Gerrit-Comment-Date: Wed, 26 Nov 2025 03:37:09 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes

1 0

[S] Change in pysim[master]: pySim-shell: re-organize Card Key Provider related options
by laforge 26 Nov '25

26 Nov '25

Attention is currently required from: dexter. laforge has posted comments on this change by dexter. ( https://gerrit.osmocom.org/c/pysim/+/41468?usp=email ) Change subject: pySim-shell: re-organize Card Key Provider related options ...................................................................... Patch Set 6: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41468?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I5093f8383551f8c9b84342ca6674c1ebdbbfc19c Gerrit-Change-Number: 41468 Gerrit-PatchSet: 6 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: dexter <pmaier(a)sysmocom.de> Gerrit-Comment-Date: Wed, 26 Nov 2025 03:37:06 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes

1 0

2026

2025

2024

2023

2022

gerrit-log November 2025