Jenkins Builder has posted comments on this change by laforge. ( https://gerrit.osmocom.org/c/pysim/+/37856?usp=email )
Change subject: WIP: contrib/fsdump2saip
......................................................................
Patch Set 1:
(1 comment)
File contrib/fsdump2saip.py:
Robot Comment from checkpatch (run ID jenkins-gerrit-lint-17813):
https://gerrit.osmocom.org/c/pysim/+/37856/comment/16d8de16_89b3c1f9?usp=em… :
PS1, Line 23: # Of course, this really only affects the file sytem aspects of the card. It's not possible
'sytem' may be misspelled - perhaps 'system'?
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37856?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I8fc23b4177f229170a6ee99eca8863518196fa51
Gerrit-Change-Number: 37856
Gerrit-PatchSet: 1
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-CC: Jenkins Builder
Gerrit-Comment-Date: Sun, 18 Aug 2024 13:28:04 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Attention is currently required from: fixeria.
laforge has posted comments on this change by laforge. ( https://gerrit.osmocom.org/c/pysim/+/37848?usp=email )
Change subject: pySim.esim.saip: maintain a parsed fileystem hierarchy
......................................................................
Patch Set 4:
(1 comment)
File pySim/esim/saip/__init__.py:
https://gerrit.osmocom.org/c/pysim/+/37848/comment/c945018f_bd760c24?usp=em… :
PS1, Line 100: def __init__(self, pename: str, l: Optional[List[Tuple]] = None, template:
:
> Not really related to this patch, but looks like you want to use `**kwargs` here to pass optional pa […]
I don't get the **kwargs argument, but I've added the name docstring.
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
Gerrit-Change-Number: 37848
Gerrit-PatchSet: 4
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Comment-Date: Sun, 18 Aug 2024 13:27:58 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: fixeria <vyanitskiy(a)sysmocom.de>
Attention is currently required from: fixeria, laforge.
Hello Jenkins Builder,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
to look at the new patch set (#4).
Change subject: pySim.esim.saip: maintain a parsed fileystem hierarchy
......................................................................
pySim.esim.saip: maintain a parsed fileystem hierarchy
With this change, the ProfileElementSequence object will maintain a
representation of the filesystem hierarchy of the eSIM profile. Every
file that is added by a ProfileElement will add a FsNode into that tree,
and each FsNode will point to the File object for the respective file.
This allows us to find files by their path, as well as add files by
path.
Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
---
M pySim/esim/saip/__init__.py
1 file changed, 599 insertions(+), 69 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/48/37848/4
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
Gerrit-Change-Number: 37848
Gerrit-PatchSet: 4
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de>
laforge has uploaded this change for review. ( https://gerrit.osmocom.org/c/pysim/+/37856?usp=email )
Change subject: WIP: contrib/fsdump2saip
......................................................................
WIP: contrib/fsdump2saip
Change-Id: I8fc23b4177f229170a6ee99eca8863518196fa51
---
A contrib/fsdump2saip.py
1 file changed, 202 insertions(+), 0 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/56/37856/1
diff --git a/contrib/fsdump2saip.py b/contrib/fsdump2saip.py
new file mode 100755
index 0000000..7b14313
--- /dev/null
+++ b/contrib/fsdump2saip.py
@@ -0,0 +1,202 @@
+#!/usr/bin/env python3
+
+# (C) 2024 by Harald Welte <laforge(a)osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+# This is a script to generate a [partial] eSIM profile from the 'fsdump' of another USIM/ISIM. This is
+# useful to generate an "as close as possible" eSIM from a physical USIM, as far as that is possible
+# programmatically and in a portable way.
+#
+# Of course, this really only affects the file sytem aspects of the card. It's not possible
+# to read the K/OPc or other authentication related parameters off a random USIM, and hence
+# we cannot replicate that. Similarly, it's not possible to export the java applets from a USIM,
+# and hence we cannot replicate those.
+
+import argparse
+
+from pySim.esim.saip import *
+from pySim.ts_102_221 import *
+
+class FsdumpToSaip:
+ def __init__(self, pes: ProfileElementSequence):
+ self.pes = pes
+
+ @staticmethod
+ def fcp_raw2saip_fcp(fname:str, fcp_raw: bytes) -> Dict:
+ """Convert a raw TLV-encoded FCP to a SAIP dict-format (as needed by asn1encode)."""
+ ftype = fname.split('.')[0]
+ # use the raw FCP as basis so we don't get stuck with potentially old decoder bugs
+ # ore future format incompatibilities
+ fcp = FcpTemplate()
+ fcp.from_tlv(fcp_raw)
+ r = {}
+
+ r['fileDescriptor'] = fcp.child_by_type(FileDescriptor).to_bytes()
+
+ file_id = fcp.child_by_type(FileIdentifier)
+ if file_id:
+ r['fileID'] = file_id.to_bytes()
+ else:
+ if ftype in ['ADF']:
+ print('%s is an ADF but has no [mandatory] file_id!' % fname)
+ #raise ValueError('%s is an ADF but has no [mandatory] file_id!' % fname)
+ r['fileID'] = b'\x7f\xff' # FIXME: auto-increment
+
+ df_name = fcp.child_by_type(DfName)
+ if ftype in ['ADF']:
+ if not df_name:
+ raise ValueError('%s is an ADF but has no [mandatory] df_name!' % fname)
+ r['dfName'] = df_name.to_bytes()
+
+ lcsi_byte = fcp.child_by_type(LifeCycleStatusInteger).to_bytes()
+ if lcsi_byte != b'\x05':
+ r['lcsi'] = lcsi_byte
+
+ sa_ref = fcp.child_by_type(SecurityAttribReferenced)
+ if sa_ref:
+ r['securityAttributesReferenced'] = sa_ref.to_bytes()
+
+ file_size = fcp.child_by_type(FileSize)
+ if ftype in ['EF']:
+ if file_size:
+ r['efFileSize'] = file_size.to_bytes()
+
+ psdo = fcp.child_by_type(PinStatusTemplate_DO)
+ if ftype in ['MF', 'ADF', 'DF']:
+ if not psdo:
+ raise ValueError('%s is an %s but has no [mandatory] PinStatusTemplateDO' % fname)
+ else:
+ r['pinStatusTemplateDO'] = psdo.to_bytes()
+
+ sfid = fcp.child_by_type(ShortFileIdentifier)
+ if sfid and sfid.decoded:
+ if ftype not in ['EF']:
+ raise ValueError('%s is not an EF but has [forbidden] shortEFID' % fname)
+ r['shortEFID'] = sfid.to_bytes()
+
+ pinfo = fcp.child_by_type(ProprietaryInformation)
+ if pinfo and ftype in ['EF']:
+ spinfo = pinfo.child_by_type(SpecialFileInfo)
+ fill_p = pinfo.child_by_type(FillingPattern)
+ repeat_p = pinfo.child_by_type(RepeatPattern)
+
+ if spinfo or fill_p or repeat_p:
+ r['proprietaryEFInfo'] = {}
+ if spinfo:
+ r['proprietaryEFInfo']['specialFileInformation'] = spinfo.to_bytes()
+ if fill_p:
+ r['proprietaryEFInfo']['fillPattern'] = fill_p.to_bytes()
+ if repeat_p:
+ r['proprietaryEFInfo']['repeatPattern'] = repeat_p.to_bytes()
+
+ # TODO: linkPath
+ return r
+
+ @staticmethod
+ def fcp_fsdump2saip(fsdump_ef: Dict):
+ """Convert a file from its "fsdump" representation to the SAIP representation of a File type
+ in the decoded format as used by the asn1tools-generated codec."""
+ # first convert the FCP
+ path = fsdump_ef['path']
+ fdesc = FsdumpToSaip.fcp_raw2saip_fcp(path[-1], h2b(fsdump_ef['fcp_raw']))
+ r = [
+ ('fileDescriptor', fdesc),
+ ]
+ # then convert the body. We're doing a straight-forward conversion without any optimization
+ # like not encoding all-ff files. This should be done by a subsequent optimizer
+ if 'body' in fsdump_ef and fsdump_ef['body']:
+ if isinstance(fsdump_ef['body'], list):
+ for b_seg in fsdump_ef['body']:
+ r.append(('fillFileContent', h2b(b_seg)))
+ else:
+ r.append(('fillFileContent', h2b(fsdump_ef['body'])))
+ print(fsdump_ef['path'])
+ return r
+
+ def add_file_from_fsdump(self, fsdump_ef: Dict):
+ fid = int(fsdump_ef['fcp']['file_identifier'])
+ # determine NAA
+ if fsdump_ef['path'][0:1] == ['MF', 'ADF.USIM']:
+ naa = NaaUsim
+ elif fsdump_ef['path'][0:1] == ['MF', 'ADF.ISIM']:
+ naa = NaaIsim
+ else:
+ print("Unable to determine NAA for %s" % fsdump_ef['path'])
+ return
+ pes.pes_by_naa[naa.name]
+ for pe in pes:
+ print("PE %s" % pe)
+ if not isinstance(pe, FsProfileElement):
+ print("Skipping PE %s" % pe)
+ continue
+ if not pe.template:
+ print("No template for PE %s" % pe )
+ continue
+ if not fid in pe.template.files_by_fid:
+ print("File %04x not available in template; must create it using GenericFileMgmt" % fid)
+
+parser = argparse.ArgumentParser()
+parser.add_argument('fsdump', help='')
+
+def has_unsupported_path_prefix(path: List[str]) -> bool:
+ # skip some paths from export as they don't exist in an eSIM profile
+ UNSUPPORTED_PATHS = [
+ ['MF', 'DF.GSM'],
+ ]
+ for p in UNSUPPORTED_PATHS:
+ prefix = path[:len(p)]
+ if prefix == p:
+ return True
+ # any ADF not USIM or ISIM are unsupported
+ SUPPORTED_ADFS = [ 'ADF.USIM', 'ADF.ISIM' ]
+ if len(path) == 2 and path[0] == 'MF' and path[1].startswith('ADF.') and path[1] not in SUPPORTED_ADFS:
+ return True
+ return False
+
+import traceback
+
+if __name__ == '__main__':
+ opts = parser.parse_args()
+
+ with open(opts.fsdump, 'r') as f:
+ fsdump = json.loads(f.read())
+
+ pes = ProfileElementSequence()
+
+ # FIXME: fsdump has strting-name path, but we need FID-list path for ProfileElementSequence
+ for path, fsdump_ef in fsdump['files'].items():
+ print("=" * 80)
+ #print(fsdump_ef)
+ if not 'fcp_raw' in fsdump_ef:
+ continue
+ if has_unsupported_path_prefix(fsdump_ef['path']):
+ print("Skipping eSIM-unsupported path %s" % ('/'.join(fsdump_ef['path'])))
+ continue
+ saip_dec = FsdumpToSaip.fcp_fsdump2saip(fsdump_ef)
+ #print(saip_dec)
+ try:
+ f = pes.add_file_at_path(Path(path), saip_dec)
+ print(repr(f))
+ except Exception as e:
+ print("EXCEPTION: %s" % traceback.format_exc())
+ #print("EXCEPTION: %s" % e)
+ continue
+
+ print("=== Tree === ")
+ pes.mf.print_tree()
+
+ # FIXME: export the actual PE Sequence
+
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37856?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newchange
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I8fc23b4177f229170a6ee99eca8863518196fa51
Gerrit-Change-Number: 37856
Gerrit-PatchSet: 1
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Attention is currently required from: fixeria, laforge.
Jenkins Builder has posted comments on this change by laforge. ( https://gerrit.osmocom.org/c/pysim/+/37848?usp=email )
Change subject: pySim.esim.saip: maintain a parsed fileystem hierarchy
......................................................................
Patch Set 3:
(2 comments)
File pySim/esim/saip/__init__.py:
Robot Comment from checkpatch (run ID jenkins-gerrit-lint-17804):
https://gerrit.osmocom.org/c/pysim/+/37848/comment/030b33ed_e24c8d78?usp=em… :
PS3, Line 1714: """Create and link an apropriate FsNode for the given 'file' and insert it.
'apropriate' may be misspelled - perhaps 'appropriate'?
Robot Comment from checkpatch (run ID jenkins-gerrit-lint-17804):
https://gerrit.osmocom.org/c/pysim/+/37848/comment/e73e1e14_0b61a9fa?usp=em… :
PS3, Line 1730: logger.debug("%s.add_file(%s) -> return cur_df=%s" % (self, file, cur_df))
trailing whitespace
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
Gerrit-Change-Number: 37848
Gerrit-PatchSet: 3
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Comment-Date: Sun, 18 Aug 2024 13:26:07 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Attention is currently required from: fixeria, laforge.
Hello Jenkins Builder,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
to look at the new patch set (#3).
The following approvals got outdated and were removed:
Verified-1 by Jenkins Builder
Change subject: pySim.esim.saip: maintain a parsed fileystem hierarchy
......................................................................
pySim.esim.saip: maintain a parsed fileystem hierarchy
With this change, the ProfileElementSequence object will maintain a
representation of the filesystem hierarchy of the eSIM profile. Every
file that is added by a ProfileElement will add a FsNode into that tree,
and each FsNode will point to the File object for the respective file.
This allows us to find files by their path, as well as add files by
path.
Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
---
M pySim/esim/saip/__init__.py
1 file changed, 599 insertions(+), 69 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/48/37848/3
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
Gerrit-Change-Number: 37848
Gerrit-PatchSet: 3
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de>
Attention is currently required from: fixeria.
laforge has posted comments on this change by laforge. ( https://gerrit.osmocom.org/c/pysim/+/37848?usp=email )
Change subject: pySim.esim.saip: maintain a parsed fileystem hierarchy
......................................................................
Patch Set 2:
(1 comment)
File pySim/esim/saip/__init__.py:
https://gerrit.osmocom.org/c/pysim/+/37848/comment/bc3d2a43_99325262?usp=em… :
PS1, Line 139: if False:
> Intentional?
yes, at least for now. I clarified it in the updated version.
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
Gerrit-Change-Number: 37848
Gerrit-PatchSet: 2
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Comment-Date: Sun, 18 Aug 2024 13:22:36 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: fixeria <vyanitskiy(a)sysmocom.de>
Attention is currently required from: laforge.
Jenkins Builder has posted comments on this change by laforge. ( https://gerrit.osmocom.org/c/pysim/+/37848?usp=email )
Change subject: pySim.esim.saip: maintain a parsed fileystem hierarchy
......................................................................
Patch Set 2:
(2 comments)
File pySim/esim/saip/__init__.py:
Robot Comment from checkpatch (run ID jenkins-gerrit-lint-17802):
https://gerrit.osmocom.org/c/pysim/+/37848/comment/e4fae582_e9d7a716?usp=em… :
PS2, Line 1713: """Create and link an apropriate FsNode for the given 'file' and insert it.
'apropriate' may be misspelled - perhaps 'appropriate'?
Robot Comment from checkpatch (run ID jenkins-gerrit-lint-17802):
https://gerrit.osmocom.org/c/pysim/+/37848/comment/672f0b04_c66ed85f?usp=em… :
PS2, Line 1729: logger.debug("%s.add_file(%s) -> return cur_df=%s" % (self, file, cur_df))
trailing whitespace
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
Gerrit-Change-Number: 37848
Gerrit-PatchSet: 2
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Gerrit-Comment-Date: Sun, 18 Aug 2024 13:22:12 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Attention is currently required from: laforge.
Hello Jenkins Builder,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
to look at the new patch set (#2).
The following approvals got outdated and were removed:
Verified-1 by Jenkins Builder
Change subject: pySim.esim.saip: maintain a parsed fileystem hierarchy
......................................................................
pySim.esim.saip: maintain a parsed fileystem hierarchy
With this change, the ProfileElementSequence object will maintain a
representation of the filesystem hierarchy of the eSIM profile. Every
file that is added by a ProfileElement will add a FsNode into that tree,
and each FsNode will point to the File object for the respective file.
This allows us to find files by their path, as well as add files by
path.
Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
---
M pySim/esim/saip/__init__.py
1 file changed, 598 insertions(+), 69 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/48/37848/2
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/37848?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
Gerrit-Change-Number: 37848
Gerrit-PatchSet: 2
Gerrit-Owner: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de>
Gerrit-Attention: laforge <laforge(a)osmocom.org>