From laforge at gnumonks.org Tue Apr 6 22:49:57 2021 From: laforge at gnumonks.org (Harald Welte) Date: Wed, 7 Apr 2021 00:49:57 +0200 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Dear Osmocom community, two months later, a lot of work has been going into pySim-shell. The following features are now available from pysim.git master: * we now have a 'tree' command to list the filesystem hierarchy * you can now select arbitrary files by FID, even those not specified in 3GPP * there is now an 'export' command, which will iterate over all records in all files and dump them in an output format that can be used by pySim-shell itself to restore the data * we have the start of a user manual including a reference to the various pySim-shell comamnds. A preview is available at https://people.osmocom.org/laforge/tmp/pysim-doc-test/html/ * you can now use python setuptools to build and install dependencies, etc. * pySim-shell can retriev the card-specific ADM PIN from a CSV file in your home The latest invention (just pushed to gerrit for review) is the integration of JSONpath support, wich allows you to specify certain parts of a decoded file or record[s] to be updated. If you've never heard of JSONpath: It is to JSON what XPath is to XML. First example on how to use this: ------------------------------------------ pySIM-shell (MF/ADF.USIM/EF.FPLMN)> read_binary_decoded [ { "mcc": 262, "mnc": 42 }, { "mcc": 262, "mnc": 42 }, { "mcc": 262, "mnc": 42 }, { "mcc": 262, "mnc": 42 } ] pySIM-shell (MF/ADF.USIM/EF.FPLMN)> update_binary_decoded --json-path [*] null pySIM-shell (MF/ADF.USIM/EF.FPLMN)> read_binary_decoded [ null, null, null, null ] pySIM-shell (MF/ADF.USIM/EF.FPLMN)> update_binary_decoded --json-path [2] '{"mcc":"262", "mnc":"42"}' pySIM-shell (MF/ADF.USIM/EF.FPLMN)> read_binary_decoded [ null, null, { "mcc": 262, "mnc": 42 }, null ] ------------------------------------------ Second example, illustrating its use in more complex JSON types: ------------------------------------------ pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded { "ms_operation_mode": "normal", "specific_facilities": { "ofm": true }, "len_of_mnc_in_imsi": 2 } pySIM-shell (MF/ADF.USIM/EF.AD)> update_binary_decoded --json-path specific_facilities.ofm false pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded { "ms_operation_mode": "normal", "specific_facilities": { "ofm": false }, "len_of_mnc_in_imsi": 2 } ------------------------------------------ Stay tuned for more pySim developments coming up. Any feedback is welcome - as is help with writing encoder/decoder methods for all the files that don't have any yet. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From mychaela.falconia at gmail.com Wed Apr 7 00:40:54 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Tue, 6 Apr 2021 16:40:54 -0800 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Harald Welte wrote: > * we now have a 'tree' command to list the filesystem hierarchy And just how do you get the card to tell you what selectable file IDs exist? I haven't seen anything like an ls operation in either the classic GSM 11.11 SIM protocol or the UICC protocol, thus the only way (that I know of) to find out what selectable file IDs exist is to do a brute force search of the 16-bit file ID space at every directory level. First select MF, then try selecting every possible 16-bit file ID from 0000 to FFFF (only skipping 3F00 for MF itself), and note which return something other than "not found" error. Follow up with a GET RESPONSE command for every SELECT which succeeded, parse the response, and report the findings. For all found file IDs which turn out to be DFs when the response is parsed, note those DF file IDs, and then repeat the brute force search inside every found DF - and then in any found nested DFs too. This brute force search is implemented in fc-simtool and fc-uicc-tool programs in my fc-sim-tools suite, my competitor to pySim: https://www.freecalypso.org/hg/fc-sim-tools/ As one would naturally expect, such brute force searches are painfully slow - IIRC, bfsearch-mf of sysmoISIM-SJA2 (just the MF tree, ADF trees have to be searched separately with bfsearch-adf) took about an hour, using HID Omnikey 3121 card reader, same model as the one currently sold in Sysmocom webshop - using an o'scope, I observed that it clocks the card at 4.8 MHz, almost up to the spec limit of 5 MHz. Because these brute force searches are so slow, I collect the captures and check them into my source repository under the data directory - so if you are curious to see what undocumented proprietary files exist on both Sysmocom and Grcard SIMs (whose existence cannot be discovered in any other way than this bfsearch), just look in the repository linked above. :-) I am not able to run pySim-shell on my Slackware system without expending more effort than I can currently justify, but I have glanced at the Python code, and I don't see anything like the just described brute force search - nor do I see it issuing any kind of secret undocumented ls-type APDU commands to the card - thus I am guessing that this 'tree' command displays nothing more than the tool's hard-coded knowledge of what files "should" exist at each given directory level, rather than what is actually found to exist. If I got this part wrong, then someone please explain what this command *actually* returns, and how it obtains this knowledge - I don't know of any way other than a brute force search. M~ From domi at tomcsanyi.net Wed Apr 7 07:23:18 2021 From: domi at tomcsanyi.net (Tomcsanyi, Domonkos) Date: Wed, 7 Apr 2021 09:23:18 +0200 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: <3A92FCC2-1F38-4F67-AA2A-ADC5AAD304BB@tomcsanyi.net> Hi Mychaela, Without looking at the code I assume the way it works is that based on the service table the tool knows what files are supposed to be on the card and queries for them - just like an ?ordinary phone? does. The point of the ST is exactly to avoid unnecessary bruteforcing/lookups of files that don?t exist. However it is correct to say that finding a file hidden on purpose (i.e. missing entry from the ST) could only be done via bruteforce imho. In case I am wrong sorry, didn?t mean to mislead anybody. Cheers, Domi > 07.04.2021 d?tummal, 2:41 id?pontban Mychaela Falconia ?rta: > > ?Harald Welte wrote: > >> * we now have a 'tree' command to list the filesystem hierarchy > > And just how do you get the card to tell you what selectable file IDs > exist? I haven't seen anything like an ls operation in either the > classic GSM 11.11 SIM protocol or the UICC protocol, thus the only way > (that I know of) to find out what selectable file IDs exist is to do a > brute force search of the 16-bit file ID space at every directory > level. First select MF, then try selecting every possible 16-bit file > ID from 0000 to FFFF (only skipping 3F00 for MF itself), and note > which return something other than "not found" error. Follow up with a > GET RESPONSE command for every SELECT which succeeded, parse the > response, and report the findings. For all found file IDs which turn > out to be DFs when the response is parsed, note those DF file IDs, and > then repeat the brute force search inside every found DF - and then in > any found nested DFs too. > > This brute force search is implemented in fc-simtool and fc-uicc-tool > programs in my fc-sim-tools suite, my competitor to pySim: > > https://www.freecalypso.org/hg/fc-sim-tools/ > > As one would naturally expect, such brute force searches are painfully > slow - IIRC, bfsearch-mf of sysmoISIM-SJA2 (just the MF tree, ADF trees > have to be searched separately with bfsearch-adf) took about an hour, > using HID Omnikey 3121 card reader, same model as the one currently > sold in Sysmocom webshop - using an o'scope, I observed that it clocks > the card at 4.8 MHz, almost up to the spec limit of 5 MHz. > > Because these brute force searches are so slow, I collect the captures > and check them into my source repository under the data directory - so > if you are curious to see what undocumented proprietary files exist on > both Sysmocom and Grcard SIMs (whose existence cannot be discovered in > any other way than this bfsearch), just look in the repository linked > above. :-) > > I am not able to run pySim-shell on my Slackware system without > expending more effort than I can currently justify, but I have glanced > at the Python code, and I don't see anything like the just described > brute force search - nor do I see it issuing any kind of secret > undocumented ls-type APDU commands to the card - thus I am guessing > that this 'tree' command displays nothing more than the tool's > hard-coded knowledge of what files "should" exist at each given > directory level, rather than what is actually found to exist. If I > got this part wrong, then someone please explain what this command > *actually* returns, and how it obtains this knowledge - I don't know > of any way other than a brute force search. > > M~ From laforge at osmocom.org Wed Apr 7 15:10:47 2021 From: laforge at osmocom.org (Harald Welte) Date: Wed, 7 Apr 2021 17:10:47 +0200 Subject: Announcement of "OsmoDevCall" on April 9, 2021 Message-ID: Dear Osmocom community, It's my pleasure to announce the next OsmoDevCall at April 9, 2021 at 20:00 CET at https://meeting4.franken.de/b/har-xbc-bsx-wvs This meeting will have the following schedule: 20:00 meet + greet 20:15 presentation: pySim-shell - the next generation of pySim 21:00 USSE: unstructured supplementary social event [*] 22:00 close of call Presentation Abstract: For more than a decade, pySim-prog has been the tool to configure/program SIM cards in research/lab/private cellular networks. Originally designed for very simplistic GSM-only SIM Cards, it was extended again and again to cover more use cases and parameters. There is a limit as to how far one can go with stuffing everything into command line arguments. In 2021, pySim-shell was created as the next generation tool. It features interactive navigation around the file system, editing capabilities, backup and restore of all [known] files, ... Attendance is free of charge and open to anyone with an interest in Osmocom. More information about OsmoDevCall, including the schedule for further upcoming events can be found at https://osmocom.org/projects/osmo-dev-con/wiki/OsmoDevCall Looking forward to meeting you on Friday. Best regards, Harald [*] this is how we started to call the "unstructured" part of osmocom developer conferences in the past, basically where anyone can talk about anything, no formal schedule or structure. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From laforge at osmocom.org Sun Apr 18 17:23:13 2021 From: laforge at osmocom.org (Harald Welte) Date: Sun, 18 Apr 2021 19:23:13 +0200 Subject: Announcement of "OsmoDevCall" on April 23, 2021 Message-ID: Dear Osmocom community, It's my pleasure to announce the next OsmoDevCall at April 23, 2021 at 20:00 CEST at https://meeting4.franken.de/b/har-xbc-bsx-wvs This meeting will have the following schedule: 20:00 meet + greet 20:15 presentation by horiz0n: "YIG & YANG (Yet ANother yiG driver)" 21:00 USSE: unstructured supplementary social event [*] 22:00 close of call Presentation Abstract: This talk will briefly introduce the working principles of YIG (Yttrium Iron Garnet) microwave circuits, their applications and finally conclude with a presentation of a recently developed driver circuit. Attendance is free of charge and open to anyone with an interest in Osmocom. More information about OsmoDevCall, including the schedule for further upcoming events can be found at https://osmocom.org/projects/osmo-dev-con/wiki/OsmoDevCall Looking forward to meeting you on Friday. Best regards, Harald [*] this is how we started to call the "unstructured" part of osmocom developer conferences in the past, basically where anyone can talk about anything, no formal schedule or structure. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)