WIP / RFC for pysim 'next generation;' aka pysim-shell
Dear Osmocom community, pySim-prog was nice when there were only 5 parameters on a SIM that we could program, and where the use case was pretty limited. Today, we have SIM/USIM/ISIM cards with hundreds of files and even more parameters to program. We cannot add a command line argument for each file to pySim-prog. Instead, this introduces an interactive command-line shell / REPL, in which one can navigate the file system of the card, read and update files both in raw format and in decoded/parsed format. The idea is primarily inspired by Henryk Ploetz' venerable cyberflex-shell, but implemented on a more modern basis using the cmd2 python module. You can see the very first prototype in the laforge/shell branch of pysim.git You can do things with it like this: ===> Start-up and authenticate with adm pin ---------------------------------------------------------------------- $ ./pysim-shell.py -p 0 Using PC/SC reader interface Autodetected card type: sysmoISIM-SJA2 AIDs on card: ['a0000000871002ffffffff8907090000', 'a0000000871004ffffffff8907090000'] Welcome to pySim-shell! pySIM-shell (3f00)> verify_adm 92990895 ---------------------------------------------------------------------- ===> interactive help ---------------------------------------------------------------------- pySIM-shell (3f00)> help Documented commands (use 'help -v' for verbose/'help <topic>' for details): ISO7816 Commands ================ read_binary select_adf select_file update_binary update_record verify_chv pySim Commands ============== intro verify_adm USIM Commands ============= read_ehplmn ust_service_activate ust_service_deactivate pySim-shell built-in commands ============================= alias help macro quit run_script shell edit history py run_pyscript set shortcuts ---------------------------------------------------------------------- ===> more interactive help ---------------------------------------------------------------------- pySIM-shell (3f00)> help read_binary usage: read_binary [-h] [--file-id FILE_ID] [--offset OFFSET] [--length LENGTH] [--record-nr RECORD_NR] Read binary data from a transparent EF optional arguments: -h, --help show this help message and exit --file-id FILE_ID File ID --offset OFFSET Byte offset for start of read --length LENGTH Number of bytes to read --record-nr RECORD_NR Number of record to read ---------------------------------------------------------------------- ===> navigating the FS and reading files ---------------------------------------------------------------------- pySIM-shell (3f00)> select_file 7f20 ['622c8202782183027f20a509800171830400018d088a01058b032f0601c60f90017083010183018183010a83010b'] pySIM-shell (3f00/7f20)> read_binary --file-id 6f07 089910070000400310 ---------------------------------------------------------------------- ===> interaction with local filesystem, i.e. I/O redirect + shell commands ---------------------------------------------------------------------- pySIM-shell (3f00)> select_adf a0000000871002 pySIM-shell (a0000000871002)> select_file 5f3b pySIM-shell (a0000000871002/5f3b)> read_binary --file-id 4f20 > /tmp/f pySIM-shell (a0000000871002/5f3b)> !cat /tmp/f ffffffffffffffff07 ---------------------------------------------------------------------- ===> piping output through shell tools like grep ---------------------------------------------------------------------- pySIM-shell (3f00)> read_ust | grep 86 Service 86 - Allowed CSG Lists and corresponding indications ---------------------------------------------------------------------- ===> enabling/disabling services ---------------------------------------------------------------------- pySIM-shell (3f00/7f20)> ust_service_activate 123 pySIM-shell (3f00/7f20)> ust_service_deactivate 123 ---------------------------------------------------------------------- It's a very first prototype, but it is really promising. The major tasks I see to make this go anywhere is: * have "File" class with encoder/decoder methods, which are registered automatically with a 'file system' layer that knows about the DF/ADF hierarchy ** this allows us to have a "read-decoded" command, which will call the decode method of the file, automatically resolved by the selected FID/path * automatic mapping of file-name -> FID and FID -> file name ** when printing (like in the path), use the human-readable names ** allow users to use human-readable names in SELECT * decode + display the TLVs / FCPs after a SELECT (like cyberflex-shell * ability to enable/disable APDU trace * dynamically register/deregster commands based on the path, i.e. offer USIM commands only when in ADF_USIM We have quite a bit of that infrastructure in the c-language libosmosim, (part of libosmocore.git), but unfortunately not in python :/ Let me know if anyone is interested in joining this effort. Regards, Harald -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Harald, I think this is a great idea and I'd love to help out, but I'm new to working on open source projects. What's the best way for me to get started in a way that would be helpful? Best, Bryan On Fri, Jan 8, 2021 at 5:54 PM Harald Welte <laforge@gnumonks.org> wrote:
Dear Osmocom community,
pySim-prog was nice when there were only 5 parameters on a SIM that we could program, and where the use case was pretty limited. Today, we have SIM/USIM/ISIM cards with hundreds of files and even more parameters to program. We cannot add a command line argument for each file to pySim-prog.
Instead, this introduces an interactive command-line shell / REPL, in which one can navigate the file system of the card, read and update files both in raw format and in decoded/parsed format.
The idea is primarily inspired by Henryk Ploetz' venerable cyberflex-shell, but implemented on a more modern basis using the cmd2 python module.
You can see the very first prototype in the laforge/shell branch of pysim.git
You can do things with it like this:
===> Start-up and authenticate with adm pin ---------------------------------------------------------------------- $ ./pysim-shell.py -p 0 Using PC/SC reader interface Autodetected card type: sysmoISIM-SJA2 AIDs on card: ['a0000000871002ffffffff8907090000', 'a0000000871004ffffffff8907090000'] Welcome to pySim-shell! pySIM-shell (3f00)> verify_adm 92990895 ----------------------------------------------------------------------
===> interactive help ---------------------------------------------------------------------- pySIM-shell (3f00)> help
Documented commands (use 'help -v' for verbose/'help <topic>' for details):
ISO7816 Commands ================ read_binary select_adf select_file update_binary update_record verify_chv
pySim Commands ============== intro verify_adm
USIM Commands ============= read_ehplmn ust_service_activate ust_service_deactivate
pySim-shell built-in commands ============================= alias help macro quit run_script shell edit history py run_pyscript set shortcuts
----------------------------------------------------------------------
===> more interactive help ---------------------------------------------------------------------- pySIM-shell (3f00)> help read_binary usage: read_binary [-h] [--file-id FILE_ID] [--offset OFFSET] [--length LENGTH] [--record-nr RECORD_NR]
Read binary data from a transparent EF
optional arguments: -h, --help show this help message and exit --file-id FILE_ID File ID --offset OFFSET Byte offset for start of read --length LENGTH Number of bytes to read --record-nr RECORD_NR Number of record to read ----------------------------------------------------------------------
===> navigating the FS and reading files ---------------------------------------------------------------------- pySIM-shell (3f00)> select_file 7f20
['622c8202782183027f20a509800171830400018d088a01058b032f0601c60f90017083010183018183010a83010b'] pySIM-shell (3f00/7f20)> read_binary --file-id 6f07 089910070000400310 ----------------------------------------------------------------------
===> interaction with local filesystem, i.e. I/O redirect + shell commands ---------------------------------------------------------------------- pySIM-shell (3f00)> select_adf a0000000871002 pySIM-shell (a0000000871002)> select_file 5f3b pySIM-shell (a0000000871002/5f3b)> read_binary --file-id 4f20 > /tmp/f pySIM-shell (a0000000871002/5f3b)> !cat /tmp/f ffffffffffffffff07 ----------------------------------------------------------------------
===> piping output through shell tools like grep ---------------------------------------------------------------------- pySIM-shell (3f00)> read_ust | grep 86 Service 86 - Allowed CSG Lists and corresponding indications ----------------------------------------------------------------------
===> enabling/disabling services ---------------------------------------------------------------------- pySIM-shell (3f00/7f20)> ust_service_activate 123 pySIM-shell (3f00/7f20)> ust_service_deactivate 123 ----------------------------------------------------------------------
It's a very first prototype, but it is really promising.
The major tasks I see to make this go anywhere is:
* have "File" class with encoder/decoder methods, which are registered automatically with a 'file system' layer that knows about the DF/ADF hierarchy ** this allows us to have a "read-decoded" command, which will call the decode method of the file, automatically resolved by the selected FID/path * automatic mapping of file-name -> FID and FID -> file name ** when printing (like in the path), use the human-readable names ** allow users to use human-readable names in SELECT * decode + display the TLVs / FCPs after a SELECT (like cyberflex-shell * ability to enable/disable APDU trace * dynamically register/deregster commands based on the path, i.e. offer USIM commands only when in ADF_USIM
We have quite a bit of that infrastructure in the c-language libosmosim, (part of libosmocore.git), but unfortunately not in python :/
Let me know if anyone is interested in joining this effort.
Regards, Harald -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/
============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Hi Bryan, On Sat, Jan 09, 2021 at 10:00:58AM -0500, bryan coxwell wrote:
I think this is a great idea and I'd love to help out, but I'm new to working on open source projects. What's the best way for me to get started in a way that would be helpful?
thanks for reaching out. By now most of the APIs/architecture is sorted out, so you could a) make sure that ts_51_011.py (sim) / ts_31_102.py (usim) / ts_31_103.py (isim) contain class definitions for all of the files specified in the latest Release 16 3GPP specifications b) write encoder and decoder methods within the EF classes for the various files of interest. See the EF_IMSI() as an example. The Decoder method should return a python dict (possibly a hierarchy of dicts) and return the binary representation, while the encoder method should take the dict and produce the binary representation. Regards, Harald -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Sounds good to me. 51.011 looks to be unchanged, which I guess is expected. Working on updating 31.102 now. Thanks, Bryan Coxwell
On Jan 11, 2021, at 8:10 AM, Harald Welte <laforge@gnumonks.org> wrote:
Hi Bryan,
On Sat, Jan 09, 2021 at 10:00:58AM -0500, bryan coxwell wrote: I think this is a great idea and I'd love to help out, but I'm new to working on open source projects. What's the best way for me to get started in a way that would be helpful?
thanks for reaching out. By now most of the APIs/architecture is sorted out, so you could
a) make sure that ts_51_011.py (sim) / ts_31_102.py (usim) / ts_31_103.py (isim) contain class definitions for all of the files specified in the latest Release 16 3GPP specifications
b) write encoder and decoder methods within the EF classes for the various files of interest. See the EF_IMSI() as an example. The Decoder method should return a python dict (possibly a hierarchy of dicts) and return the binary representation, while the encoder method should take the dict and produce the binary representation.
Regards, Harald -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
On Fri, Jan 08, 2021 at 11:53:51PM +0100, Harald Welte wrote:
You can see the very first prototype in the laforge/shell branch of pysim.git
Just a quick update: During past weeks I've been on and off spending a bit of time to bring this idea further along. It's already quite mature now. * many bugs have been fixed * SW1/SW2 are now parsed into human readable strings * many file specific encoders/decoders added * return to 'SELECT" command is now parsed into json: ---------------------------------------------------------------------- pySIM-shell (MF)> select ADF.ISIM { "file_descriptor": { "shareable": true, "file_type": "df", "structure": "no_info_given" }, "file_identifier": "FF01", "df_name": "A0000000871004FFFFFFFF8907090000", "proprietary_info": { "uicc_characteristics": "71", "available_memory": 101640 }, "life_cycle_status_int": "operational_activated" "security_attrib_compact": "00", "pin_status_template_do": "900170830101830181830 } pySIM-shell (MF/ADF.ISIM)> select EF.P-CSCF { "file_descriptor": { "shareable": true, "file_type": "working_ef", "structure": "linear_fixed", "record_len": 0, "num_of_rec": 0 }, "file_identifier": "6F09", "proprietary_info": { "proprietary_D0": "20", "proprietary_D2": "0F" }, "life_cycle_status_int": "operational_activated" "security_attrib_ref_expanded": "6F0603", "file_size": 1024, "short_file_id": "" } ---------------------------------------------------------------------- It's already reached a state where it can be used to perform useful tasks. Be warned: It's still very early and nothing has been tested with cards other than sysmoISIM-SJA2 at this point. I think within the next weeks I'll probably try to clean up the patches and get the current state merged to master. There's still a lot of work to be done, including: * option for storing per-ICCID ADM keys in some config file, so you don't have to enter them over and over again when frequently changing with the same cards * bulk read / write commands to read/write all records within one file * decide on some general rules on how to strucure the JSON output, such as including the file path and record number in some metadata? * automatically pad filss/records with fff to their size as determined from the card * ability to select/read/write arbitrary FID, e.g. for non-standard proprietary files that are not part of the ETSI/3GPP specs Regards, Harald -- - Harald Welte <laforge@osmocom.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
[Responding to Harald's updates regarding pysim-shell] Just as an FYI, if there is anyone else besides me who strongly dislikes Python, and/or finds Python3-based stuff extremely difficult to get working under Slackware, I just recently (between Jan 24 and now) implemented my own alternative SIM and UICC manipulation tools written in pure C, interfacing to CCID card "readers" via pcsc-lite. The tools live in my freecalypso-hwlab Hg repository: https://www.freecalypso.org/hg/freecalypso-hwlab/ simtool directory under the top level contains fc-simtool, and uicc directory contains fc-uicc-tool. As with most of my software, the source code is the only documentation at the moment. fc-simtool speaks GSM 11.11 protocol to the card, fc-uicc-tool speaks the UICC protocol per ETSI TS 102 221. The principal difference is in the CLA byte of each command and the format of SELECT responses. I wrote fc-simtool first, then ported some of the commands to fc-uicc-tool, but not all functionality of the former is replicated in the latter, only a subset. Both tools operate as an interactive shell, with a minimal scriping facility also included. Right now not a whole lot of functionality is implemented, just enough for the use cases of interest to me at the moment: * fc-simtool can be used to test SIMs for 2G compatibility, i.e., distinguish good SIMs that support GSM 11.11 from evil ones that have this support artificially removed, even though the physical 2G RAN is still operational. In USA there is only one physical operator with a GSM/2G network, but a whole slew of MVNOs running on it; some MVNO SIMs are still good, and fc-simtool works to test and identify them. * In his talk about SIM cards at the last CCC, Harald made a statement along the lines of "no one stores contacts on SIMs any more". That statement is factually wrong because I am still alive - once I die, Harald's statement will become correct, but for as long as I am alive, there is One Person On The Planet who does store contacts on SIMs - and that person is me! fc-simtool aids defiant users like me by providing commands to dump, restore and manually edit SIM phonebooks, and it also provides a save-sms-bin command to save the content of EF_SMS (SIM SMS store) in a binary file which can then be fully decoded with pcm-sms-decode from FC host tools package. * fc-uicc-tool in its current state is mostly for testing whether or not a given card has USIM and/or ISIM applications in addition to classic SIM. It can dump EF_DIR in decoded form, but it also allows a select-aid command to be issued manually, without going through EF_DIR. The latter capability is intended to detect whether or not a given card "really truly" has USIM/ISIM or not, even if someone wrote all FF bytes into EF_DIR to hide them. By the ETSI spec the SELECT by AID command allows truncated AIDs, i.e., one can send just the first 7 bytes of USIM or ISIM AID that are known without reading EF_DIR, and at least sysmoISIM-SJA2 accepts such truncated AIDs - I have yet to test on other cards. My ideal dream SIMs for running my own GSM/2G network (a network whose sole purpose is to provide service to classic GSM/2G phones) would be those that have only the classic SIM application, and NO USIM or ISIM. And I would really like for USIM and ISIM to be truly-truly Not There, rather than just hidden from EF_DIR. I am currently in negotiations with Grcard in China, who are telling me that they can supply me with such SIMs, and at an affordable price too, without a cost-prohibitive MOQ. They are in the process of sending me a few sample pieces (hoping they will actually go out before China closes for Lunar New Year), I will test them with fc-simtool and fc-uicc-tool (the latter to verify the absence of unwanted USIM and ISIM), and if these sample cards really are what I want, then I will place a bigger order with custom printing (FreeCalypso Community SIM), and I will make 10-pack sets of these SIMs available to the GSM/2G community in the same manner as how Sysmocom sells their USIM/ISIM-enabled version. 1FF+2FF form factor of course, with the 2FF card being a fully solid piece, NO 3FF or 4FF cuts - the whole purpose is to provide service to users of traditional GSM/2G phones, *not* "modern" abomination smartphones. If you need to stick one of these SIMs into an Abomination phone, cut it down yourself. And of course fc-simtool will be the officially recommended tool for programming these GSM-only SIMs. :-) Hasta la Victoria, Siempre, Mother Mychaela of FreeCalypso
Hi Mychaela, On Wed, Feb 03, 2021 at 11:01:19PM -0800, Mychaela Falconia wrote:
My ideal dream SIMs for running my own GSM/2G network (a network whose sole purpose is to provide service to classic GSM/2G phones) would be those that have only the classic SIM application, and NO USIM or ISIM. And I would really like for USIM and ISIM to be truly-truly Not There, rather than just hidden from EF_DIR.
In theory, you should be able to remove the respective applications from most java based cards with the right credentials via SCP02. Did you try?
I am currently in negotiations with Grcard in China, who are telling me that they can supply me with such SIMs, and at an affordable price too, without a cost-prohibitive MOQ.
sysmocom can certainly also provide such cards, should you have a related requirement. It requires creating a card profile without ADF.USIM + ADF.ISIM, which certainly can be done. I think the cards will still react to APDUs with CLA != 0xA0, but you would not have any ADF.USIM or ADF.ISIM present. One clear disadvantage of using SIM without USIM functionality even on a 2G radio network is that you can never use mutual authentication between network and phone (which is a pure protocol related aspect, so it could be implemented in any phone/baseband whose software can be modified - including FreeCalypso. Regards, Harald -- - Harald Welte <laforge@osmocom.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Hi Harald,
In theory, you should be able to remove the respective applications from most java based cards with the right credentials
For the "right credentials" part, I have a 10-pack of sysmoISIM-SJA2 cards which I bought with ADM keys included. The key material email from your webshop gives ADM1 and KI[CDK][123] for each card - are these the right credentials?
via SCP02. Did you try?
I am afraid the learning curve will probably be too steep for me to climb at this moment - but I am making a mental note to read up on SCP02, whatever it is, when and if I get the mental space for it.
sysmocom can certainly also provide such cards, should you have a related requirement.
What would be the MOQ and minimum order total cost for the following?: * Application customization: either classic GSM only or classic GSM plus USIM (TBD), but no ISIM; * 1FF+2FF-only cut, with the 2FF piece being fully solid w/o 3FF or 4FF cuts; * Custom printing.
It requires creating a card profile without ADF.USIM + ADF.ISIM, which certainly can be done.
Just how much control do you have over your cards? Your docs give the impression that your vendor/supplier more or less forced you into a new CardOS platform by declaring EOL on the one you had for SJS1, and the comments in your Python code for programming Ki/OPc/etc make it sound like these aspects were something you had to reverse-eng from what your vendor gave you, rather than something you actually designed yourselves. I mean, if your own team had actually designed the custom non-standard files for storing keys and algorithm selections, surely you would not have come up with the awkward design where you have to repeatedly store into each of SIM, USIM and ISIM, without making it clear at all whether or not these files are all linked or separate... If you were to make a version without ADF.ISIM or without both ADF.ISIM and ADF.USIM, what would the architecture look like for storing Ki (or K/OPc) and algorithm selections? I am also concerned about the following stanza in sysmo_isim_sja2.py: sysmo_isimsja2_algorithms = ( (SYSMO_ISIMSJA2_ALGO_COMP12V1, 'COMP128v1'), (SYSMO_ISIMSJA2_ALGO_COMP12V2, 'COMP128v2'), (SYSMO_ISIMSJA2_ALGO_COMP12V3, 'XOR-2G'), (SYSMO_ISIMSJA2_ALGO_MILENAGE, 'MILENAGE'), (SYSMO_ISIMSJA2_ALGO_SHA1AKA , 'SHA1-AKA'), (SYSMO_ISIMSJA2_ALGO_XOR, 'XOR'), ) Why is SYSMO_ISIMSJA2_ALGO_COMP12V3 mapped to 'XOR-2G' and not 'COMP128v3'? Does sysmoISIM-SJA2 (and presumably any custom config based on your current SJA2 CardOS) actually support COMP128v3 or not? If it doesn't, one has to do Milenage even for a 2G-only network, which is certainly an extra cognitive burden...
One clear disadvantage of using SIM without USIM functionality even on a 2G radio network is that you can never use mutual authentication between network and phone (which is a pure protocol related aspect, so it could be implemented in any phone/baseband whose software can be modified - including FreeCalypso.
Yes, I have thought about implementing USIM and 3G-style AKA support in FreeCalypso. But as my life circumstances stand currently, I don't know if I will live long enough to bring FC to a point where it would be a practically usable phone handset (whether on new or pre-existing Calypso hw), one that can actually replace historical phones with solid blob firmwares for everyday use - and in the absence of handset functionality which I could use with an end user hat on, I see no point in expending major effort to add new big functionality to what is essentially a toy (current AT-command-controlled FC modem). M~
Hi Mychaela, Just a side note from me, addressing your wish to not use Python for SIM operations. I have been using srsLTE for about 5 years now, and about 3 years ago physical USIM support was added to it. Naturally it is a very thin layer, pretty much only supporting requesting 3G AUTH vectors from the card, however it is written in C or C++ (I can’t remember right now which). Maybe it is useful for you :). Cheers, Domi
04.02.2021 dátummal, 13:42 időpontban Mychaela Falconia <mychaela.falconia@gmail.com> írta:
Hi Harald,
In theory, you should be able to remove the respective applications from most java based cards with the right credentials
For the "right credentials" part, I have a 10-pack of sysmoISIM-SJA2 cards which I bought with ADM keys included. The key material email from your webshop gives ADM1 and KI[CDK][123] for each card - are these the right credentials?
via SCP02. Did you try?
I am afraid the learning curve will probably be too steep for me to climb at this moment - but I am making a mental note to read up on SCP02, whatever it is, when and if I get the mental space for it.
sysmocom can certainly also provide such cards, should you have a related requirement.
What would be the MOQ and minimum order total cost for the following?:
* Application customization: either classic GSM only or classic GSM plus USIM (TBD), but no ISIM;
* 1FF+2FF-only cut, with the 2FF piece being fully solid w/o 3FF or 4FF cuts;
* Custom printing.
It requires creating a card profile without ADF.USIM + ADF.ISIM, which certainly can be done.
Just how much control do you have over your cards? Your docs give the impression that your vendor/supplier more or less forced you into a new CardOS platform by declaring EOL on the one you had for SJS1, and the comments in your Python code for programming Ki/OPc/etc make it sound like these aspects were something you had to reverse-eng from what your vendor gave you, rather than something you actually designed yourselves. I mean, if your own team had actually designed the custom non-standard files for storing keys and algorithm selections, surely you would not have come up with the awkward design where you have to repeatedly store into each of SIM, USIM and ISIM, without making it clear at all whether or not these files are all linked or separate...
If you were to make a version without ADF.ISIM or without both ADF.ISIM and ADF.USIM, what would the architecture look like for storing Ki (or K/OPc) and algorithm selections?
I am also concerned about the following stanza in sysmo_isim_sja2.py:
sysmo_isimsja2_algorithms = ( (SYSMO_ISIMSJA2_ALGO_COMP12V1, 'COMP128v1'), (SYSMO_ISIMSJA2_ALGO_COMP12V2, 'COMP128v2'), (SYSMO_ISIMSJA2_ALGO_COMP12V3, 'XOR-2G'), (SYSMO_ISIMSJA2_ALGO_MILENAGE, 'MILENAGE'), (SYSMO_ISIMSJA2_ALGO_SHA1AKA , 'SHA1-AKA'), (SYSMO_ISIMSJA2_ALGO_XOR, 'XOR'), )
Why is SYSMO_ISIMSJA2_ALGO_COMP12V3 mapped to 'XOR-2G' and not 'COMP128v3'? Does sysmoISIM-SJA2 (and presumably any custom config based on your current SJA2 CardOS) actually support COMP128v3 or not? If it doesn't, one has to do Milenage even for a 2G-only network, which is certainly an extra cognitive burden...
One clear disadvantage of using SIM without USIM functionality even on a 2G radio network is that you can never use mutual authentication between network and phone (which is a pure protocol related aspect, so it could be implemented in any phone/baseband whose software can be modified - including FreeCalypso.
Yes, I have thought about implementing USIM and 3G-style AKA support in FreeCalypso. But as my life circumstances stand currently, I don't know if I will live long enough to bring FC to a point where it would be a practically usable phone handset (whether on new or pre-existing Calypso hw), one that can actually replace historical phones with solid blob firmwares for everyday use - and in the absence of handset functionality which I could use with an end user hat on, I see no point in expending major effort to add new big functionality to what is essentially a toy (current AT-command-controlled FC modem).
M~
Hi Domi,
I have been using srsLTE for about 5 years now, and about 3 years ago physical USIM support was added to it.
Did you really mean srsLTE as in network side, or did you mean srsUE for LTE UE side? The latter seems to make a lot more sense for USIM support.
Naturally it is a very thin layer, pretty much only supporting requesting 3G AUTH vectors from the card,
I would think that an LTE UE implementation would need to read at least the IMSI from the card, so it knows how to present itself to the network, and probably some other important files.
however it is written in C or C++ (I can't remember right now which). Maybe it is useful for you :).
My fc-simtool and fc-uicc-tool are now pretty mature (they do what I need right now, and future functionality growth will be just adding higher-level operation commands on top of my already working foundation), so I have no need for other people's code (almost certainly with different coding style and philosophy) from random sources. M~
I have been using srsLTE for about 5 years now, and about 3 years ago physical USIM support was added to it.
Did you really mean srsLTE as in network side, or did you mean srsUE for LTE UE side? The latter seems to make a lot more sense for USIM support.
srsLTE is the global project ( https://github.com/srslte/srslte ) and it encompasses srsENB / srsEPC for the network side, srsUE for the mobile side along with a bunch of utility internal libraries shared between those components. Rgds, Sylvain
Hi Mychaela, On Thu, Feb 04, 2021 at 04:42:35AM -0800, Mychaela Falconia wrote:
In theory, you should be able to remove the respective applications from most java based cards with the right credentials
For the "right credentials" part, I have a 10-pack of sysmoISIM-SJA2 cards which I bought with ADM keys included. The key material email from your webshop gives ADM1 and KI[CDK][123] for each card - are these the right credentials?
yes, please see the user manual on which keys to use for authentication via SCP02 (par as global platform)
via SCP02. Did you try?
I am afraid the learning curve will probably be too steep for me to climb at this moment - but I am making a mental note to read up on SCP02, whatever it is, when and if I get the mental space for it.
There's a nice command line tool called GlobalPlatformPro (written In java) available from https://github.com/martinpaljak/GlobalPlatformPro Using it you can authenticate using SCP02 and then perform JavaCard application management on the card. I've never tried the "-delete" option for any of the ISIM/USIM applications, but it does look rather easy to try, if you are interested in that kind of thing. Please note, once you deleted it, there is likely no way to recover, as sysmocom cannot distribute third-party card applications in installable form.
sysmocom can certainly also provide such cards, should you have a related requirement.
What would be the MOQ and minimum order total cost for the following?:
MOQ for custom production batches at the factory is 1000 units. We can also re-program white or sysmocom-printed cards at sysmocom, if smaller batches are required (but that excludes custom printing).
* Application customization: either classic GSM only or classic GSM plus USIM (TBD), but no ISIM;
I'd do that for free in your case. But even if we'd charge, it's probably less than 4 hours of work in total.
* 1FF+2FF-only cut, with the 2FF piece being fully solid w/o 3FF or 4FF cuts;
No extra cost associated with that.
* Custom printing.
This is likely going to be the most significant factor. Printing for any batch below 10k cards has a significant price tag associated. I would prefer to keep the discussions on the osmocom lists of technical nature; feel free to contact sales@sysmocom.de for commercial information.
Just how much control do you have over your cards?
We control the configuration/profile/personalization process to the last bit, but not the CardOS. We can choose the OS and version, but we cannot modify the OS code.
Your docs give the impression that your vendor/supplier more or less forced you into a new CardOS platform by declaring EOL on the one you had for SJS1
Well, so is life in the technology industry. Both hardware (in this case the Samsung chip used) and software / OS go end-of-life for a variety of reasons. We've been able to source the old chips + OS for many years before, so I'm not complaining.
and the comments in your Python code for programming Ki/OPc/etc make it sound like these aspects were something you had to reverse-eng from what your vendor gave you, rather than something you actually designed yourselves.
sysmocom did not have to reverse-engineer the SJS1 nor SJA2 cards.
I mean, if your own team had actually designed the custom non-standard files for storing keys and algorithm selections, surely you would not have come up with the awkward design where you have to repeatedly store into each of SIM, USIM and ISIM, without making it clear at all whether or not these files are all linked or separate...
I disagree. This was a deliberate design decision for the SJA2 cards, in order to ensure maximum flexibility. If you link/share those files, then all applications must always share the same key material and algorithm. However, I can very well imagine legitimate use cases where you want to use different key materials for the IMS/VoLTE authentication than the key material used for the underlying radio network.
If you were to make a version without ADF.ISIM or without both ADF.ISIM and ADF.USIM, what would the architecture look like for storing Ki (or K/OPc) and algorithm selections?
if it's just SIM, then there's only one set of keys configured in DF.SYSTEM. If you want USIM and/or ISIM, those keys can either be shared or different.
I am also concerned about the following stanza in sysmo_isim_sja2.py:
sysmo_isimsja2_algorithms = ( (SYSMO_ISIMSJA2_ALGO_COMP12V1, 'COMP128v1'), (SYSMO_ISIMSJA2_ALGO_COMP12V2, 'COMP128v2'), (SYSMO_ISIMSJA2_ALGO_COMP12V3, 'XOR-2G'), (SYSMO_ISIMSJA2_ALGO_MILENAGE, 'MILENAGE'), (SYSMO_ISIMSJA2_ALGO_SHA1AKA , 'SHA1-AKA'), (SYSMO_ISIMSJA2_ALGO_XOR, 'XOR'), )
Why is SYSMO_ISIMSJA2_ALGO_COMP12V3 mapped to 'XOR-2G' and not 'COMP128v3'?
That is a question to Philipp (Cc) who is the main author and maintainer of that program. IT does look *very* strange to me, too :)
Does sysmoISIM-SJA2 (and presumably any custom config based on your current SJA2 CardOS) actually support COMP128v3 or not?
They should support it for sure, though I don't think anyone has tried so far. If you have the ADM1 keys, you are happy to try. -- - Harald Welte <laforge@osmocom.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Hi Harald,
I've never tried the "-delete" option for any of the ISIM/USIM applications, but it does look rather easy to try, if you are interested in that kind of thing.
Right now my curiosity in this area isn't strong enough to invest the time into installing and learning the necessary tools, but I may indeed try it one day.
Please note, once you deleted it, there is likely no way to recover, as sysmocom cannot distribute third-party card applications in installable form.
Yes, I realize that I may blow the card - but I got a pack of 10 of them for no purpose other than experiments, so... And because they are your current standard webshop product, I could always buy another pack if need arises.
I disagree. This was a deliberate design decision for the SJA2 cards, in order to ensure maximum flexibility. If you link/share those files, then all applications must always share the same key material and algorithm.
Then the actual linked vs. separate arrangement on the card should be documented - meaning explicitly document what is shared/linked vs. what is separate per SIM/USIM/ISIM. Your manual does not explicitly say one way or the other, and the comments in sysmo_isim_sja2.py make it sound like it is some kind of unknown (which made me think of reverse eng) whether the files are separate or linked. M~
On Sat, Feb 06, 2021 at 01:47:08PM -0800, Mychaela Falconia wrote:
Then the actual linked vs. separate arrangement on the card should be documented - meaning explicitly document what is shared/linked vs.
I've added a paragraph to the user manual about it. Nobody brought this up before. pysim-shell will also be able to show you which file is shared with what other file. -- - Harald Welte <laforge@osmocom.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Harald wrote:
pysim-shell will also be able to show you which file is shared with what other file.
How does it divine this knowledge? Perhaps I didn't read the ETSI UICC spec thoroughly enough regarding all of the nested TLVs returned in response to SELECT, but I don't see anything in any of the specs I've read that would allow a host talking to a UICC to know whether some two different SELECTable paths refer to the same underlying file. I can SELECT one file and read its content via READ BINARY or READ RECORD as appropriate, then do the same on the other file, and see that the byte content is the same - but how would I know if they are linked or separate files that just happen to have the same byte content? OK, I can envision an invasive test where you authenticate with the ADM1 key (for admin-write-only files), write into one file, and see if the other also magically changed - but is there any non-invasive test that doesn't involve writing? M~
Dear Osmocom community, two months later, a lot of work has been going into pySim-shell. The following features are now available from pysim.git master: * we now have a 'tree' command to list the filesystem hierarchy * you can now select arbitrary files by FID, even those not specified in 3GPP * there is now an 'export' command, which will iterate over all records in all files and dump them in an output format that can be used by pySim-shell itself to restore the data * we have the start of a user manual including a reference to the various pySim-shell comamnds. A preview is available at https://people.osmocom.org/laforge/tmp/pysim-doc-test/html/ * you can now use python setuptools to build and install dependencies, etc. * pySim-shell can retriev the card-specific ADM PIN from a CSV file in your home The latest invention (just pushed to gerrit for review) is the integration of JSONpath support, wich allows you to specify certain parts of a decoded file or record[s] to be updated. If you've never heard of JSONpath: It is to JSON what XPath is to XML. First example on how to use this: ------------------------------------------ pySIM-shell (MF/ADF.USIM/EF.FPLMN)> read_binary_decoded [ { "mcc": 262, "mnc": 42 }, { "mcc": 262, "mnc": 42 }, { "mcc": 262, "mnc": 42 }, { "mcc": 262, "mnc": 42 } ] pySIM-shell (MF/ADF.USIM/EF.FPLMN)> update_binary_decoded --json-path [*] null pySIM-shell (MF/ADF.USIM/EF.FPLMN)> read_binary_decoded [ null, null, null, null ] pySIM-shell (MF/ADF.USIM/EF.FPLMN)> update_binary_decoded --json-path [2] '{"mcc":"262", "mnc":"42"}' pySIM-shell (MF/ADF.USIM/EF.FPLMN)> read_binary_decoded [ null, null, { "mcc": 262, "mnc": 42 }, null ] ------------------------------------------ Second example, illustrating its use in more complex JSON types: ------------------------------------------ pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded { "ms_operation_mode": "normal", "specific_facilities": { "ofm": true }, "len_of_mnc_in_imsi": 2 } pySIM-shell (MF/ADF.USIM/EF.AD)> update_binary_decoded --json-path specific_facilities.ofm false pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded { "ms_operation_mode": "normal", "specific_facilities": { "ofm": false }, "len_of_mnc_in_imsi": 2 } ------------------------------------------ Stay tuned for more pySim developments coming up. Any feedback is welcome - as is help with writing encoder/decoder methods for all the files that don't have any yet. Regards, Harald -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Harald Welte wrote:
* we now have a 'tree' command to list the filesystem hierarchy
And just how do you get the card to tell you what selectable file IDs exist? I haven't seen anything like an ls operation in either the classic GSM 11.11 SIM protocol or the UICC protocol, thus the only way (that I know of) to find out what selectable file IDs exist is to do a brute force search of the 16-bit file ID space at every directory level. First select MF, then try selecting every possible 16-bit file ID from 0000 to FFFF (only skipping 3F00 for MF itself), and note which return something other than "not found" error. Follow up with a GET RESPONSE command for every SELECT which succeeded, parse the response, and report the findings. For all found file IDs which turn out to be DFs when the response is parsed, note those DF file IDs, and then repeat the brute force search inside every found DF - and then in any found nested DFs too. This brute force search is implemented in fc-simtool and fc-uicc-tool programs in my fc-sim-tools suite, my competitor to pySim: https://www.freecalypso.org/hg/fc-sim-tools/ As one would naturally expect, such brute force searches are painfully slow - IIRC, bfsearch-mf of sysmoISIM-SJA2 (just the MF tree, ADF trees have to be searched separately with bfsearch-adf) took about an hour, using HID Omnikey 3121 card reader, same model as the one currently sold in Sysmocom webshop - using an o'scope, I observed that it clocks the card at 4.8 MHz, almost up to the spec limit of 5 MHz. Because these brute force searches are so slow, I collect the captures and check them into my source repository under the data directory - so if you are curious to see what undocumented proprietary files exist on both Sysmocom and Grcard SIMs (whose existence cannot be discovered in any other way than this bfsearch), just look in the repository linked above. :-) I am not able to run pySim-shell on my Slackware system without expending more effort than I can currently justify, but I have glanced at the Python code, and I don't see anything like the just described brute force search - nor do I see it issuing any kind of secret undocumented ls-type APDU commands to the card - thus I am guessing that this 'tree' command displays nothing more than the tool's hard-coded knowledge of what files "should" exist at each given directory level, rather than what is actually found to exist. If I got this part wrong, then someone please explain what this command *actually* returns, and how it obtains this knowledge - I don't know of any way other than a brute force search. M~
Hi Mychaela, Without looking at the code I assume the way it works is that based on the service table the tool knows what files are supposed to be on the card and queries for them - just like an “ordinary phone” does. The point of the ST is exactly to avoid unnecessary bruteforcing/lookups of files that don’t exist. However it is correct to say that finding a file hidden on purpose (i.e. missing entry from the ST) could only be done via bruteforce imho. In case I am wrong sorry, didn’t mean to mislead anybody. Cheers, Domi
07.04.2021 dátummal, 2:41 időpontban Mychaela Falconia <mychaela.falconia@gmail.com> írta:
Harald Welte wrote:
* we now have a 'tree' command to list the filesystem hierarchy
And just how do you get the card to tell you what selectable file IDs exist? I haven't seen anything like an ls operation in either the classic GSM 11.11 SIM protocol or the UICC protocol, thus the only way (that I know of) to find out what selectable file IDs exist is to do a brute force search of the 16-bit file ID space at every directory level. First select MF, then try selecting every possible 16-bit file ID from 0000 to FFFF (only skipping 3F00 for MF itself), and note which return something other than "not found" error. Follow up with a GET RESPONSE command for every SELECT which succeeded, parse the response, and report the findings. For all found file IDs which turn out to be DFs when the response is parsed, note those DF file IDs, and then repeat the brute force search inside every found DF - and then in any found nested DFs too.
This brute force search is implemented in fc-simtool and fc-uicc-tool programs in my fc-sim-tools suite, my competitor to pySim:
https://www.freecalypso.org/hg/fc-sim-tools/
As one would naturally expect, such brute force searches are painfully slow - IIRC, bfsearch-mf of sysmoISIM-SJA2 (just the MF tree, ADF trees have to be searched separately with bfsearch-adf) took about an hour, using HID Omnikey 3121 card reader, same model as the one currently sold in Sysmocom webshop - using an o'scope, I observed that it clocks the card at 4.8 MHz, almost up to the spec limit of 5 MHz.
Because these brute force searches are so slow, I collect the captures and check them into my source repository under the data directory - so if you are curious to see what undocumented proprietary files exist on both Sysmocom and Grcard SIMs (whose existence cannot be discovered in any other way than this bfsearch), just look in the repository linked above. :-)
I am not able to run pySim-shell on my Slackware system without expending more effort than I can currently justify, but I have glanced at the Python code, and I don't see anything like the just described brute force search - nor do I see it issuing any kind of secret undocumented ls-type APDU commands to the card - thus I am guessing that this 'tree' command displays nothing more than the tool's hard-coded knowledge of what files "should" exist at each given directory level, rather than what is actually found to exist. If I got this part wrong, then someone please explain what this command *actually* returns, and how it obtains this knowledge - I don't know of any way other than a brute force search.
M~
participants (6)
-
bryan coxwell -
Harald Welte -
Harald Welte -
Mychaela Falconia -
Sylvain Munaut -
Tomcsanyi, Domonkos