From laforge at osmocom.org Tue Feb 2 18:38:07 2021 From: laforge at osmocom.org (Harald Welte) Date: Tue, 2 Feb 2021 19:38:07 +0100 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: On Fri, Jan 08, 2021 at 11:53:51PM +0100, Harald Welte wrote: > You can see the very first prototype in the laforge/shell branch of pysim.git Just a quick update: During past weeks I've been on and off spending a bit of time to bring this idea further along. It's already quite mature now. * many bugs have been fixed * SW1/SW2 are now parsed into human readable strings * many file specific encoders/decoders added * return to 'SELECT" command is now parsed into json: ---------------------------------------------------------------------- pySIM-shell (MF)> select ADF.ISIM { "file_descriptor": { "shareable": true, "file_type": "df", "structure": "no_info_given" }, "file_identifier": "FF01", "df_name": "A0000000871004FFFFFFFF8907090000", "proprietary_info": { "uicc_characteristics": "71", "available_memory": 101640 }, "life_cycle_status_int": "operational_activated" "security_attrib_compact": "00", "pin_status_template_do": "900170830101830181830 } pySIM-shell (MF/ADF.ISIM)> select EF.P-CSCF { "file_descriptor": { "shareable": true, "file_type": "working_ef", "structure": "linear_fixed", "record_len": 0, "num_of_rec": 0 }, "file_identifier": "6F09", "proprietary_info": { "proprietary_D0": "20", "proprietary_D2": "0F" }, "life_cycle_status_int": "operational_activated" "security_attrib_ref_expanded": "6F0603", "file_size": 1024, "short_file_id": "" } ---------------------------------------------------------------------- It's already reached a state where it can be used to perform useful tasks. Be warned: It's still very early and nothing has been tested with cards other than sysmoISIM-SJA2 at this point. I think within the next weeks I'll probably try to clean up the patches and get the current state merged to master. There's still a lot of work to be done, including: * option for storing per-ICCID ADM keys in some config file, so you don't have to enter them over and over again when frequently changing with the same cards * bulk read / write commands to read/write all records within one file * decide on some general rules on how to strucure the JSON output, such as including the file path and record number in some metadata? * automatically pad filss/records with fff to their size as determined from the card * ability to select/read/write arbitrary FID, e.g. for non-standard proprietary files that are not part of the ETSI/3GPP specs Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From mychaela.falconia at gmail.com Thu Feb 4 07:01:19 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Wed, 3 Feb 2021 23:01:19 -0800 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: [Responding to Harald's updates regarding pysim-shell] Just as an FYI, if there is anyone else besides me who strongly dislikes Python, and/or finds Python3-based stuff extremely difficult to get working under Slackware, I just recently (between Jan 24 and now) implemented my own alternative SIM and UICC manipulation tools written in pure C, interfacing to CCID card "readers" via pcsc-lite. The tools live in my freecalypso-hwlab Hg repository: https://www.freecalypso.org/hg/freecalypso-hwlab/ simtool directory under the top level contains fc-simtool, and uicc directory contains fc-uicc-tool. As with most of my software, the source code is the only documentation at the moment. fc-simtool speaks GSM 11.11 protocol to the card, fc-uicc-tool speaks the UICC protocol per ETSI TS 102 221. The principal difference is in the CLA byte of each command and the format of SELECT responses. I wrote fc-simtool first, then ported some of the commands to fc-uicc-tool, but not all functionality of the former is replicated in the latter, only a subset. Both tools operate as an interactive shell, with a minimal scriping facility also included. Right now not a whole lot of functionality is implemented, just enough for the use cases of interest to me at the moment: * fc-simtool can be used to test SIMs for 2G compatibility, i.e., distinguish good SIMs that support GSM 11.11 from evil ones that have this support artificially removed, even though the physical 2G RAN is still operational. In USA there is only one physical operator with a GSM/2G network, but a whole slew of MVNOs running on it; some MVNO SIMs are still good, and fc-simtool works to test and identify them. * In his talk about SIM cards at the last CCC, Harald made a statement along the lines of "no one stores contacts on SIMs any more". That statement is factually wrong because I am still alive - once I die, Harald's statement will become correct, but for as long as I am alive, there is One Person On The Planet who does store contacts on SIMs - and that person is me! fc-simtool aids defiant users like me by providing commands to dump, restore and manually edit SIM phonebooks, and it also provides a save-sms-bin command to save the content of EF_SMS (SIM SMS store) in a binary file which can then be fully decoded with pcm-sms-decode from FC host tools package. * fc-uicc-tool in its current state is mostly for testing whether or not a given card has USIM and/or ISIM applications in addition to classic SIM. It can dump EF_DIR in decoded form, but it also allows a select-aid command to be issued manually, without going through EF_DIR. The latter capability is intended to detect whether or not a given card "really truly" has USIM/ISIM or not, even if someone wrote all FF bytes into EF_DIR to hide them. By the ETSI spec the SELECT by AID command allows truncated AIDs, i.e., one can send just the first 7 bytes of USIM or ISIM AID that are known without reading EF_DIR, and at least sysmoISIM-SJA2 accepts such truncated AIDs - I have yet to test on other cards. My ideal dream SIMs for running my own GSM/2G network (a network whose sole purpose is to provide service to classic GSM/2G phones) would be those that have only the classic SIM application, and NO USIM or ISIM. And I would really like for USIM and ISIM to be truly-truly Not There, rather than just hidden from EF_DIR. I am currently in negotiations with Grcard in China, who are telling me that they can supply me with such SIMs, and at an affordable price too, without a cost-prohibitive MOQ. They are in the process of sending me a few sample pieces (hoping they will actually go out before China closes for Lunar New Year), I will test them with fc-simtool and fc-uicc-tool (the latter to verify the absence of unwanted USIM and ISIM), and if these sample cards really are what I want, then I will place a bigger order with custom printing (FreeCalypso Community SIM), and I will make 10-pack sets of these SIMs available to the GSM/2G community in the same manner as how Sysmocom sells their USIM/ISIM-enabled version. 1FF+2FF form factor of course, with the 2FF card being a fully solid piece, NO 3FF or 4FF cuts - the whole purpose is to provide service to users of traditional GSM/2G phones, *not* "modern" abomination smartphones. If you need to stick one of these SIMs into an Abomination phone, cut it down yourself. And of course fc-simtool will be the officially recommended tool for programming these GSM-only SIMs. :-) Hasta la Victoria, Siempre, Mother Mychaela of FreeCalypso From laforge at osmocom.org Thu Feb 4 11:03:49 2021 From: laforge at osmocom.org (Harald Welte) Date: Thu, 4 Feb 2021 12:03:49 +0100 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Hi Mychaela, On Wed, Feb 03, 2021 at 11:01:19PM -0800, Mychaela Falconia wrote: > My ideal dream SIMs for running my own GSM/2G network (a network whose > sole purpose is to provide service to classic GSM/2G phones) would be > those that have only the classic SIM application, and NO USIM or ISIM. > And I would really like for USIM and ISIM to be truly-truly Not There, > rather than just hidden from EF_DIR. In theory, you should be able to remove the respective applications from most java based cards with the right credentials via SCP02. Did you try? > I am currently in negotiations with Grcard in China, who are telling > me that they can supply me with such SIMs, and at an affordable price > too, without a cost-prohibitive MOQ. sysmocom can certainly also provide such cards, should you have a related requirement. It requires creating a card profile without ADF.USIM + ADF.ISIM, which certainly can be done. I think the cards will still react to APDUs with CLA != 0xA0, but you would not have any ADF.USIM or ADF.ISIM present. One clear disadvantage of using SIM without USIM functionality even on a 2G radio network is that you can never use mutual authentication between network and phone (which is a pure protocol related aspect, so it could be implemented in any phone/baseband whose software can be modified - including FreeCalypso. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From mychaela.falconia at gmail.com Thu Feb 4 12:42:35 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Thu, 4 Feb 2021 04:42:35 -0800 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Hi Harald, > In theory, you should be able to remove the respective applications from > most java based cards with the right credentials For the "right credentials" part, I have a 10-pack of sysmoISIM-SJA2 cards which I bought with ADM keys included. The key material email from your webshop gives ADM1 and KI[CDK][123] for each card - are these the right credentials? > via SCP02. Did you try? I am afraid the learning curve will probably be too steep for me to climb at this moment - but I am making a mental note to read up on SCP02, whatever it is, when and if I get the mental space for it. > sysmocom can certainly also provide such cards, should you have a > related requirement. What would be the MOQ and minimum order total cost for the following?: * Application customization: either classic GSM only or classic GSM plus USIM (TBD), but no ISIM; * 1FF+2FF-only cut, with the 2FF piece being fully solid w/o 3FF or 4FF cuts; * Custom printing. > It requires creating a card profile without > ADF.USIM + ADF.ISIM, which certainly can be done. Just how much control do you have over your cards? Your docs give the impression that your vendor/supplier more or less forced you into a new CardOS platform by declaring EOL on the one you had for SJS1, and the comments in your Python code for programming Ki/OPc/etc make it sound like these aspects were something you had to reverse-eng from what your vendor gave you, rather than something you actually designed yourselves. I mean, if your own team had actually designed the custom non-standard files for storing keys and algorithm selections, surely you would not have come up with the awkward design where you have to repeatedly store into each of SIM, USIM and ISIM, without making it clear at all whether or not these files are all linked or separate... If you were to make a version without ADF.ISIM or without both ADF.ISIM and ADF.USIM, what would the architecture look like for storing Ki (or K/OPc) and algorithm selections? I am also concerned about the following stanza in sysmo_isim_sja2.py: sysmo_isimsja2_algorithms = ( (SYSMO_ISIMSJA2_ALGO_COMP12V1, 'COMP128v1'), (SYSMO_ISIMSJA2_ALGO_COMP12V2, 'COMP128v2'), (SYSMO_ISIMSJA2_ALGO_COMP12V3, 'XOR-2G'), (SYSMO_ISIMSJA2_ALGO_MILENAGE, 'MILENAGE'), (SYSMO_ISIMSJA2_ALGO_SHA1AKA , 'SHA1-AKA'), (SYSMO_ISIMSJA2_ALGO_XOR, 'XOR'), ) Why is SYSMO_ISIMSJA2_ALGO_COMP12V3 mapped to 'XOR-2G' and not 'COMP128v3'? Does sysmoISIM-SJA2 (and presumably any custom config based on your current SJA2 CardOS) actually support COMP128v3 or not? If it doesn't, one has to do Milenage even for a 2G-only network, which is certainly an extra cognitive burden... > One clear disadvantage of using SIM without USIM functionality even on a > 2G radio network is that you can never use mutual authentication between > network and phone (which is a pure protocol related aspect, so it could > be implemented in any phone/baseband whose software can be modified - > including FreeCalypso. Yes, I have thought about implementing USIM and 3G-style AKA support in FreeCalypso. But as my life circumstances stand currently, I don't know if I will live long enough to bring FC to a point where it would be a practically usable phone handset (whether on new or pre-existing Calypso hw), one that can actually replace historical phones with solid blob firmwares for everyday use - and in the absence of handset functionality which I could use with an end user hat on, I see no point in expending major effort to add new big functionality to what is essentially a toy (current AT-command-controlled FC modem). M~ From domi at tomcsanyi.net Fri Feb 5 08:38:09 2021 From: domi at tomcsanyi.net (Tomcsanyi, Domonkos) Date: Fri, 5 Feb 2021 09:38:09 +0100 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Hi Mychaela, Just a side note from me, addressing your wish to not use Python for SIM operations. I have been using srsLTE for about 5 years now, and about 3 years ago physical USIM support was added to it. Naturally it is a very thin layer, pretty much only supporting requesting 3G AUTH vectors from the card, however it is written in C or C++ (I can?t remember right now which). Maybe it is useful for you :). Cheers, Domi > 04.02.2021 d?tummal, 13:42 id?pontban Mychaela Falconia ?rta: > > ?Hi Harald, > >> In theory, you should be able to remove the respective applications from >> most java based cards with the right credentials > > For the "right credentials" part, I have a 10-pack of sysmoISIM-SJA2 > cards which I bought with ADM keys included. The key material email > from your webshop gives ADM1 and KI[CDK][123] for each card - are > these the right credentials? > >> via SCP02. Did you try? > > I am afraid the learning curve will probably be too steep for me to > climb at this moment - but I am making a mental note to read up on > SCP02, whatever it is, when and if I get the mental space for it. > >> sysmocom can certainly also provide such cards, should you have a >> related requirement. > > What would be the MOQ and minimum order total cost for the following?: > > * Application customization: either classic GSM only or classic GSM > plus USIM (TBD), but no ISIM; > > * 1FF+2FF-only cut, with the 2FF piece being fully solid w/o 3FF or > 4FF cuts; > > * Custom printing. > >> It requires creating a card profile without >> ADF.USIM + ADF.ISIM, which certainly can be done. > > Just how much control do you have over your cards? Your docs give the > impression that your vendor/supplier more or less forced you into a > new CardOS platform by declaring EOL on the one you had for SJS1, and > the comments in your Python code for programming Ki/OPc/etc make it > sound like these aspects were something you had to reverse-eng from > what your vendor gave you, rather than something you actually designed > yourselves. I mean, if your own team had actually designed the custom > non-standard files for storing keys and algorithm selections, surely > you would not have come up with the awkward design where you have to > repeatedly store into each of SIM, USIM and ISIM, without making it > clear at all whether or not these files are all linked or separate... > > If you were to make a version without ADF.ISIM or without both > ADF.ISIM and ADF.USIM, what would the architecture look like for > storing Ki (or K/OPc) and algorithm selections? > > I am also concerned about the following stanza in sysmo_isim_sja2.py: > > sysmo_isimsja2_algorithms = ( > (SYSMO_ISIMSJA2_ALGO_COMP12V1, 'COMP128v1'), > (SYSMO_ISIMSJA2_ALGO_COMP12V2, 'COMP128v2'), > (SYSMO_ISIMSJA2_ALGO_COMP12V3, 'XOR-2G'), > (SYSMO_ISIMSJA2_ALGO_MILENAGE, 'MILENAGE'), > (SYSMO_ISIMSJA2_ALGO_SHA1AKA , 'SHA1-AKA'), > (SYSMO_ISIMSJA2_ALGO_XOR, 'XOR'), > ) > > Why is SYSMO_ISIMSJA2_ALGO_COMP12V3 mapped to 'XOR-2G' and not > 'COMP128v3'? Does sysmoISIM-SJA2 (and presumably any custom config > based on your current SJA2 CardOS) actually support COMP128v3 or not? > If it doesn't, one has to do Milenage even for a 2G-only network, > which is certainly an extra cognitive burden... > >> One clear disadvantage of using SIM without USIM functionality even on a >> 2G radio network is that you can never use mutual authentication between >> network and phone (which is a pure protocol related aspect, so it could >> be implemented in any phone/baseband whose software can be modified - >> including FreeCalypso. > > Yes, I have thought about implementing USIM and 3G-style AKA support > in FreeCalypso. But as my life circumstances stand currently, I don't > know if I will live long enough to bring FC to a point where it would > be a practically usable phone handset (whether on new or pre-existing > Calypso hw), one that can actually replace historical phones with > solid blob firmwares for everyday use - and in the absence of handset > functionality which I could use with an end user hat on, I see no > point in expending major effort to add new big functionality to what > is essentially a toy (current AT-command-controlled FC modem). > > M~ From mychaela.falconia at gmail.com Fri Feb 5 17:35:55 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Fri, 5 Feb 2021 09:35:55 -0800 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Hi Domi, > I have been using srsLTE for about 5 years now, and about 3 years ago > physical USIM support was added to it. Did you really mean srsLTE as in network side, or did you mean srsUE for LTE UE side? The latter seems to make a lot more sense for USIM support. > Naturally it is a very thin layer, pretty much only supporting requesting > 3G AUTH vectors from the card, I would think that an LTE UE implementation would need to read at least the IMSI from the card, so it knows how to present itself to the network, and probably some other important files. > however it is written in C or C++ (I can't remember right now which). > Maybe it is useful for you :). My fc-simtool and fc-uicc-tool are now pretty mature (they do what I need right now, and future functionality growth will be just adding higher-level operation commands on top of my already working foundation), so I have no need for other people's code (almost certainly with different coding style and philosophy) from random sources. M~ From 246tnt at gmail.com Sat Feb 6 07:41:22 2021 From: 246tnt at gmail.com (Sylvain Munaut) Date: Sat, 6 Feb 2021 08:41:22 +0100 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: > > I have been using srsLTE for about 5 years now, and about 3 years ago > > physical USIM support was added to it. > > Did you really mean srsLTE as in network side, or did you mean srsUE > for LTE UE side? The latter seems to make a lot more sense for USIM > support. srsLTE is the global project ( https://github.com/srslte/srslte ) and it encompasses srsENB / srsEPC for the network side, srsUE for the mobile side along with a bunch of utility internal libraries shared between those components. Rgds, Sylvain From laforge at osmocom.org Sat Feb 6 12:37:50 2021 From: laforge at osmocom.org (Harald Welte) Date: Sat, 6 Feb 2021 13:37:50 +0100 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Hi Mychaela, On Thu, Feb 04, 2021 at 04:42:35AM -0800, Mychaela Falconia wrote: > > In theory, you should be able to remove the respective applications from > > most java based cards with the right credentials > > For the "right credentials" part, I have a 10-pack of sysmoISIM-SJA2 > cards which I bought with ADM keys included. The key material email > from your webshop gives ADM1 and KI[CDK][123] for each card - are > these the right credentials? yes, please see the user manual on which keys to use for authentication via SCP02 (par as global platform) > > via SCP02. Did you try? > > I am afraid the learning curve will probably be too steep for me to > climb at this moment - but I am making a mental note to read up on > SCP02, whatever it is, when and if I get the mental space for it. There's a nice command line tool called GlobalPlatformPro (written In java) available from https://github.com/martinpaljak/GlobalPlatformPro Using it you can authenticate using SCP02 and then perform JavaCard application management on the card. I've never tried the "-delete" option for any of the ISIM/USIM applications, but it does look rather easy to try, if you are interested in that kind of thing. Please note, once you deleted it, there is likely no way to recover, as sysmocom cannot distribute third-party card applications in installable form. > > sysmocom can certainly also provide such cards, should you have a > > related requirement. > > What would be the MOQ and minimum order total cost for the following?: MOQ for custom production batches at the factory is 1000 units. We can also re-program white or sysmocom-printed cards at sysmocom, if smaller batches are required (but that excludes custom printing). > * Application customization: either classic GSM only or classic GSM > plus USIM (TBD), but no ISIM; I'd do that for free in your case. But even if we'd charge, it's probably less than 4 hours of work in total. > * 1FF+2FF-only cut, with the 2FF piece being fully solid w/o 3FF or > 4FF cuts; No extra cost associated with that. > * Custom printing. This is likely going to be the most significant factor. Printing for any batch below 10k cards has a significant price tag associated. I would prefer to keep the discussions on the osmocom lists of technical nature; feel free to contact sales at sysmocom.de for commercial information. > Just how much control do you have over your cards? We control the configuration/profile/personalization process to the last bit, but not the CardOS. We can choose the OS and version, but we cannot modify the OS code. > Your docs give the impression that your vendor/supplier more or less > forced you into a new CardOS platform by declaring EOL on the one you > had for SJS1 Well, so is life in the technology industry. Both hardware (in this case the Samsung chip used) and software / OS go end-of-life for a variety of reasons. We've been able to source the old chips + OS for many years before, so I'm not complaining. > and > the comments in your Python code for programming Ki/OPc/etc make it > sound like these aspects were something you had to reverse-eng from > what your vendor gave you, rather than something you actually designed > yourselves. sysmocom did not have to reverse-engineer the SJS1 nor SJA2 cards. > I mean, if your own team had actually designed the custom > non-standard files for storing keys and algorithm selections, surely > you would not have come up with the awkward design where you have to > repeatedly store into each of SIM, USIM and ISIM, without making it > clear at all whether or not these files are all linked or separate... I disagree. This was a deliberate design decision for the SJA2 cards, in order to ensure maximum flexibility. If you link/share those files, then all applications must always share the same key material and algorithm. However, I can very well imagine legitimate use cases where you want to use different key materials for the IMS/VoLTE authentication than the key material used for the underlying radio network. > If you were to make a version without ADF.ISIM or without both > ADF.ISIM and ADF.USIM, what would the architecture look like for > storing Ki (or K/OPc) and algorithm selections? if it's just SIM, then there's only one set of keys configured in DF.SYSTEM. If you want USIM and/or ISIM, those keys can either be shared or different. > I am also concerned about the following stanza in sysmo_isim_sja2.py: > > sysmo_isimsja2_algorithms = ( > (SYSMO_ISIMSJA2_ALGO_COMP12V1, 'COMP128v1'), > (SYSMO_ISIMSJA2_ALGO_COMP12V2, 'COMP128v2'), > (SYSMO_ISIMSJA2_ALGO_COMP12V3, 'XOR-2G'), > (SYSMO_ISIMSJA2_ALGO_MILENAGE, 'MILENAGE'), > (SYSMO_ISIMSJA2_ALGO_SHA1AKA , 'SHA1-AKA'), > (SYSMO_ISIMSJA2_ALGO_XOR, 'XOR'), > ) > > Why is SYSMO_ISIMSJA2_ALGO_COMP12V3 mapped to 'XOR-2G' and not > 'COMP128v3'? That is a question to Philipp (Cc) who is the main author and maintainer of that program. IT does look *very* strange to me, too :) > Does sysmoISIM-SJA2 (and presumably any custom config > based on your current SJA2 CardOS) actually support COMP128v3 or not? They should support it for sure, though I don't think anyone has tried so far. If you have the ADM1 keys, you are happy to try. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From mychaela.falconia at gmail.com Sat Feb 6 21:47:08 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Sat, 6 Feb 2021 13:47:08 -0800 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Hi Harald, > I've never tried the "-delete" option for any of the ISIM/USIM applications, > but it does look rather easy to try, if you are interested in that kind of > thing. Right now my curiosity in this area isn't strong enough to invest the time into installing and learning the necessary tools, but I may indeed try it one day. > Please note, once you deleted it, there is likely no way to recover, as > sysmocom cannot distribute third-party card applications in installable form. Yes, I realize that I may blow the card - but I got a pack of 10 of them for no purpose other than experiments, so... And because they are your current standard webshop product, I could always buy another pack if need arises. > I disagree. This was a deliberate design decision for the SJA2 cards, > in order to ensure maximum flexibility. If you link/share those files, > then all applications must always share the same key material and > algorithm. Then the actual linked vs. separate arrangement on the card should be documented - meaning explicitly document what is shared/linked vs. what is separate per SIM/USIM/ISIM. Your manual does not explicitly say one way or the other, and the comments in sysmo_isim_sja2.py make it sound like it is some kind of unknown (which made me think of reverse eng) whether the files are separate or linked. M~ From mychaela.falconia at gmail.com Sun Feb 7 03:30:35 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Sat, 6 Feb 2021 19:30:35 -0800 Subject: Looking for historical sysmoSIM-GR2 card Message-ID: Hello fellow GSM hackers, Would anyone here happen to have a sysmoSIM-GR2 card (once sold by Sysmocom aeons ago, but long since discontinued) which they would be willing to sell? I have a friend who has sysmoSIM-GR1 and sysmoUSIM-GR1 (other long-discontinued historical cards from aeons ago), but no sysmoSIM-GR2. I am looking for just one card (doesn't matter if it is fully intact, broken out 2FF, or even cut down to 3FF), and I am willing to pay premium price for it, plus I will pay for private courier shipping to bypass USPS which can't be trusted with anything valuable these days. So if anyone has one of these cards which they would be willing to part with, and could use a little money, please give me a holler. Oh, and if the card's SUPER ADM PIN has been changed to something other than the default 88888888, I will need to be given that key. Background info: I am trying to get some new cards from Grcard, and I ordered samples. These samples were originally scheduled to arrive in USA next week, but then some snafu happened, and the shipment got returned to the sender. I was given a non-answer as to what the problem issue is, and I basically have to wait an indefinitely long time (at least past LNY 2-week holiday, plus however much longer after that) to get a resent package - and it looks like they are going to resend via a slower method too. Meanwhile I have implemented Grcard2 custom commands in fc-simtool based on this wiki page: https://osmocom.org/projects/cellular-infrastructure/wiki/GrcardSIM2 I am now looking for a way to test my Grcard2 command implementation against some card that is *known* to speak Grcard2 protocol and not some other (hence sysmoSIM-GR2 would be ideal), and when I receive the new sample cards from Grcard at some future time (probably many weeks or even months out), I will be able to compare, and determine if they are also Grcard2 or something else. (If the new cards turn out to be different from both Grcard1 and Grcard2, I will call them Grcard3. :) Hopeful, Mother Mychaela of FreeCalypso From laforge at osmocom.org Sun Feb 7 19:24:40 2021 From: laforge at osmocom.org (Harald Welte) Date: Sun, 7 Feb 2021 20:24:40 +0100 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: On Sat, Feb 06, 2021 at 01:47:08PM -0800, Mychaela Falconia wrote: > Then the actual linked vs. separate arrangement on the card should be > documented - meaning explicitly document what is shared/linked vs. I've added a paragraph to the user manual about it. Nobody brought this up before. pysim-shell will also be able to show you which file is shared with what other file. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From mychaela.falconia at gmail.com Sun Feb 7 21:55:35 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Sun, 7 Feb 2021 13:55:35 -0800 Subject: WIP / RFC for pysim 'next generation;' aka pysim-shell In-Reply-To: References: Message-ID: Harald wrote: > pysim-shell will also be able to show you which file is shared with what > other file. How does it divine this knowledge? Perhaps I didn't read the ETSI UICC spec thoroughly enough regarding all of the nested TLVs returned in response to SELECT, but I don't see anything in any of the specs I've read that would allow a host talking to a UICC to know whether some two different SELECTable paths refer to the same underlying file. I can SELECT one file and read its content via READ BINARY or READ RECORD as appropriate, then do the same on the other file, and see that the byte content is the same - but how would I know if they are linked or separate files that just happen to have the same byte content? OK, I can envision an invasive test where you authenticate with the ADM1 key (for admin-write-only files), write into one file, and see if the other also magically changed - but is there any non-invasive test that doesn't involve writing? M~ From morteza.ali.ahmadi at gmail.com Sun Feb 7 18:40:22 2021 From: morteza.ali.ahmadi at gmail.com (morteza ali Ahmadi) Date: Sun, 7 Feb 2021 22:10:22 +0330 Subject: Question about OsmoGGSN project Message-ID: Hi, osmocom team members We have run OsmoGGSN project using the attached config file in this email and the following command: osmo-ggsn As you can see in the config file, we have set IP 127.0.0.2 for GGSN. After executing the above command, a tunnel named tun4 is created. To communicate with SGSN, we use the SGSN emulator in OsmoGGSN as follow: sgsnemu -l 127.0.0.1 -r 127.0.0.2 After executing the above command, we see that 2 packets are sent from SGSN to GGSN. These packets are "Echo request" and "Create PDP context request" and GGSN responses to SGSN with the packets "Echo response" and "Create PDP context response". To test tun4, we send a GTP packet containing an arbitrary query to GGSN and we see that this GTP packet is received in Loopback (lo) interface and also, the DNS packet is received in tun4 interface. But there is no DNS response to our query neither in tun4 interface nor lo interface. A screenshot of the lo interface and tun4 interface in the wireshark is attached in this email (right is lo and left is tun4). How can we receive DNS responses to our DNS query? (Does tun4 require routing or something else?) -- *When there is much light, The shadow is deep...* -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: osmo-ggsn.cfg Type: application/octet-stream Size: 1075 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: log.png Type: image/png Size: 207848 bytes Desc: not available URL: From laforge at osmocom.org Mon Feb 8 20:27:04 2021 From: laforge at osmocom.org (Harald Welte) Date: Mon, 8 Feb 2021 21:27:04 +0100 Subject: Question about OsmoGGSN project In-Reply-To: References: Message-ID: Hi, On Sun, Feb 07, 2021 at 10:10:22PM +0330, morteza ali Ahmadi wrote: > How can we receive DNS responses to our DNS query? (Does tun4 require > routing or something else?) It is a normal linux network device and you will need to set up routing normally. Like routing to any other net-device. There's no difference with any other net-device. You can use routing, packet filtering, traffic shaping, NAT, ... as usual. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From ivan.babanov at gmail.com Tue Feb 9 10:55:17 2021 From: ivan.babanov at gmail.com (Ivan Babanov) Date: Tue, 9 Feb 2021 13:55:17 +0300 Subject: Ettus USRP E100 usage for current osmo-trx Message-ID: Hello all! Does anybody have any actual information about the possibility to run osmo-trx on Ettus E100 board? I see some references in manuals about the possibility of running it, but I can't find details on how to do it. I tried several ways: 1. Build from source on target HW. Unfortunately "autoreconf" crashes with some perl module dependency. 2. Install binaries. Binaries for arm available only for Debian 9, so I built an image and installed Debian 9.0 on SD-card. Installation of binaries was unsuccessful because of some unmet dependencies. 3. Build from scratch on Debian 9.0. I was partially successful in it. I was able to build osmocore and osmo-trx. The problem was to run it because now, uhd_find_devices shows an empty list. It looks like that kernel module usrp_e.ko should be loaded, but I have no idea where to find its sources. And one more question: Is it a proper place to ask questions related to BTS and TRX? Should I move my question to some other mail list? Thank you Ivan From laforge at osmocom.org Tue Feb 9 15:55:56 2021 From: laforge at osmocom.org (Harald Welte) Date: Tue, 9 Feb 2021 16:55:56 +0100 Subject: Ettus USRP E100 usage for current osmo-trx In-Reply-To: References: Message-ID: Hi Ivan, I'm not aware of any SUR PExxx users within the Osmocom community. There may (be / have been) some, but if, they are probably rare. >From the Osmocom stack thre shouldn't really be any problem: * we build our code for other embedded ARM targets like ** sysmoBTS devices (using yocto/poky/OE) ** Raspbian builds are part of network:osmocom:nightly So the Osmocom code will run on ARM based Linux systems. All of your questions / issues seem really be related to the USRP E100 itself, and its drivers and/or OS level issues. On Tue, Feb 09, 2021 at 01:55:17PM +0300, Ivan Babanov wrote: > And one more question: Is it a proper place to ask questions related > to BTS and TRX? Should I move my question to some other mail list? sure, the list is right - I just think that the problems you mentioned so far are not really rooted in the Osmocom software, but the hardware / driver / OS, and I would expect most people have no experience with the E100. Regards, Haradl -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From ralph at schmid.xxx Wed Feb 10 15:13:59 2021 From: ralph at schmid.xxx (Ralph A. Schmid, dk5ras) Date: Wed, 10 Feb 2021 16:13:59 +0100 Subject: Looking for historical sysmoSIM-GR2 card In-Reply-To: References: Message-ID: <007001d6ffbf$5baab980$13002c80$@schmid.xxx> How can I recognize such a card, from some marking or so? Then I can check what I have around... With best regards Ralph. > -----Original Message----- > From: OpenBSC [mailto:openbsc-bounces at lists.osmocom.org] On Behalf Of > Mychaela Falconia > Sent: Sunday, February 7, 2021 4:31 AM > To: openbsc > Subject: Looking for historical sysmoSIM-GR2 card > > Hello fellow GSM hackers, > > Would anyone here happen to have a sysmoSIM-GR2 card (once sold by From morteza.ali.ahmadi at gmail.com Wed Feb 10 18:23:54 2021 From: morteza.ali.ahmadi at gmail.com (morteza ali Ahmadi) Date: Wed, 10 Feb 2021 21:53:54 +0330 Subject: Question about OsmoGGSN project In-Reply-To: References: Message-ID: Ok, thanks Harald. I'll check it out and share my results with you. On Mon, Feb 8, 2021 at 11:57 PM Harald Welte wrote: > Hi, > > On Sun, Feb 07, 2021 at 10:10:22PM +0330, morteza ali Ahmadi wrote: > > How can we receive DNS responses to our DNS query? (Does tun4 require > > routing or something else?) > > It is a normal linux network device and you will need to set up routing > normally. Like routing to any other net-device. There's no difference > with any other net-device. You can use routing, packet filtering, traffic > shaping, NAT, ... as usual. > > -- > - Harald Welte > http://laforge.gnumonks.org/ > > ============================================================================ > "Privacy in residential applications is a desirable marketing option." > (ETSI EN 300 175-7 Ch. > A6) > -- *When there is much light, The shadow is deep...* -------------- next part -------------- An HTML attachment was scrubbed... URL: From mychaela.falconia at gmail.com Wed Feb 10 18:47:24 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Wed, 10 Feb 2021 10:47:24 -0800 Subject: Looking for historical sysmoSIM-GR2 card In-Reply-To: <007001d6ffbf$5baab980$13002c80$@schmid.xxx> References: <007001d6ffbf$5baab980$13002c80$@schmid.xxx> Message-ID: Hi Ralph, > How can I recognize such a card, from some marking or so? As I mentioned in my original solicitation, I have a friend who has sysmoSIM-GR1 and sysmoUSIM-GR1 cards, but no sysmoSIM-GR2. My friend took some pictures of his GR1 cards, i.e., Sysmocom's other historical long-discontinued cards: https://www.freecalypso.org/members/falcon/pictures/SIMs/sysmo_GR1_SIMs.jpeg As you can see from these pictures, Sysmocom's brand printing on these ancient cards is the same as on sysmoUSIM-SJS1 which they only recently discontinued: http://shop.sysmocom.de/products/sysmousim-sjs1 Given that sysmoSIM-GR2 must be newer than sysmoSIM-GR1 but older than sysmoUSIM-SJS1 (not sure about the chronological ordering between sysmoSIM-GR2 and sysmoUSIM-GR1), and given that Sysmocom's brand printing remained unchanged between sysmoSIM-GR1, sysmoUSIM-GR1 and sysmoUSIM-SJS1, I can only reason that sysmoSIM-GR2 must have the same brand printing on it - although I have no way of knowing what the background color was, as they apparently used a different color for each variant. (Their current sysmoISIM-SJA2 cards are not only unattractive black, but have different brand printing too, shifting advertising focus from Sysmocom to Osmocom and from GSM to newer G's.) It is also worth noting that as one can see in the above photo, sysmoSIM-GR1 and sysmoUSIM-GR1 cards had 8 contact pads in the IC area rather than just the required 6. I naturally have no idea if these extra C4 and C8 contacts have any functional circuits connected to them (USB-ICC or somesuch) or if they are entirely non-functional, but they are there. As I was negotiating with Grcard in China at the end of Jan into the beginning of Feb about getting a few sample pieces of their current GSM-only (no USIM or ISIM) card that can also be cut in 2FF-only form factor if the customer so desires, they sent me some pictures of those sample pieces they were going to send me, before the shipping process hit a snafu. The cards in those pictures were solid white without any markings whatsoever, but lo and behold, the IC area exhibited 8 contacts just like in those sysmoSIM-GR1 and sysmoUSIM-GR1 pictures! I can only reason that Grcard's main products these days are probably for LTE/5G in 4FF or triple-cut form factor and thus have only 6 contacts, but they still have those very old-style cards with 8 contacts in the IC area. So to summarize, I don't know exactly how sysmoSIM-GR2 cards looked, but the available evidence points to them having the following visual characteristics: * Same Sysmocom brand printing as sysmoSIM-GR1, sysmoUSIM-GR1 and sysmoUSIM-SJS1, although with an unknown background color; * 8 contacts in the IC area; * Either 2FF-only or 2FF+3FF cut (I have no way of knowing), but no 4FF cut, as the latter is incompatible with having 8 contacts. > Then I can check what I have around... If you have a card that is still fully intact, it should be extremely easy to identify - Sysmocom's brand printing is something one can't miss, and it would certainly say sysmoSIM-GR2 on it somewhere, in the same place where other Sysmocom cards have their respective model names printed. If the card is broken out as 2FF or 3FF, it wouldn't be as obvious, but on all other historical Sysmocom cards the 2FF piece also carries the Sysmocom brand name and the model name, although naturally in much smaller print. (Looking at the picture of sysmoUSIM-GR1 my friend sent me, it looks like that one was cut for 2FF+3FF, but breaking it out as 3FF would break the printing text that spans the whole 2FF piece - perhaps sysmoSIM-GR2 was the same in this regard?) It would certainly help if Harald or someone else from Sysmocom could tell us what background color their old sysmoSIM-GR2 cards were printed in - then knowing the color would help in sifting through broken-out 2FF or 3FF cards. And yes, in the interest of full disclosure, I am seeking to re-create a product that (if successful) would probably be equivalent to the long-discontinued sysmoSIM-GR2. Or to be more precise, exactly recreating sysmoSIM-GR2 is not my specific goal (how can I recreate something I don't have, have never seen, and don't really know - obviously impossible), but *if* the cards which Grcard supposedly has available currently (stuck in shipping snafu right now) are the same platform as what sysmoSIM-GR2 used (called GrcardSIM2 in Osmocom land), then indeed the new cards I am trying to get made with pretty printing should be equivalent to sysmoSIM-GR2 - but right now there are too many unknowns. M~ From saturn at rocketship.com Thu Feb 11 22:03:45 2021 From: saturn at rocketship.com (Saturn Rocket) Date: Thu, 11 Feb 2021 23:03:45 +0100 Subject: Inexpensive Huawei GSM Picocells Message-ID: An HTML attachment was scrubbed... URL: From laforge at osmocom.org Fri Feb 12 09:50:46 2021 From: laforge at osmocom.org (Harald Welte) Date: Fri, 12 Feb 2021 10:50:46 +0100 Subject: Inexpensive Huawei GSM Picocells In-Reply-To: References: Message-ID: Hi! On Thu, Feb 11, 2021 at 11:03:45PM +0100, Saturn Rocket wrote: > Huawei BTS3900B, 900/1800MHz variant. New in box, quite a few available > with free international shipping. > > Supposedly speaks Abis. Well, if it's a BTS, by definition it speaks Abis. However, Abis is not fully standardized, and every vendor has their own dialect. We reverse-engineered the dialects of Siemens, Nokia, Ericsson and ip.access to support them in OpenBSC, OsmoNITB and today OsmoBSC. However, what is required for anyone looking at implementing this, is at the very least some protocol traces from a working installation of those (or at least any other) Huawei BTS towards a Huawei BSC. Also, as the documentation states it uses IPsec, there is the question on how to configure the BTS with the correct configuration/credentials to connect to your own IPsec gateway. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From saturn at rocketship.com Fri Feb 12 16:45:45 2021 From: saturn at rocketship.com (Saturn Rocket) Date: Fri, 12 Feb 2021 17:45:45 +0100 Subject: Inexpensive Huawei GSM Picocells In-Reply-To: References: Message-ID: > Well, if it's a BTS, by definition it speaks Abis. However, Abis is not > fully standardized, and every vendor has their own dialect. We reverse-engineered > the dialects of Siemens, Nokia, Ericsson and ip.access to support them in OpenBSC, > OsmoNITB and today OsmoBSC. Whoops, Brain blip on my part... It speaks the Huawei flavour of Abis over IP, and not over E1. There seems to be quite a few bits of documentation on scribd https://www.scribd.com/search?content_type=documents&page=1&query=bts3900b&language=1 Came across this blog too: https://nickvsnetworking.com/tag/bts3900/ Seems they've been experimenting with an larger sibling of this Picocell. Best, -Saturn From laforge at gnumonks.org Fri Feb 12 18:30:50 2021 From: laforge at gnumonks.org (Harald Welte) Date: Fri, 12 Feb 2021 19:30:50 +0100 Subject: Inexpensive Huawei GSM Picocells In-Reply-To: References: Message-ID: On Fri, Feb 12, 2021 at 05:45:45PM +0100, Saturn Rocket wrote: > Seems they've been experimenting with an larger sibling of this Picocell. Well, the fact that Huawei calls their Macro BTS 3900 and the Pico 3900B doesn't really mean they have anything in common, except the branding. I would typically be very surprised if a lot of similarity in terms of code / system architecture is found between femto/picocells and the large macro cells. However, the Abis protocol dialect will more likely be the same, as a vendor doesn't want to rewrite the BSC just because a new BTS model is released. Based on what I read about the 3900B, I think it will be very hard to support it. Contrary to all the other vendors / BTSs we've worked with, there doesn't really seem to be a real LMT for it (LMT means Local Maintenance Terminal). That LMT is what you connect via UART or Ethernet to the BTS in order to configure it. Ericsson calls it OMT. For Huawei, there's also a "LMT", but it actually connects over IP to the BSC (!) and then accesses the BTS over the Backhaul from the BSC side. I guess they didn't really understand the "Local" part? The actual BTS software is then also installed from the BSC. So without having the BSC, It will definitely be much harder to even configure such a BTS. Nokia/Siemens/Ericsson/ip.access are definitely simpler that way (not listing osmobts based BTSs here, as they obviously are easiest to set up). -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From domi at tomcsanyi.net Sat Feb 13 12:44:23 2021 From: domi at tomcsanyi.net (Tomcsanyi, Domonkos) Date: Sat, 13 Feb 2021 13:44:23 +0100 Subject: Inexpensive Huawei GSM Picocells In-Reply-To: References: Message-ID: <205E41EA-B574-4B32-85BE-21E143293E06@tomcsanyi.net> Interesting to hear about an LMT not being present, because all Huawei products I encountered provided a usable WebUI via a local ethernet port. Of course I must disclose these were a lot newer products, mainly supporting LTE and NR, sometimes UMTS. MML scripting is also present on those for quicker configuration. Oh well, it wasn?t like that in the old days I guess :-). Kind regards, Domi > 12.02.2021 d?tummal, 19:40 id?pontban Harald Welte ?rta: > > ?On Fri, Feb 12, 2021 at 05:45:45PM +0100, Saturn Rocket wrote: >> Seems they've been experimenting with an larger sibling of this Picocell. > > Well, the fact that Huawei calls their Macro BTS 3900 and the Pico 3900B > doesn't really mean they have anything in common, except the branding. I would > typically be very surprised if a lot of similarity in terms of code / system > architecture is found between femto/picocells and the large macro cells. > > However, the Abis protocol dialect will more likely be the same, as a vendor doesn't > want to rewrite the BSC just because a new BTS model is released. > > Based on what I read about the 3900B, I think it will be very hard to support it. > > Contrary to all the other vendors / BTSs we've worked with, there doesn't really > seem to be a real LMT for it (LMT means Local Maintenance Terminal). That LMT is > what you connect via UART or Ethernet to the BTS in order to configure it. Ericsson > calls it OMT. > > For Huawei, there's also a "LMT", but it actually connects over IP to the BSC (!) > and then accesses the BTS over the Backhaul from the BSC side. I guess they > didn't really understand the "Local" part? > > The actual BTS software is then also installed from the BSC. > > So without having the BSC, It will definitely be much harder to even > configure such a BTS. Nokia/Siemens/Ericsson/ip.access are definitely > simpler that way (not listing osmobts based BTSs here, as they obviously > are easiest to set up). > > -- > - Harald Welte http://laforge.gnumonks.org/ > ============================================================================ > "Privacy in residential applications is a desirable marketing option." > (ETSI EN 300 175-7 Ch. A6) From laforge at gnumonks.org Sat Feb 13 13:29:30 2021 From: laforge at gnumonks.org (Harald Welte) Date: Sat, 13 Feb 2021 14:29:30 +0100 Subject: Inexpensive Huawei GSM Picocells In-Reply-To: <205E41EA-B574-4B32-85BE-21E143293E06@tomcsanyi.net> References: <205E41EA-B574-4B32-85BE-21E143293E06@tomcsanyi.net> Message-ID: Hi Domi, On Sat, Feb 13, 2021 at 01:44:23PM +0100, Tomcsanyi, Domonkos wrote: > Interesting to hear about an LMT not being present, because all Huawei products I encountered provided a usable WebUI via a local ethernet port. Of course I must disclose these were a lot newer products, mainly supporting LTE and NR, sometimes UMTS. MML scripting is also present on those for quicker configuration. those were also likely macro base stations, right? Where you a) expect no physical access to the general public, b) have a dedicated Ethernet port for the "site LAN" For femto / small cell products, you typically expect them to be in publicly accessible locations, where virtually anyone can get physical access. So the vendors lock them down considerably more, with no "site lan", enforced use of IPsec, etc. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From laforge at osmocom.org Sat Feb 13 17:08:16 2021 From: laforge at osmocom.org (Harald Welte) Date: Sat, 13 Feb 2021 18:08:16 +0100 Subject: testing TTCN3 test stability Message-ID: I think this might be of interest to some of the other developers. Soemtimes we have a problem that the test suite itself is not 100% stable, and it's not easy to reproduce a test that fails sporadically every Nth time. As the "execute" statement in the "control" section actually returns a verdict type, you can use the following construct in a control section: control { for (var integer i := 0; i < 10000; i := i+1) { var verdicttype v; v := execute( TC_paging_ps_ptp_lac() ); if (v != pass) { break; } } } This way you keep running the test suite until the given test fails for the first time. Then go do something else for some hour(s) and by the time you get back, hopefully you will have reproduced the problem. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From northmirko at gmail.com Sat Feb 13 18:27:34 2021 From: northmirko at gmail.com (Mirko Kovacevic) Date: Sat, 13 Feb 2021 19:27:34 +0100 Subject: testing TTCN3 test stability In-Reply-To: References: Message-ID: Harald/Osmocom team gave TTCN3 real meaning and public attention. Before you came up, TTCN3 was a prisoner of worthless, dry academic papers. On Sat, 13 Feb 2021, 18:08 Harald Welte, wrote: > I think this might be of interest to some of the other developers. > > Soemtimes we have a problem that the test suite itself is not 100% stable, > and it's not easy to reproduce a test that fails sporadically every Nth > time. > > As the "execute" statement in the "control" section actually returns a > verdict > type, you can use the following construct in a control section: > > control { > for (var integer i := 0; i < 10000; i := i+1) { > var verdicttype v; > v := execute( TC_paging_ps_ptp_lac() ); > if (v != pass) { > break; > } > } > } > > This way you keep running the test suite until the given test fails for > the first time. Then go do something else for some hour(s) and by the > time you get back, hopefully you will have reproduced the problem. > > Regards, > Harald > -- > - Harald Welte > http://laforge.gnumonks.org/ > > ============================================================================ > "Privacy in residential applications is a desirable marketing option." > (ETSI EN 300 175-7 Ch. > A6) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From domi at tomcsanyi.net Sat Feb 13 20:31:14 2021 From: domi at tomcsanyi.net (Tomcsanyi, Domonkos) Date: Sat, 13 Feb 2021 21:31:14 +0100 Subject: Inexpensive Huawei GSM Picocells In-Reply-To: References: Message-ID: Hi Harald, 100% correct indeed. Kind regards, Domi > 13.02.2021 d?tummal, 14:30 id?pontban Harald Welte ?rta: > > ?Hi Domi, > >> On Sat, Feb 13, 2021 at 01:44:23PM +0100, Tomcsanyi, Domonkos wrote: >> Interesting to hear about an LMT not being present, because all Huawei products I encountered provided a usable WebUI via a local ethernet port. Of course I must disclose these were a lot newer products, mainly supporting LTE and NR, sometimes UMTS. MML scripting is also present on those for quicker configuration. > > those were also likely macro base stations, right? Where you > a) expect no physical access to the general public, > b) have a dedicated Ethernet port for the "site LAN" > > For femto / small cell products, you typically expect them to be in publicly > accessible locations, where virtually anyone can get physical access. So the > vendors lock them down considerably more, with no "site lan", enforced use of > IPsec, etc. > > -- > - Harald Welte http://laforge.gnumonks.org/ > ============================================================================ > "Privacy in residential applications is a desirable marketing option." > (ETSI EN 300 175-7 Ch. A6) From laforge at osmocom.org Sun Feb 14 08:16:46 2021 From: laforge at osmocom.org (Harald Welte) Date: Sun, 14 Feb 2021 09:16:46 +0100 Subject: testing TTCN3 test stability In-Reply-To: References: Message-ID: On Sat, Feb 13, 2021 at 07:27:34PM +0100, Mirko Kovacevic wrote: > Harald/Osmocom team gave TTCN3 real meaning and public attention. Before > you came up, TTCN3 was a prisoner of worthless, dry academic papers. thanks for the feedback. I think the real difference only is that we develop our TTCN-3 code as open source. There were and are quite a number of users of TTCN-3 before outside of academia - but for some strange reason none of them (I know of) seems to have been doing open source development. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From laforge at osmocom.org Tue Feb 16 15:23:12 2021 From: laforge at osmocom.org (Harald Welte) Date: Tue, 16 Feb 2021 16:23:12 +0100 Subject: RFC: generic CTRL interface for VTY config settings Message-ID: Dear all, we always had said the VTY is a human CLI, while the CTRL interface is for interfacing with other software. This was primarily related to asking users to not parse the VTY output of "show" commands or the like. For configuration via VTY, we have to provide a stable interface anyway, as otherwise loading config files would not be possible across [non-major] software upgrades. While that is right in principle, the fact that we don't have a generic configuration store / API / MIB means that one would explicitly have to add CTRL command for all relevant settings, which may have been realistic in 2010, but is unrealistic with the hundreds of VTY configuration parameters today. I was thinking wither it might make sense to add a generic CTRL interface GET/SET for "configuration" mode VTY settings. This would mean we write code once, and immediately expose all our VTY nodes to CTRL. Thinking of the following example done via VTY: ----------------------------------- OsmoBSC> enable OsmoBSC# configure terminal OsmoBSC(config)# network OsmoBSC(config-net)# bts 0 OsmoBSC(config-net-bts)# trx 1 OsmoBSC(config-net-bts-trx)# timeslot 0 OsmoBSC(config-net-bts-trx-ts)# training_sequence_code 3 ----------------------------------- We could have something like the following CTRL command: SET network.bts.0.trx.1.timeslot.0.training_sequence_code 3 The devil is a bit in the details of the syntax * the normal '.' as separator looks generally OK, I think we have [almost?] no VTY commands that include a ',' * we somehow need some kind of separator to tell where individual VTY commands are separated, like SET network."bts 0"."trx 1"."timeslot 0".training_sequence_code 3 or SET network|bts.0|trx.1|timeslot.0|training_sequence_code 3 or SET network#bts.0#trx.1#timeslot.0#training_sequence_code 3 or SET network.bts#0.trx#1.timeslot#0.training_sequence_code 3 SET network.bts at 0.trx@1.timeslot at 0.training_sequence_code 3 I think the last examples would be more "natural" for existing CTRL interface code, as the dot separat separates tokens, and we simply expand the '#' or '@' to spaces in the context of VTY commands The generic vty-cfg-via-CTRL code then would basically emulate a user entering the sequence of commands - including the "exclusion" of not allowing multiple concurrent VTY or vty-cfg-via-CTRL users at the same time What do you guys think? Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From nhofmeyr at sysmocom.de Tue Feb 16 16:23:10 2021 From: nhofmeyr at sysmocom.de (Neels Hofmeyr) Date: Tue, 16 Feb 2021 17:23:10 +0100 Subject: RFC: generic CTRL interface for VTY config settings In-Reply-To: References: Message-ID: <20210216162222.GB1646@my.box> I welcome a solution like this very much. However, it wouldn't be a real solution to the human vs machine interaction issue. There are a lot of cans of worms hidden in this problem space and our status quo. The notion that we should only use the CTRL interface for automated interaction sounds like a good idea, but is quite impractical IMO. For TTCN3 tests or at congress, for example, it is so simple to just interact with the VTY instead of first implementing CTRL commands that do the same thing (and merging patches and so on), and the CTRL also has some problems in its protocol design... So in my own opinion / daily practice, I have kind of let go of this notion that the VTY is for humans only. What you are proposing is an interesting solution for structured input into a program, but there is no easy way for *querying* structured information from a program's VTY existing implementation in that way. We can't only expose those VTY items that don't return any output (we have no way to distinguish those -- so we also can't easily translate GET and SET to VTY commands), so this will probably just return the same vty_out() as a VTY command would print, only on CTRL. That kind of obsoletes the CTRL interface, might as well just use the VTY, if it returns the same output anyway. Related to that is the lack of structure in the VTY: we don't formally compose a response and send it, instead we vty_out() maybe N times, maybe not... We'd have to redirect the vty_out() to CTRL somehow. There's also the question of receiving larger responses, think a list of subscribers -- so far CTRL clients expect the entire response in one read(). I'm not entirely sure whether it is a fundamental flaw in the CTRL implementation, but AFAIR I have often looked at CTRL clients and found a lack of structuring there, on a protocol level. It would certainly be a problem if a multi-packet response by accident had a "GET_REPLY" at the start of a subsequent packet. We don't properly escape response content... Doing both input and querying properly would IMO mean a lot of re-implementing. Every now and then I thought a bit about maybe implementing an object model where a command returns certain named values / lists thereof, and the VTY is then a renderer to put those into human readable formats, while the CTRL would return those values raw in a nicely parseable way (a bit like SQL responses). But that would also mean quite a lot of more effort to implement a VTY command compared to the simple C-code and vty_out() we use now. (let alone re-implementing every VTY command we have so far.) In the end I so far mostly conclude that it is too much effort to rewrite the entire VTY, and that it is simplest / most practical to actually ignore the CTRL interface completely, also for machine interaction :/ ~N On Tue, Feb 16, 2021 at 04:23:12PM +0100, Harald Welte wrote: > Dear all, > > we always had said the VTY is a human CLI, while the CTRL interface is for > interfacing with other software. This was primarily related to asking users > to not parse the VTY output of "show" commands or the like. For configuration > via VTY, we have to provide a stable interface anyway, as otherwise loading > config files would not be possible across [non-major] software upgrades. > > While that is right in principle, the fact that we don't have a generic configuration > store / API / MIB means that one would explicitly have to add CTRL command for all relevant > settings, which may have been realistic in 2010, but is unrealistic with the hundreds of > VTY configuration parameters today. > > I was thinking wither it might make sense to add a generic CTRL interface GET/SET > for "configuration" mode VTY settings. This would mean we write code once, > and immediately expose all our VTY nodes to CTRL. > > Thinking of the following example done via VTY: > > ----------------------------------- > OsmoBSC> enable > OsmoBSC# configure terminal > OsmoBSC(config)# network > OsmoBSC(config-net)# bts 0 > OsmoBSC(config-net-bts)# trx 1 > OsmoBSC(config-net-bts-trx)# timeslot 0 > OsmoBSC(config-net-bts-trx-ts)# training_sequence_code 3 > ----------------------------------- > > We could have something like the following CTRL command: > > SET network.bts.0.trx.1.timeslot.0.training_sequence_code 3 > > The devil is a bit in the details of the syntax > > * the normal '.' as separator looks generally OK, I think we have [almost?] no > VTY commands that include a ',' > > * we somehow need some kind of separator to tell where individual VTY commands are > separated, like > > SET network."bts 0"."trx 1"."timeslot 0".training_sequence_code 3 > or > SET network|bts.0|trx.1|timeslot.0|training_sequence_code 3 > or > SET network#bts.0#trx.1#timeslot.0#training_sequence_code 3 > or > SET network.bts#0.trx#1.timeslot#0.training_sequence_code 3 > SET network.bts at 0.trx@1.timeslot at 0.training_sequence_code 3 > > I think the last examples would be more "natural" for existing CTRL interface code, > as the dot separat separates tokens, and we simply expand the '#' or '@' to spaces in > the context of VTY commands > > The generic vty-cfg-via-CTRL code then would basically emulate a user entering > the sequence of commands - including the "exclusion" of not allowing multiple concurrent > VTY or vty-cfg-via-CTRL users at the same time > > What do you guys think? > > Regards, > Harald > > -- > - Harald Welte http://laforge.gnumonks.org/ > ============================================================================ > "Privacy in residential applications is a desirable marketing option." > (ETSI EN 300 175-7 Ch. A6) -- - Neels Hofmeyr http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Gesch?ftsf?hrer / Managing Directors: Harald Welte From laforge at osmocom.org Tue Feb 16 17:49:26 2021 From: laforge at osmocom.org (Harald Welte) Date: Tue, 16 Feb 2021 18:49:26 +0100 Subject: RFC: generic CTRL interface for VTY config settings In-Reply-To: <20210216162222.GB1646@my.box> References: <20210216162222.GB1646@my.box> Message-ID: On Tue, Feb 16, 2021 at 05:23:10PM +0100, Neels Hofmeyr wrote: > I welcome a solution like this very much. However, it wouldn't be a real > solution to the human vs machine interaction issue. There are a lot of cans of > worms hidden in this problem space and our status quo. It is not a *solution*. The only real solution is https://osmocom.org/issues/1975 > The notion that we should only use the CTRL interface for automated interaction > sounds like a good idea, but is quite impractical IMO. For TTCN3 tests [...] I would exclude tests here, as a) tests need to do super strange things at times, like resetting state, which never happens in production use b) tests also want to test the VTY interface itself > congress, for example, it is so simple to just interact with the VTY instead of > and so on), and the CTRL also has some problems in its protocol design... So in > my own opinion / daily practice, I have kind of let go of this notion that the > VTY is for humans only. It depends on what. I'm happy with programmatic access to the VTY for simple operations like changing a certain value (basically, anything below CONFIG_NODE). I am still fundamentally and very strictly opposed to using the "show" and related commands from programs, as that is super volatile and breaks every time the output is formatted differently. > What you are proposing is an interesting solution for structured input into a > program, but there is no easy way for *querying* structured information from a > program's VTY existing implementation in that way. We can't only expose those > VTY items that don't return any output (we have no way to distinguish those -- > so we also can't easily translate GET and SET to VTY commands), Anything below the config node is not supposed to generate any output - except in error cases where CMD_WARNING is a clear indicator that something went wrong. The "get" part is probably more tricky as we'd have to generate a full 'write terminal' internally and then walk that from the "generic" code > so this will probably just return the same vty_out() as a VTY command would print, only on as stated above, this is intended for setting and hopefully getting single configuration values, nothing else. > There's also the question of receiving larger responses, think a list of > subscribers -- that also is not the point. You don't have lists of subscribers below the config node. > In the end I so far mostly conclude that it is too much effort to rewrite the > entire VTY, and that it is simplest / most practical to actually ignore the > CTRL interface completely, also for machine interaction :/ This proposal is intended as an interim improvement for some use cases, until we hopefully at some point get to https://osmocom.org/issues/1975 -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From keith at rhizomatica.org Tue Feb 16 18:40:38 2021 From: keith at rhizomatica.org (Keith) Date: Tue, 16 Feb 2021 12:40:38 -0600 Subject: RFC: generic CTRL interface for VTY config settings In-Reply-To: References: Message-ID: <70b4b86b-6118-aeb5-f935-6ef86b5e90d9@rhizomatica.org> On 16/02/2021 09:23, Harald Welte wrote: > Dear all, > > we always had said the VTY is a human CLI, while the CTRL interface is for > interfacing with other software. This was primarily related to asking users > to not parse the VTY output of "show" commands or the like. You mean like: expect -c 'spawn -noecho telnet 0 4260; expect >; send enable\r; expect #; send "show pdp-context ggsn ggsn0\r"; expect #' | egrep 'IMSI|Control|End-User' | gawk 'BEGIN { RS="IMSI" } /./ { sub(/:[^\$]*/, "", $10); sub(/\r/, "", $14); print "\033[32;1m SGSN: \ ?"$10"\033[30D\033[20C \033[33;1mIP: "$13" "$14"\033[36;1m \033[100D \033[50C IMSI: "$2"\033[1D \033[0m" }' | sort -t. -n -k3,3n -k4,4n -k6,6n -k7,7n :)) Yes, I'm guilty of a lot of this... But this proposal doesn't actually seem to be about that, but rather about an automagic way to expose anything added to the config nodes with SET... > SET network.bts#0.trx#1.timeslot#0.training_sequence_code 3 > which sounds good to me, and I like the dot separator. and replacing space with #. I don't think I have a lot of everyday use for modifying runtime configuration, although I have thought recently that maybe the best way to recover quickly from a power failure induced msc restart and loss of volatile VLR is to assign two LAC to each site and alternate them on restarts, forcing all the MS to LU as soon as the network is back up. I would probably do this by modifying a config file (with sed or such) before starting the daemons, but maybe a CTRL command is better, which would have to be followed by a BTS SI update, right?? but I don't mean to hijack the thread with that.. :) Anyway, Matt at UW recently added osmopy/osmo_ipa to the rhizomatica code, so maybe we'd make more use of CTRL in future. k. From laforge at osmocom.org Tue Feb 16 19:49:24 2021 From: laforge at osmocom.org (Harald Welte) Date: Tue, 16 Feb 2021 20:49:24 +0100 Subject: RFC: generic CTRL interface for VTY config settings In-Reply-To: <70b4b86b-6118-aeb5-f935-6ef86b5e90d9@rhizomatica.org> References: <70b4b86b-6118-aeb5-f935-6ef86b5e90d9@rhizomatica.org> Message-ID: Hi Keith, On Tue, Feb 16, 2021 at 12:40:38PM -0600, Keith wrote: > > SET network.bts#0.trx#1.timeslot#0.training_sequence_code 3 > > which sounds good to me, and I like the dot separator. and replacing > space with #. thanks. > I don't think I have a lot of everyday use for modifying runtime > configuration, although I have thought recently that maybe the best way > to recover quickly from a power failure induced msc restart and loss of > volatile VLR is to assign two LAC to each site and alternate them on > restarts, forcing all the MS to LU as soon as the network is back up. > > I would probably do this by modifying a config file (with sed or such) > before starting the daemons. The point is to change runtime configuration (from external tools / management) without restarting the respective program. Sure, if you restart anyway, you can modify the config all you want. However, You don't normally want to restart a BSC and loose all calls of all BTSs (which can be many hundred concurrent calls) just because you want to change e.g. the permitted ciphers, codec configuration, transmit power reduction, ... of one specific cell. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From pespin at sysmocom.de Wed Feb 17 21:30:25 2021 From: pespin at sysmocom.de (Pau Espin Pedrol) Date: Wed, 17 Feb 2021 22:30:25 +0100 Subject: RFC: generic CTRL interface for VTY config settings In-Reply-To: References: Message-ID: <3c30d279-3e1d-5937-1ace-f067c9ae66b9@sysmocom.de> Hi, > SET network|bts.0|trx.1|timeslot.0|training_sequence_code 3 > or > SET network#bts.0#trx.1#timeslot.0#training_sequence_code 3 I'd probably go for this one from all the ones you proposed. This way, you keep the usual CTRL structure ("." separating/slitting parameters), while you simply add a new character to specify "levels" or "nodes" (as per VTY terminology). Hence, each node is parsed the same way as the final command, and you can "easily" feed that into navigating the existing VTY tree. Example in pythonic way: str = "network|bts.0|trx.1|timeslot.0|training_sequence_code 3" nodes = str.split('|') for node in nodes: argv = node.split('.') if attempt_to_match_vty_node_or_command(argv) < 0: return cmd_reply_error; return The question here is how to define information in code to apply getter/setters for CTRL commands... I also find that the CTRL format in general is a bit restraining, both in specifying parameters, etc. as well as in reply content. Somebody may not like the idea, (and may be not directly related to the topic at hand), but perhaps looking at something similar to a REST interface which provides json output and which can easily be interacted using python or even curl directly. REST interface provides already way to provide levels/nodes (URL directories)as well as parameters "?foo=bar&hello=bye"). So using HTTP may not be needed, but it may make sense to reuse some REST related ideas, like URLs and json output (with C implementation in 1 file like cjson). Regards, Pau -- - Pau Espin Pedrol http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte From laforge at gnumonks.org Thu Feb 18 01:05:26 2021 From: laforge at gnumonks.org (Harald Welte) Date: Thu, 18 Feb 2021 02:05:26 +0100 Subject: RFC: generic CTRL interface for VTY config settings In-Reply-To: <3c30d279-3e1d-5937-1ace-f067c9ae66b9@sysmocom.de> References: <3c30d279-3e1d-5937-1ace-f067c9ae66b9@sysmocom.de> Message-ID: Hi Pau, On Wed, Feb 17, 2021 at 10:30:25PM +0100, Pau Espin Pedrol wrote: > > SET network|bts.0|trx.1|timeslot.0|training_sequence_code 3 > > or > > SET network#bts.0#trx.1#timeslot.0#training_sequence_code 3 > > I'd probably go for this one from all the ones you proposed. This way, you > keep the usual CTRL structure ("." separating/slitting parameters), while > you simply add a new character to specify "levels" or "nodes" (as per VTY > terminology). ACK. > The question here is how to define information in code to apply > getter/setters for CTRL commands... Not sure I'm following you here? > Somebody may not like the idea, (and may be not directly related to the > topic at hand), but perhaps looking at something similar to a REST interface > which provides json output and which can easily be interacted using python > or even curl directly. > REST interface provides already way to provide levels/nodes (URL > directories)as well as parameters "?foo=bar&hello=bye"). > So using HTTP may not be needed, but it may make sense to reuse some REST > related ideas, like URLs and json output (with C implementation in 1 file > like cjson). CTRL started with the idea that somebody wants to perform basic SNMP style SET/GET/TRAP operations, but we didn't want to import the complexity of "Simple" SNMP parsing/encoding/... in a non-blocking fashion. It was never intended to do anything else but setting or getting a single value in each command, and sending the occasional TRAP. I do think whatever we do as a next iteration (if any) should center around that external MIB / configuration store idea. At that point the osmo-* program becomes a client to the config store, ideally with transaction/commit/rollback logic, and anyone can come up with whatever interface they want towards that config store - if it's not already a well-documented and established interface to begin with. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From nhofmeyr at sysmocom.de Thu Feb 18 12:17:53 2021 From: nhofmeyr at sysmocom.de (Neels Hofmeyr) Date: Thu, 18 Feb 2021 13:17:53 +0100 Subject: RFC: generic CTRL interface for VTY config settings In-Reply-To: References: Message-ID: <20210218121753.GA12346@my.box> On Tue, Feb 16, 2021 at 04:23:12PM +0100, Harald Welte wrote: > SET network.bts.0.trx.1.timeslot.0.training_sequence_code 3 > > The devil is a bit in the details of the syntax > > * the normal '.' as separator looks generally OK, I think we have [almost?] no > VTY commands that include a ',' > > * we somehow need some kind of separator to tell where individual VTY commands are > separated, like > > SET network."bts 0"."trx 1"."timeslot 0".training_sequence_code 3 > or > SET network|bts.0|trx.1|timeslot.0|training_sequence_code 3 > or > SET network#bts.0#trx.1#timeslot.0#training_sequence_code 3 > or > SET network.bts#0.trx#1.timeslot#0.training_sequence_code 3 > SET network.bts at 0.trx@1.timeslot at 0.training_sequence_code 3 We also have numerous commands where there are spaces that would look weird when written as e.g. # or @: network handover1 power budget hysteresis (<0-999>|default) We have commands where the values are interleaved with keyword tokens, i.e. where there is no clean name -> value relation: neighbor lac <0-65535> arfcn <0-1023> bsic (<0-63>|any) We have commands that are additive: si2quater neighbor-list add earfcn <0-65535> thresh-hi <0-31> thresh-lo <0-32> prio <0-8> qrxlv <0-32> meas <0-8> si2quater neighbor-list del earfcn <0-65535> Commands without a value part: allow-attach no periodic location update Trying to translate all of these cases into CTRL-like name->value feels very hacky to me, with many quirks and pitfalls waiting on us around numerous corners. We'll run into ambiguities, escaping issues, and unreadable commands. A cleaner approach seems to me to have a CTRL command that invokes VTY lines, with a defined line separator, with VTY commands as-is in the "value" part of the CTRL command. E.g.: SET config network;bts 0;trx 1;timeslot 0;training_sequence_code 3 Then we don't need to worry about translating VTY commands to valid CTRL identifiers, and except for concatenating multiple lines the VTY commands can be run 1:1 without translation mangling in-between. Many problems gone. As line separator, ';' comes to mind from common programming languages, or '%' or '#' since those two are otherwise used for comments in the VTY and will not likely cause ambiguities. We could even send LF as line separator like: SET config network bts 0 trx 1 timeslot 0 training_sequence_code 3 so that the "value" part of the CTRL command is *exactly* the same as we would feed to a telnet client. At this point I'm asking myself again, what is the aim here -- an external program that can connect to CTRL might as well also connect to VTY and can already issue these config commands there. Is the aim to save one connection? I still have the impression that this idea is well meant but in reality misses practicality and usefulness. It's just a one-shot telnet client piped through CTRL, in the worst case with lots of mangling involved. where is the benefit? Would be nice if it made sense and things would fall into place, but AFAICT it unfortunately simply doesn't work out. VTY as we implemented it is too free-format and unstructured, we won't be able to tame it well. ~N From matthias.simon at nokia.com Thu Feb 18 13:25:39 2021 From: matthias.simon at nokia.com (Simon, Matthias (Nokia - DE/Ulm)) Date: Thu, 18 Feb 2021 13:25:39 +0000 Subject: Introducing ntt. Modern tools for TTCN-3 Message-ID: Hi, I am Matthias, a TTCN-3 tool developer from Nokia. Thank you for opening your work. Your repository osmo-ttcn3-hacks helped me to understand how other developers use TTCN-3 for testing. This gave me valuable insights for tool development. I hope I can give something back by opening up our internal TTCN-3 tools as well. Harald suggested to introduce them on this list. I'd appreciate your feedback. Our project is called ntt and it provides various tools for working with TTCN-3 [1]. For example, you can create a tags file: $ ntt tags ./osmo-ttcn3-hacks/bts >TAGS Or you can list things. To filter cyclic imports, for example: $ ntt list imports ./osmo-ttcn3-hacks/bts | tsort 1>/dev/null But the most interesting piece is probably the TTCN-3 language server: ntt implements the language server protocol. This makes ntt a universal TTCN-3 language plugin for virtually any editor or IDE [2]. It's still very much beta, but we are finishing the Go to Definition feature (as replacement for ctags files). The next feature would be adding various diagnostics. It would be great if you could give ntt a try and share your experience or editor configuration with me. This is my first open source project and we still have to figure some stuff out. So please excuse if there are still some rough edges. Cheers, Matthias [1] https://nokia.github.io/ntt/ [2] https://nokia.github.io/ntt/editors/ From laforge at osmocom.org Fri Feb 19 08:54:29 2021 From: laforge at osmocom.org (Harald Welte) Date: Fri, 19 Feb 2021 09:54:29 +0100 Subject: Idea for 2021: Regular "OsmoDevCall" In-Reply-To: References: Message-ID: Dear Osmocom community, On Thu, Dec 31, 2020 at 12:04:45PM +0100, Harald Welte wrote: > as the pandemic continues and physical meetings are out of the question > for the forseeable future, it would be a good idea to have a periodic > virtual online meeting of the interested Osmocom community. After some discussion in https://osmocom.org/issues/4928 and on IRC, I've now put together a wiki page at https://osmocom.org/projects/osmo-dev-con/wiki/OsmoDevCall Right now there's still a chance to influence the scheduling (current proposal is Fridays 8pm to 10pm CET). If you would want to participate but think this is a really bad time, please speak up now :) Once it is clear where we will host the call, I'd be announcing the first event, which could then be as early as Februray 26th. I'll keep you posted. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From laforge at osmocom.org Fri Feb 19 11:52:08 2021 From: laforge at osmocom.org (Harald Welte) Date: Fri, 19 Feb 2021 12:52:08 +0100 Subject: Introducing ntt. Modern tools for TTCN-3 In-Reply-To: References: Message-ID: Hi Matthias, thanks for reaching out. On Thu, Feb 18, 2021 at 01:25:39PM +0000, Simon, Matthias (Nokia - DE/Ulm) wrote: > Our project is called ntt and it provides various tools for working with > TTCN-3 [1]. For example, you can create a tags file: > > $ ntt tags ./osmo-ttcn3-hacks/bts >TAGS I just built it and tried that - looks fine so far. > Or you can list things. To filter cyclic imports, for example: > > $ ntt list imports ./osmo-ttcn3-hacks/bts | tsort 1>/dev/null BTS_Tests.ttcn:1889:7: illegal character U+0025 '%' I guess we are using several TITAN specific bits an features which may not be supported or intended to support in ntt? by the way, did you consider posting to the Eclipse TITAN forum? I'm sure folks over there are also interested to hear about new/additional open source TTCN3 tools. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From matthias.simon at nokia.com Fri Feb 19 14:37:44 2021 From: matthias.simon at nokia.com (Simon, Matthias (Nokia - DE/Ulm)) Date: Fri, 19 Feb 2021 14:37:44 +0000 Subject: Introducing ntt. Modern tools for TTCN-3 In-Reply-To: References: , Message-ID: Hey, > I just built it and tried that - looks fine so far. Not all TTCN-3 language constructs have the same test coverage. For example: internally we don't have tool support for TTCN-3 Procedure Based Communication, therefore we don't have tests checking if signature definitions are visited. On https://github.com/nokia/vscode-ttcn3/#troubleshooting-go-to-definition-does-not-work you'll find some additional information. > BTS_Tests.ttcn:1889:7: illegal character U+0025 '%' These are Titan macros. I just added support in ntt v0.6.2. Thanks for pointing out, Harald. Cheers, Matthias ________________________________________ From: Harald Welte Sent: Friday, February 19, 2021 12:52 PM To: Simon, Matthias (Nokia - DE/Ulm) Cc: openbsc at lists.osmocom.org Subject: Re: Introducing ntt. Modern tools for TTCN-3 Hi Matthias, thanks for reaching out. On Thu, Feb 18, 2021 at 01:25:39PM +0000, Simon, Matthias (Nokia - DE/Ulm) wrote: > Our project is called ntt and it provides various tools for working with > TTCN-3 [1]. For example, you can create a tags file: > > $ ntt tags ./osmo-ttcn3-hacks/bts >TAGS I just built it and tried that - looks fine so far. > Or you can list things. To filter cyclic imports, for example: > > $ ntt list imports ./osmo-ttcn3-hacks/bts | tsort 1>/dev/null BTS_Tests.ttcn:1889:7: illegal character U+0025 '%' I guess we are using several TITAN specific bits an features which may not be supported or intended to support in ntt? by the way, did you consider posting to the Eclipse TITAN forum? I'm sure folks over there are also interested to hear about new/additional open source TTCN3 tools. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From mychaela.falconia at gmail.com Sun Feb 21 18:35:14 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Sun, 21 Feb 2021 10:35:14 -0800 Subject: OTA RFM on sysmoISIM-SJA2 cards Message-ID: Hello Osmocom community, Has anyone here ever played with the RFM (Remote File Management) feature on sysmoUSIM-SJS1 and sysmoISIM-SJA2 cards? I know that a lot of people play with RAM (Remote Application Management) for installing Java applets, but I am more interested in RFM for doing a different kind of OTA programming - I am seeking to recreate the workflow of traditional GSM network operators where "blank" (not yet activated, but ready to activate) SIM cards sit on store shelves with their IMSI and secret keys (Ki/K/OPc etc) already programmed at the factory, but blank EF_MSISDN because the future user's phone number is not known yet. Customers activate these SIMs by reading the ICCID from the card to customer service over the phone, or salespeople in stores scan the ICCID barcode - and then the operator's customer management system matches that ICCID with its knowledge of the IMSI and secret keys, and the service gets activated on the new SIM. And then the operator's network uses SMS-PP SIM data download to program the EF_MSISDN record in the newly activated SIM - I know full well that a phone does not need to know its own MSISDN to make and receive calls, but every classic GSM dumbphone has a menu command for "Show my number" or whatever it's called, this command displays the MSISDN record from the SIM, and traditional operators program this record OTA so that this menu command will work. I am seeking to recreate this OTA programming step. I just got the needed KI[CD][23] OTA keys for my sysmoUSIM-SJS1 cards (thanks Sysmocom support!), and I am able to exercise RFM successfully on these cards by uncommenting these lines in the shadysim.py script: # for RFM testing ac.test_rfm() exit(0) It appears that the "tribal" knowledge (not written in any formal document, AFAICT) of how to use the RFM feature on sysmoUSIM-SJS1 cards exists only in the following code stanzas in shadysim.py, code that never executes unless you uncomment that ac.test_rfm() call: def send_wrapped_apdu_rfm_sim(self, data): # TAR RFM SIM: B00010, sysmoSIM SJS1: MSL = 6, second keyset return self.send_wrapped_apdu_internal(data, 'B00010', 6, 2, 2) def send_wrapped_apdu_rfm_usim(self, data): # TAR RFM USIM: B00011, sysmoSIM SJS1: MSL = 6, third keyset return self.send_wrapped_apdu_internal(data, 'B00011', 6, 3, 3) It was only thanks to the above code lines and comments that I learned that I need to use keyset 2 for SIM RFM, and how else would we know the needed magic TAR if not for the above code and comments? In any case, the RFM test function of shadysim.py works like a charm on my sysmoUSIM-SJS1 cards with the right keys (successfully displays the IMSI read out via RFM), and I am now going to work on my own C code that will replace Python and do what I need. However, I also tried the exact same shadysim.py RFM test function on the newer sysmoISIM-SJA2 cards, and it does NOT work. I run the exact same shadysim.py (modified only to uncomment the RFM test) that works against sysmoUSIM-SJS1, but when I run it against sysmoISIM-SJA2 and specify the respective card's KIC2 and KID2 from the webshop key data email, I get this output: ICCID: 8988211000000471501f ('', '') Here is the output with a good sysmoUSIM-SJS1: ICCID: 8988211000000386808f ('089910070000306808', '9000') Given that the code stays exactly the same and I am merely specifying different keys as needed for each card, there must be something different about the new sysmoISIM-SJA2 cards with respect to RFM. Perhaps the TAR is different? Perhaps the association of which keyset goes for what is different? Some other differences like different crypto algorithms being used? Perhaps a migration from 3DES to AES? I am fine with just using sysmoUSIM-SJS1 for development of my C tools (tools which will hopefully be extended later to work with other vendors' SIMs beyond just Sysmocom), but it would be nice to fill in the knowledge gap regarding sysmoISIM-SJA2 and get these cards to work as well. In hacking fellowship, Mother Mychaela From mychaela.falconia at gmail.com Mon Feb 22 02:48:29 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Sun, 21 Feb 2021 18:48:29 -0800 Subject: OTA RFM on sysmoISIM-SJA2 cards In-Reply-To: References: Message-ID: Hello again everyone, The mystery I reported this morning regarding the test_rfm function of shadysim.py working on sysmoUSIM-SJS1 but not on sysmoISIM-SJA2 has been solved. I have written my own suite of C tools to replace the Python tool for my purposes, and in the process of studying the Python code for the purpose of recreating its logic in C I found the bug that makes the original code not work with the new SIMs. The bug is in the padding logic. When the application message payload is encrypted with any variant of DES (including the two-key 3DES used on Sysmocom SIMs), the length of the ciphertext has to be a multiple of 8 bytes - hence if the plaintext length is not a multiple of 8 bytes, the plaintext needs to be padded. But what should happen if the plaintext length going into the cipher just happens to be a perfect multiple of 8 bytes? The correct answer (ought to be obvious) is to apply no padding at all, i.e., zero bytes of padding. However, the Python code in shadysim.py adds 8 bytes of padding, and sets the number of padding bytes in the header to 8. The resulting encrypted message should be considered malformed per standard specs, but sysmoUSIM-SJS1 cards are liberal in what they accept in this instance, thus the bug went unnoticed. The newer sysmoISIM-SJA2 cards do not accept such malformed messages with invalid padding, and it just so happens that the message generated by the test_rfm function has 40 bytes of plaintext going into the cipher, perfectly divisible by 8 - hence the test_rfm function fails on the new cards. I am not able to produce a patch for shadysim.py because Python is a read-only language for me: I can kinda-sorta read Python code and figure out what it's doing, so I can reproduce its logic in C or Bourne shell (my two languages), but writing or modifying Python code is a territory I don't feel ready to venture into. My new C tools for OTA SIM programming which do work with both sysmoUSIM-SJS1 and sysmoISIM-SJA2 cards reside here: https://www.freecalypso.org/hg/fc-ota-tools/ These tools go as far as generating the hex string of bytes to feed to the SIM in an ENVELOPE command; to actually interact with the SIM in a CCID "reader" and send this ENVELOPE command, fc-simtool needs to be used: https://www.freecalypso.org/hg/fc-pcsc-tools/ In hacking fellowship, Mother Mychaela From nhofmeyr at sysmocom.de Mon Feb 22 10:58:29 2021 From: nhofmeyr at sysmocom.de (Neels Hofmeyr) Date: Mon, 22 Feb 2021 11:58:29 +0100 Subject: Introducing ntt. Modern tools for TTCN-3 In-Reply-To: References: Message-ID: <20210222105829.GA27354@my.box> On Thu, Feb 18, 2021 at 01:25:39PM +0000, Simon, Matthias (Nokia - DE/Ulm) wrote: > $ ntt tags ./osmo-ttcn3-hacks/bts >TAGS I use universal-ctags (a spin-off from exuberant-ctags) to produce TTCN3 tags. However, it misses many template/function definitions that have keywords like optional, present etc. in them, so half the time my tag jump doesn't work. > But the most interesting piece is probably the TTCN-3 language server: ntt implements the > language server protocol. This makes ntt a universal TTCN-3 language plugin for > virtually any editor or IDE [2]. I use a ttcn.vim syntax file I found somewhere, the header says: Maintainer: Stefan Karlsson It works well. Never heard of the "language server protocol". All i really need is syntax highlighting and tags. Otherwise I always use vim's ctrl-p and ctrl-n for auto completion (and tags to look up argument ordering). The vim config example on > [2] https://nokia.github.io/ntt/editors/ looks interesting, what benefits would I get from using ntt in vim? In your logo, is that a squirrel? ~N From matthias.simon at nokia.com Mon Feb 22 12:07:03 2021 From: matthias.simon at nokia.com (Simon, Matthias (Nokia - DE/Ulm)) Date: Mon, 22 Feb 2021 12:07:03 +0000 Subject: Introducing ntt. Modern tools for TTCN-3 In-Reply-To: <20210222105829.GA27354@my.box> References: , <20210222105829.GA27354@my.box> Message-ID: > looks interesting, what benefits would I get from using ntt in vim? Tags files are simple and simple is good. So, you could use just the `ntt tags` command for native TTCN-3 support. This should help you with the templates. Tags files have one small drawback, though: They lack scope information (local definitions) and TTCN-3 semantics (ambiguous symbols). But if this really justifies a fancy pants language server, depends on how easy the server is to set up and how mature ntt becomes. So currently you don't get that many benefits. But, I expect the benefits of the TTCN-3 language server will play out in the late game: * when more features will be integrated into ntt. Like automatic formatting, semantically correct renaming, semantic highlighting, code-smell detection, ... * when third party software (ctags, highlight.js, ttcn3.vim, ...) requires updates due to new TTCN-3 language features, like map-types, object oriented extension, ... > In your logo, is that a squirrel? This is when you ask software engineer to design a logo (https://ahseeit.com//king-include/uploads/2020/12/building-ask-network-engineer-design-frontend-fbyuvakrishnamemes-4688096727.jpg) It's based on the Go mascot (https://golang.org), because I like Go and ntt is written in it. Cheers, Matthias ________________________________________ From: Neels Hofmeyr Sent: Monday, February 22, 2021 11:58 AM To: Simon, Matthias (Nokia - DE/Ulm) Cc: openbsc at lists.osmocom.org Subject: Re: Introducing ntt. Modern tools for TTCN-3 On Thu, Feb 18, 2021 at 01:25:39PM +0000, Simon, Matthias (Nokia - DE/Ulm) wrote: > $ ntt tags ./osmo-ttcn3-hacks/bts >TAGS I use universal-ctags (a spin-off from exuberant-ctags) to produce TTCN3 tags. However, it misses many template/function definitions that have keywords like optional, present etc. in them, so half the time my tag jump doesn't work. > But the most interesting piece is probably the TTCN-3 language server: ntt implements the > language server protocol. This makes ntt a universal TTCN-3 language plugin for > virtually any editor or IDE [2]. I use a ttcn.vim syntax file I found somewhere, the header says: Maintainer: Stefan Karlsson It works well. Never heard of the "language server protocol". All i really need is syntax highlighting and tags. Otherwise I always use vim's ctrl-p and ctrl-n for auto completion (and tags to look up argument ordering). The vim config example on > [2] https://nokia.github.io/ntt/editors/ looks interesting, what benefits would I get from using ntt in vim? In your logo, is that a squirrel? ~N From vyanitskiy at sysmocom.de Mon Feb 22 22:00:49 2021 From: vyanitskiy at sysmocom.de (Vadim Yanitskiy) Date: Mon, 22 Feb 2021 23:00:49 +0100 Subject: OTA RFM on sysmoISIM-SJA2 cards In-Reply-To: References: Message-ID: Hi Mychaela, On 2/22/21 3:48 AM, Mychaela Falconia wrote: > I am not able to produce a patch for shadysim.py because Python is a > read-only language for me: I can kinda-sorta read Python code and > figure out what it's doing, so I can reproduce its logic in C or > Bourne shell (my two languages), but writing or modifying Python code > is a territory I don't feel ready to venture into. thanks for your detailed report, I have submitted a bugfix: https://git.osmocom.org/sim/sim-tools/log/?h=fixeria/fixes Please let us know if it works for you, so we'll merge it to master. I hope you don't mind that I copy-pasted some sentences from your original message into the commit description. Best regards, Vadim. -- - Vadim Yanitskiy http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte From mychaela.falconia at gmail.com Mon Feb 22 23:55:02 2021 From: mychaela.falconia at gmail.com (Mychaela Falconia) Date: Mon, 22 Feb 2021 15:55:02 -0800 Subject: OTA RFM on sysmoISIM-SJA2 cards In-Reply-To: References: Message-ID: Hi Vadim, > thanks for your detailed report, I have submitted a bugfix: > > https://git.osmocom.org/sim/sim-tools/log/?h=fixeria/fixes LGTM. > Please let us know if it works for you, so we'll merge it to master. I have tested the PoC test_rfm() function in your new version, and it now emits packets that are 8 bytes shorter than before, matching what my own ota-smswrap-sjs1 C-language utility generates. The new way works with both sysmoUSIM-SJS1 and sysmoISIM-SJA2, using each card's respective different keys from webshop emails. KIC2 and KID2 are the keys needed for this operation. Please note, however, that the only feature of shadysim.py I ever tested (both before and after your fix) is the test_rfm() function which can only be exercised by editing the code to uncomment the two lines that invoke it and exit - there is no command line option for this function, which itself is of course nothing more than a PoC. I have never exercised this program's main intended function of manipulating Java card applets - at the present moment I have no need for those, and I don't know when (if ever) I will get to play with STK and Java applets for it - right now there are too many other areas of GSM that are more interesting to me. :) > I hope you don't mind that I copy-pasted some sentences from your > original message into the commit description. Sure thing - technical knowledge is meant to be shared and reproduced in different media and fora. In other SIM card news, it looks like Grcard folks (back from their LNY holidays) are reshipping their sample cards to me via a different route (going through HK this time), thus there is a chance that I might finally receive them this week or the next. I am very anxious to find out if they are the same card platform as the short-lived sysmoSIM-GR2, or if it's a newer evolutionary revision given how many years have passed - given that they once made the evolutionary change from GR1 to GR2, we have every reason to expect that other evolutionary changes have followed since then. One of the features I hope to see included on these new cards is RFM - I will be perfectly fine with a non-Java card with no ability to install applications, but I do desire RFM - being able to write the MSISDN record OTA like real operators do is just so cool! Would anyone happen to know if the short-lived historical sysmoSIM-GR2 cards supported RFM or not? I reason that GR1 cards probably lacked this feature, but I do wonder about GR2. In hacking fellowship, Mother Mychaela From laforge at osmocom.org Tue Feb 23 18:38:40 2021 From: laforge at osmocom.org (Harald Welte) Date: Tue, 23 Feb 2021 19:38:40 +0100 Subject: Announcement of "OsmoDevCall" In-Reply-To: References: Message-ID: Dear Osmocom community, It's my pleasure to announce the first OsmoDevCall at February 26, 2021 at 20:00 CET at https://meeting4.franken.de/b/har-xbc-bsx-wvs This first meeting will have the following schedule: 20:00 meet + greet 20:15 presentation about Cell Broadcast and OsmoCBC 20:45 USSE: unstructured supplementary social event [*] 22:00 close of call Attendance is free of charge and open to anyone with an interest in Osmocom. More information about OsmoDevCall, including the schedule for further upcoming events can be found at https://osmocom.org/projects/osmo-dev-con/wiki/OsmoDevCall Looking forward to meeting you on Friday. Best regards, Harald [*] this is how we started to call the "unstructured" part of osmocom developer conferences in the past, basically where anyone can talk about anything, no formal schedule or structure. -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From laforge at osmocom.org Wed Feb 24 09:15:29 2021 From: laforge at osmocom.org (Harald Welte) Date: Wed, 24 Feb 2021 10:15:29 +0100 Subject: Looking for historical sysmoSIM-GR2 card In-Reply-To: References: Message-ID: Hi Mychaela, I have _one_ such card remaining. It is part of a batch that was used at the 30C3 event GSM network. Given that it's the last one I have, I am unfortunately not going to give it away, sorry. If it is important to you, I can set up a SSH-accessible system and put the card into a CCID reader so you can remotely work on it. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From pespin at sysmocom.de Fri Feb 26 16:33:47 2021 From: pespin at sysmocom.de (Pau Espin Pedrol) Date: Fri, 26 Feb 2021 17:33:47 +0100 Subject: Osmocom CNI release 202102 Message-ID: <0c5afc56-951f-c8a5-39fe-5da1a5af46c8@sysmocom.de> Dear all, I am pleased to announce new tagged releases for Osmocom Cellular Network Infrastructure components. Find more information about the release here [1]. Osmocom "Latest" repositories in OBS [2] should already contain packages for the new versions. OpenEmbedded related meta-layers such as meta-telephony usual stable/testing branch "201705" [3] have also been updated to build recipes for new versions. Regards, Pau [1] https://osmocom.org/news/132 [2] https://osmocom.org/projects/cellular-infrastructure/wiki/Latest_Builds [3] https://git.osmocom.org/meta-telephony/log/?h=201705 -- - Pau Espin Pedrol http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte From garrett.allen1990 at gmail.com Fri Feb 26 21:44:53 2021 From: garrett.allen1990 at gmail.com (Garrett) Date: Fri, 26 Feb 2021 21:44:53 +0000 Subject: Announcement of "OsmoDevCall" In-Reply-To: References: , Message-ID: <75E55A63-450B-4CEA-96CC-BC54E3B19593@hxcore.ol> An HTML attachment was scrubbed... URL: