August 2025 - gerrit-log - lists.osmocom.org

[XS] Change in pysim[master]: x509 cert: fix weird cert check

by laforge

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/40524?usp=email ) ( 1 is the latest approved patch-set. No files were changed between the latest approved patch-set and the submitted one. )Change subject: x509 cert: fix weird cert check ...................................................................... x509 cert: fix weird cert check Change-Id: I18beab0e1b24579724704c4141a2c457b2d4cf99 --- M pySim/esim/x509_cert.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: laforge: Looks good to me, approved Jenkins Builder: Verified diff --git a/pySim/esim/x509_cert.py b/pySim/esim/x509_cert.py index c8a0386..8139a09 100644 --- a/pySim/esim/x509_cert.py +++ b/pySim/esim/x509_cert.py @@ -149,7 +149,7 @@ check_signed(c, self.root_cert) return parent_cert = self.intermediate_certs.get(aki, None) - if not aki: + if not parent_cert: raise VerifyError('Could not find intermediate certificate for AuthKeyId %s' % b2h(aki)) check_signed(c, parent_cert) # if we reach here, we passed (no exception raised) -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40524?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I18beab0e1b24579724704c4141a2c457b2d4cf99 Gerrit-Change-Number: 40524 Gerrit-PatchSet: 8 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 month

1
0
0 0

[S] Change in pysim[master]: smdpp: add proper brp cert support

by laforge

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/40525?usp=email ) Change subject: smdpp: add proper brp cert support ...................................................................... smdpp: add proper brp cert support Change-Id: I6906732f7d193a9c2234075f4a82df5e0ed46100 --- M .gitignore M osmo-smdpp.py 2 files changed, 11 insertions(+), 5 deletions(-) Approvals: laforge: Looks good to me, approved Jenkins Builder: Verified diff --git a/.gitignore b/.gitignore index d40e503..7a2e150 100644 --- a/.gitignore +++ b/.gitignore @@ -11,5 +11,6 @@ dist tags smdpp-data/certs/DPtls/CERT_S_SM_DP_TLS_NIST.pem +smdpp-data/certs/DPtls/CERT_S_SM_DP_TLS_BRP.pem smdpp-data/generated smdpp-data/certs/dhparam2048.pem diff --git a/osmo-smdpp.py b/osmo-smdpp.py index f5a759f..ea45917 100755 --- a/osmo-smdpp.py +++ b/osmo-smdpp.py @@ -342,7 +342,9 @@ else: self.dp_pb.cert_from_der_file(os.path.join(cert_dir, 'DPpb', 'CERT_S_SM_DPpb_ECDSA_NIST.der')) self.dp_pb.privkey_from_pem_file(os.path.join(cert_dir, 'DPpb', 'SK_S_SM_DPpb_ECDSA_NIST.pem')) - self.rss = rsp.RspSessionStore(os.path.join(DATA_DIR, "sm-dp-sessions")) + # Use different session database files for BRP and NIST to avoid file locking during concurrent runs + session_db_suffix = "BRP" if use_brainpool else "NIST" + self.rss = rsp.RspSessionStore(os.path.join(DATA_DIR, f"sm-dp-sessions-{session_db_suffix}")) @app.handle_errors(ApiError) def handle_apierror(self, request: IRequest, failure): @@ -773,18 +775,21 @@ parser.add_argument("-c", "--certdir", help=f"cert subdir relative to {DATA_DIR}", default="certs") parser.add_argument("-s", "--nossl", help="do NOT use ssl", action='store_true', default=False) parser.add_argument("-v", "--verbose", help="dump more raw info", action='store_true', default=False) + parser.add_argument("-b", "--brainpool", help="Use Brainpool curves instead of NIST", + action='store_true', default=False) args = parser.parse_args() logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING) common_cert_path = os.path.join(DATA_DIR, args.certdir) - hs = SmDppHttpServer(server_hostname=HOSTNAME, ci_certs_path=os.path.join(common_cert_path, 'CertificateIssuer'), common_cert_path=common_cert_path, use_brainpool=False) + hs = SmDppHttpServer(server_hostname=HOSTNAME, ci_certs_path=os.path.join(common_cert_path, 'CertificateIssuer'), common_cert_path=common_cert_path, use_brainpool=args.brainpool) if(args.nossl): hs.app.run(args.host, args.port) else: - cert_derpath = Path(common_cert_path) / 'DPtls' / 'CERT_S_SM_DP_TLS_NIST.der' - cert_pempath = Path(common_cert_path) / 'DPtls' / 'CERT_S_SM_DP_TLS_NIST.pem' - cert_skpath = Path(common_cert_path) / 'DPtls' / 'SK_S_SM_DP_TLS_NIST.pem' + curve_type = 'BRP' if args.brainpool else 'NIST' + cert_derpath = Path(common_cert_path) / 'DPtls' / f'CERT_S_SM_DP_TLS_{curve_type}.der' + cert_pempath = Path(common_cert_path) / 'DPtls' / f'CERT_S_SM_DP_TLS_{curve_type}.pem' + cert_skpath = Path(common_cert_path) / 'DPtls' / f'SK_S_SM_DP_TLS_{curve_type}.pem' dhparam_path = Path(common_cert_path) / "dhparam2048.pem" if not dhparam_path.exists(): print("Generating dh params, this takes a few seconds..") -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40525?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I6906732f7d193a9c2234075f4a82df5e0ed46100 Gerrit-Change-Number: 40525 Gerrit-PatchSet: 7 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 month

1
0
0 0

[M] Change in pysim[master]: smdpp: less verbose by default

by laforge

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/40468?usp=email ) Change subject: smdpp: less verbose by default ...................................................................... smdpp: less verbose by default Those data blobs are huge. Change-Id: I04a72b8f52417862d4dcba1f0743700dd942ef49 --- M osmo-smdpp.py M pySim/esim/bsp.py M pySim/esim/es8p.py 3 files changed, 86 insertions(+), 50 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved diff --git a/osmo-smdpp.py b/osmo-smdpp.py index c3fd6d5..3051416 100755 --- a/osmo-smdpp.py +++ b/osmo-smdpp.py @@ -47,6 +47,9 @@ from pySim.esim.x509_cert import oid, cert_policy_has_oid, cert_get_auth_key_id from pySim.esim.x509_cert import CertAndPrivkey, CertificateSet, cert_get_subject_key_id, VerifyError +import logging +logger = logging.getLogger(__name__) + # HACK: make this configurable DATA_DIR = './smdpp-data' HOSTNAME = 'testsmdpplus1.example.com' # must match certificates! @@ -82,18 +85,18 @@ for policy in cert_policies_ext.value: policy_oid = policy.policy_identifier.dotted_string - print(f"Found certificate policy: {policy_oid}") + logger.debug(f"Found certificate policy: {policy_oid}") if policy_oid == '2.23.146.1.2.1.2': - print("Detected EUM certificate variant: O (old)") + logger.debug("Detected EUM certificate variant: O (old)") return 'O' elif policy_oid == '2.23.146.1.2.1.0.0.0': - print("Detected EUM certificate variant: Ov3/A/B/C (new)") + logger.debug("Detected EUM certificate variant: Ov3/A/B/C (new)") return 'NEW' except x509.ExtensionNotFound: - print("No Certificate Policies extension found") + logger.debug("No Certificate Policies extension found") except Exception as e: - print(f"Error checking certificate policies: {e}") + logger.debug(f"Error checking certificate policies: {e}") def parse_permitted_eins_from_cert(eum_cert) -> List[str]: """Extract permitted IINs from EUM certificate using the appropriate method @@ -106,17 +109,15 @@ if cert_variant == 'O': # Old variant - use nameConstraints extension - print("Using nameConstraints parsing for variant O certificate") permitted_iins.extend(_parse_name_constraints_eins(eum_cert)) else: # New variants (Ov3, A, B, C) - use GSMA permittedEins extension - print("Using GSMA permittedEins parsing for newer certificate variant") permitted_iins.extend(_parse_gsma_permitted_eins(eum_cert)) unique_iins = list(set(permitted_iins)) - print(f"Total unique permitted IINs found: {len(unique_iins)}") + logger.debug(f"Total unique permitted IINs found: {len(unique_iins)}") return unique_iins def _parse_gsma_permitted_eins(eum_cert) -> List[str]: @@ -130,7 +131,7 @@ for ext in eum_cert.extensions: if ext.oid == permitted_eins_oid: - print(f"Found GSMA permittedEins extension: {ext.oid}") + logger.debug(f"Found GSMA permittedEins extension: {ext.oid}") # Get the DER-encoded extension value ext_der = ext.value.value if hasattr(ext.value, 'value') else ext.value @@ -156,14 +157,14 @@ all(c in '0123456789ABCDEF' for c in iin_clean) and len(iin_clean) % 2 == 0): permitted_iins.append(iin_clean) - print(f"Found permitted IIN (GSMA): {iin_clean}") + logger.debug(f"Found permitted IIN (GSMA): {iin_clean}") else: - print(f"Invalid IIN format: {iin_string} (cleaned: {iin_clean})") + logger.debug(f"Invalid IIN format: {iin_string} (cleaned: {iin_clean})") except Exception as e: - print(f"Error parsing GSMA permittedEins extension: {e}") + logger.debug(f"Error parsing GSMA permittedEins extension: {e}") except Exception as e: - print(f"Error accessing GSMA certificate extensions: {e}") + logger.debug(f"Error accessing GSMA certificate extensions: {e}") return permitted_iins @@ -178,13 +179,11 @@ x509.oid.ExtensionOID.NAME_CONSTRAINTS ) - print("Found nameConstraints extension (variant O)") name_constraints = name_constraints_ext.value # Check permittedSubtrees for IIN constraints if name_constraints.permitted_subtrees: for subtree in name_constraints.permitted_subtrees: - print(f"Processing permitted subtree: {subtree}") if isinstance(subtree, x509.DirectoryName): for attribute in subtree.value: @@ -196,12 +195,12 @@ all(c in '0123456789ABCDEF' for c in serial_value) and len(serial_value) % 2 == 0): permitted_iins.append(serial_value) - print(f"Found permitted IIN (nameConstraints/DN): {serial_value}") + logger.debug(f"Found permitted IIN (nameConstraints/DN): {serial_value}") except x509.ExtensionNotFound: - print("No nameConstraints extension found") + logger.debug("No nameConstraints extension found") except Exception as e: - print(f"Error parsing nameConstraints: {e}") + logger.debug(f"Error parsing nameConstraints: {e}") return permitted_iins @@ -209,29 +208,29 @@ def validate_eid_range(eid: str, eum_cert) -> bool: """Validate that EID is within the permitted EINs of the EUM certificate.""" if not eid or len(eid) != 32: - print(f"Invalid EID format: {eid}") + logger.debug(f"Invalid EID format: {eid}") return False try: permitted_eins = parse_permitted_eins_from_cert(eum_cert) if not permitted_eins: - print("Warning: No permitted EINs found in EUM certificate") + logger.debug("Warning: No permitted EINs found in EUM certificate") return False eid_normalized = eid.upper() - print(f"Validating EID {eid_normalized} against {len(permitted_eins)} permitted EINs") + logger.debug(f"Validating EID {eid_normalized} against {len(permitted_eins)} permitted EINs") for permitted_ein in permitted_eins: if eid_normalized.startswith(permitted_ein): - print(f"EID {eid_normalized} matches permitted EIN {permitted_ein}") + logger.debug(f"EID {eid_normalized} matches permitted EIN {permitted_ein}") return True - print(f"EID {eid_normalized} is not in any permitted EIN list") + logger.debug(f"EID {eid_normalized} is not in any permitted EIN list") return False except Exception as e: - print(f"Error validating EID: {e}") + logger.debug(f"Error validating EID: {e}") return False def build_status_code(subject_code: str, reason_code: str, subject_id: Optional[str], message: Optional[str]) -> Dict: @@ -299,11 +298,11 @@ def ci_get_cert_for_pkid(self, ci_pkid: bytes) -> Optional[x509.Certificate]: """Find CI certificate for given key identifier.""" for cert in self.ci_certs: - print("cert: %s" % cert) + logger.debug("cert: %s" % cert) subject_exts = list(filter(lambda x: isinstance(x.value, x509.SubjectKeyIdentifier), cert.extensions)) - print(subject_exts) + logger.debug(subject_exts) subject_pkid = subject_exts[0].value - print(subject_pkid) + logger.debug(subject_pkid) if subject_pkid and subject_pkid.key_identifier == ci_pkid: return cert return None @@ -356,7 +355,7 @@ validate_request_headers(request) content = json.loads(request.content.read()) - print("Rx JSON: %s" % json.dumps(content)) + logger.debug("Rx JSON: %s" % json.dumps(content)) set_headers(request) output = func(self, request, content) @@ -364,7 +363,7 @@ return '' build_resp_header(output) - print("Tx JSON: %s" % json.dumps(output)) + logger.debug("Tx JSON: %s" % json.dumps(output)) return json.dumps(output) return _api_wrapper @@ -383,7 +382,7 @@ euiccInfo1_bin = b64decode(content['euiccInfo1']) euiccInfo1 = rsp.asn1.decode('EUICCInfo1', euiccInfo1_bin) - print("Rx euiccInfo1: %s" % euiccInfo1) + logger.debug("Rx euiccInfo1: %s" % euiccInfo1) #euiccInfo1['svn'] # TODO: If euiccCiPKIdListForSigningV3 is present ... @@ -427,9 +426,9 @@ 'serverAddress': self.server_hostname, 'serverChallenge': serverChallenge, } - print("Tx serverSigned1: %s" % serverSigned1) + logger.debug("Tx serverSigned1: %s" % serverSigned1) serverSigned1_bin = rsp.asn1.encode('ServerSigned1', serverSigned1) - print("Tx serverSigned1: %s" % rsp.asn1.decode('ServerSigned1', serverSigned1_bin)) + logger.debug("Tx serverSigned1: %s" % rsp.asn1.decode('ServerSigned1', serverSigned1_bin)) output = {} output['serverSigned1'] = b64encode2str(serverSigned1_bin) @@ -458,7 +457,7 @@ authenticateServerResp_bin = b64decode(content['authenticateServerResponse']) authenticateServerResp = rsp.asn1.decode('AuthenticateServerResponse', authenticateServerResp_bin) - print("Rx %s: %s" % authenticateServerResp) + logger.debug("Rx %s: %s" % authenticateServerResp) if authenticateServerResp[0] == 'authenticateResponseError': r_err = authenticateServerResp[1] #r_err['transactionId'] @@ -510,7 +509,7 @@ raise ApiError('8.1', '6.1', 'Verification failed (euiccSignature1 over euiccSigned1)') ss.eid = ss.euicc_cert.subject.get_attributes_for_oid(x509.oid.NameOID.SERIAL_NUMBER)[0].value - print("EID (from eUICC cert): %s" % ss.eid) + logger.debug("EID (from eUICC cert): %s" % ss.eid) # Verify EID is within permitted range of EUM certificate if not validate_eid_range(ss.eid, eum_cert): @@ -589,7 +588,7 @@ prepDownloadResp_bin = b64decode(content['prepareDownloadResponse']) prepDownloadResp = rsp.asn1.decode('PrepareDownloadResponse', prepDownloadResp_bin) - print("Rx %s: %s" % prepDownloadResp) + logger.debug("Rx %s: %s" % prepDownloadResp) if prepDownloadResp[0] == 'downloadResponseError': r_err = prepDownloadResp[1] @@ -611,23 +610,23 @@ # store otPK.EUICC.ECKA in session state ss.euicc_otpk = euiccSigned2['euiccOtpk'] - print("euiccOtpk: %s" % (b2h(ss.euicc_otpk))) + logger.debug("euiccOtpk: %s" % (b2h(ss.euicc_otpk))) # Generate a one-time ECKA key pair (ot{PK,SK}.DP.ECKA) using the curve indicated by the Key Parameter # Reference value of CERT.DPpb.ECDDSA - print("curve = %s" % self.dp_pb.get_curve()) + logger.debug("curve = %s" % self.dp_pb.get_curve()) ss.smdp_ot = ec.generate_private_key(self.dp_pb.get_curve()) # extract the public key in (hopefully) the right format for the ES8+ interface ss.smdp_otpk = ss.smdp_ot.public_key().public_bytes(Encoding.X962, PublicFormat.UncompressedPoint) - print("smdpOtpk: %s" % b2h(ss.smdp_otpk)) - print("smdpOtsk: %s" % b2h(ss.smdp_ot.private_bytes(Encoding.DER, PrivateFormat.PKCS8, NoEncryption()))) + logger.debug("smdpOtpk: %s" % b2h(ss.smdp_otpk)) + logger.debug("smdpOtsk: %s" % b2h(ss.smdp_ot.private_bytes(Encoding.DER, PrivateFormat.PKCS8, NoEncryption()))) ss.host_id = b'mahlzeit' # Generate Session Keys using the CRT, otPK.eUICC.ECKA and otSK.DP.ECKA according to annex G euicc_public_key = ec.EllipticCurvePublicKey.from_encoded_point(ss.smdp_ot.curve, ss.euicc_otpk) ss.shared_secret = ss.smdp_ot.exchange(ec.ECDH(), euicc_public_key) - print("shared_secret: %s" % b2h(ss.shared_secret)) + logger.debug("shared_secret: %s" % b2h(ss.shared_secret)) # TODO: Check if this order requires a Confirmation Code verification @@ -661,22 +660,22 @@ request.setResponseCode(204) pendingNotification_bin = b64decode(content['pendingNotification']) pendingNotification = rsp.asn1.decode('PendingNotification', pendingNotification_bin) - print("Rx %s: %s" % pendingNotification) + logger.debug("Rx %s: %s" % pendingNotification) if pendingNotification[0] == 'profileInstallationResult': profileInstallRes = pendingNotification[1] pird = profileInstallRes['profileInstallationResultData'] transactionId = b2h(pird['transactionId']) ss = self.rss.get(transactionId, None) if ss is None: - print("Unable to find session for transactionId") - return + logger.warning(f"Unable to find session for transactionId: {transactionId}") + return None # Will return HTTP 204 with empty body profileInstallRes['euiccSignPIR'] # TODO: use original data, don't re-encode? pird_bin = rsp.asn1.encode('ProfileInstallationResultData', pird) # verify eUICC signature if not self._ecdsa_verify(ss.euicc_cert, profileInstallRes['euiccSignPIR'], pird_bin): raise Exception('ECDSA signature verification failed on notification') - print("Profile Installation Final Result: ", pird['finalResult']) + logger.debug("Profile Installation Final Result: %s", pird['finalResult']) # remove session state del self.rss[transactionId] elif pendingNotification[0] == 'otherSignedNotification': @@ -701,7 +700,7 @@ iccid = other_notif.get('iccid', None) if iccid: iccid = swap_nibbles(b2h(iccid)) - print("handleNotification: EID %s: %s of %s" % (eid, pmo, iccid)) + logger.debug("handleNotification: EID %s: %s of %s" % (eid, pmo, iccid)) else: raise ValueError(pendingNotification) @@ -714,7 +713,7 @@ @rsp_api_wrapper def cancelSession(self, request: IRequest, content: dict) -> dict: """See ES9+ CancelSession in SGP.22 Section 5.6.5""" - print("Rx JSON: %s" % content) + logger.debug("Rx JSON: %s" % content) transactionId = content['transactionId'] # Verify that the received transactionId is known and relates to an ongoing RSP session @@ -724,7 +723,7 @@ cancelSessionResponse_bin = b64decode(content['cancelSessionResponse']) cancelSessionResponse = rsp.asn1.decode('CancelSessionResponse', cancelSessionResponse_bin) - print("Rx %s: %s" % cancelSessionResponse) + logger.debug("Rx %s: %s" % cancelSessionResponse) if cancelSessionResponse[0] == 'cancelSessionResponseError': # FIXME: print some error @@ -760,8 +759,11 @@ parser.add_argument("-p", "--port", help="TCP port to bind HTTP to", default=8000) parser.add_argument("-c", "--certdir", help=f"cert subdir relative to {DATA_DIR}", default="certs") parser.add_argument("-s", "--nossl", help="do NOT use ssl", action='store_true', default=False) + parser.add_argument("-v", "--verbose", help="dump more raw info", action='store_true', default=False) args = parser.parse_args() + logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING) + common_cert_path = os.path.join(DATA_DIR, args.certdir) hs = SmDppHttpServer(server_hostname=HOSTNAME, ci_certs_path=os.path.join(common_cert_path, 'CertificateIssuer'), common_cert_path=common_cert_path, use_brainpool=False) if(args.nossl): diff --git a/pySim/esim/bsp.py b/pySim/esim/bsp.py index fb4c0b3..70e4a1f 100644 --- a/pySim/esim/bsp.py +++ b/pySim/esim/bsp.py @@ -73,7 +73,7 @@ block_nr = self.block_nr ciphertext = self._encrypt(padded_data) logger.debug("encrypt(block_nr=%u, s_enc=%s, plaintext=%s, padded=%s) -> %s", - block_nr, b2h(self.s_enc), b2h(data), b2h(padded_data), b2h(ciphertext)) + block_nr, b2h(self.s_enc)[:20], b2h(data)[:20], b2h(padded_data)[:20], b2h(ciphertext)[:20]) return ciphertext def decrypt(self, data:bytes) -> bytes: @@ -149,10 +149,20 @@ temp_data = self.mac_chain + tag_and_length + data old_mcv = self.mac_chain c_mac = self._auth(temp_data) + + # DEBUG: Show MAC computation details + logger.debug(f"MAC_DEBUG: tag=0x{tag:02x}, lcc={lcc}") + logger.debug(f"MAC_DEBUG: tag_and_length: {tag_and_length.hex()}") + logger.debug(f"MAC_DEBUG: mac_chain[:20]: {old_mcv[:20].hex()}") + logger.debug(f"MAC_DEBUG: temp_data[:20]: {temp_data[:20].hex()}") + logger.debug(f"MAC_DEBUG: c_mac: {c_mac.hex()}") + # The output data is computed by concatenating the following data: the tag, the final length, the result of step 2 and the C-MAC value. ret = tag_and_length + data + c_mac + logger.debug(f"MAC_DEBUG: final_output[:20]: {ret[:20].hex()}") + logger.debug("auth(tag=0x%x, mcv=%s, s_mac=%s, plaintext=%s, temp=%s) -> %s", - tag, b2h(old_mcv), b2h(self.s_mac), b2h(data), b2h(temp_data), b2h(ret)) + tag, b2h(old_mcv)[:20], b2h(self.s_mac)[:20], b2h(data)[:20], b2h(temp_data)[:20], b2h(ret)[:20]) return ret def verify(self, ciphertext: bytes) -> bool: @@ -204,6 +214,11 @@ s_enc = out[l:2*l] s_mac = out[l*2:3*l] + logger.debug(f"BSP_KDF_DEBUG: kdf_out = {b2h(out)}") + logger.debug(f"BSP_KDF_DEBUG: initial_mcv = {b2h(initial_mac_chaining_value)}") + logger.debug(f"BSP_KDF_DEBUG: s_enc = {b2h(s_enc)}") + logger.debug(f"BSP_KDF_DEBUG: s_mac = {b2h(s_mac)}") + return s_enc, s_mac, initial_mac_chaining_value @@ -231,9 +246,21 @@ """Encrypt + MAC a single plaintext TLV. Returns the protected ciphertext.""" assert tag <= 255 assert len(plaintext) <= self.max_payload_size - logger.debug("encrypt_and_mac_one(tag=0x%x, plaintext=%s)", tag, b2h(plaintext)) + + # DEBUG: Show what we're processing + logger.debug(f"BSP_DEBUG: encrypt_and_mac_one(tag=0x{tag:02x}, plaintext_len={len(plaintext)})") + logger.debug(f"BSP_DEBUG: plaintext[:20]: {plaintext[:20].hex()}") + logger.debug(f"BSP_DEBUG: s_enc[:20]: {self.c_algo.s_enc[:20].hex()}") + logger.debug(f"BSP_DEBUG: s_mac[:20]: {self.m_algo.s_mac[:20].hex()}") + + logger.debug("encrypt_and_mac_one(tag=0x%x, plaintext=%s)", tag, b2h(plaintext)[:20]) ciphered = self.c_algo.encrypt(plaintext) + logger.debug(f"BSP_DEBUG: ciphered[:20]: {ciphered[:20].hex()}") + maced = self.m_algo.auth(tag, ciphered) + logger.debug(f"BSP_DEBUG: final_result[:20]: {maced[:20].hex()}") + logger.debug(f"BSP_DEBUG: final_result_len: {len(maced)}") + return maced def encrypt_and_mac(self, tag: int, plaintext:bytes) -> List[bytes]: diff --git a/pySim/esim/es8p.py b/pySim/esim/es8p.py index 8bd7e14..9c76084 100644 --- a/pySim/esim/es8p.py +++ b/pySim/esim/es8p.py @@ -25,6 +25,9 @@ from pySim.esim.bsp import BspInstance from pySim.esim import PMO +import logging +logger = logging.getLogger(__name__) + # Given that GSMA RSP uses ASN.1 in a very weird way, we actually cannot encode the full data type before # signing, but we have to build parts of it separately first, then sign that, so we can put the signature # into the same sequence as the signed data. We use the existing pySim TLV code for this. @@ -196,8 +199,12 @@ # 'initialiseSecureChannelRequest' bpp_seq = rsp.asn1.encode('InitialiseSecureChannelRequest', iscr) # firstSequenceOf87 + logger.debug("BPP_ENCODE_DEBUG: Encrypting ConfigureISDP with BSP keys") + logger.debug(f"BPP_ENCODE_DEBUG: BSP S-ENC: {bsp.c_algo.s_enc.hex()}") + logger.debug(f"BPP_ENCODE_DEBUG: BSP S-MAC: {bsp.m_algo.s_mac.hex()}") bpp_seq += encode_seq(0xa0, bsp.encrypt_and_mac(0x87, conf_idsp_bin)) # sequenceOF88 + logger.debug("BPP_ENCODE_DEBUG: MAC-only StoreMetadata with BSP keys") bpp_seq += encode_seq(0xa1, bsp.mac_only(0x88, smr_bin)) if self.ppp: # we have to use session keys -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40468?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I04a72b8f52417862d4dcba1f0743700dd942ef49 Gerrit-Change-Number: 40468 Gerrit-PatchSet: 12 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-CC: lynxis lazus <lynxis(a)fe80.eu>

1 month

1
0
0 0

[M] Change in pysim[master]: smdpp: fix asn1tool OBJECT IDENTIFIER decoding

by laforge

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/40526?usp=email ) Change subject: smdpp: fix asn1tool OBJECT IDENTIFIER decoding ...................................................................... smdpp: fix asn1tool OBJECT IDENTIFIER decoding While at it make the linter happy. The feature to ignore blocks is making slow progress: https://github.com/astral-sh/ruff/issues/3711# Change-Id: Ic678e6c4a4c1a01de87a8dce26f4a5e452e8562a --- M osmo-smdpp.py 1 file changed, 113 insertions(+), 29 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, approved diff --git a/osmo-smdpp.py b/osmo-smdpp.py index ea45917..27923d4 100755 --- a/osmo-smdpp.py +++ b/osmo-smdpp.py @@ -17,37 +17,123 @@ # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. -from cryptography.hazmat.primitives.asymmetric.utils import decode_dss_signature, encode_dss_signature -from cryptography import x509 -from cryptography.exceptions import InvalidSignature -from cryptography.hazmat.primitives import hashes -from cryptography.hazmat.primitives.asymmetric import ec, dh -from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat, PrivateFormat, NoEncryption, ParameterFormat -from pathlib import Path -import json -import sys -import argparse -import uuid -import os -import functools -from typing import Optional, Dict, List -from pprint import pprint as pp - -import base64 -from base64 import b64decode -from klein import Klein -from twisted.web.iweb import IRequest +# asn1tools issue https://github.com/eerimoq/asn1tools/issues/194 +# must be first here import asn1tools +import asn1tools.codecs.ber +import asn1tools.codecs.der +# do not move the code +def fix_asn1_oid_decoding(): + fix_asn1_schema = """ + TestModule DEFINITIONS ::= BEGIN + TestOid ::= SEQUENCE { + oid OBJECT IDENTIFIER + } + END + """ -from osmocom.utils import h2b, b2h, swap_nibbles + fix_asn1_asn1 = asn1tools.compile_string(fix_asn1_schema, codec='der') + fix_asn1_oid_string = '2.999.10' + fix_asn1_encoded = fix_asn1_asn1.encode('TestOid', {'oid': fix_asn1_oid_string}) + fix_asn1_decoded = fix_asn1_asn1.decode('TestOid', fix_asn1_encoded) -import pySim.esim.rsp as rsp -from pySim.esim import saip, PMO -from pySim.esim.es8p import * -from pySim.esim.x509_cert import oid, cert_policy_has_oid, cert_get_auth_key_id -from pySim.esim.x509_cert import CertAndPrivkey, CertificateSet, cert_get_subject_key_id, VerifyError + if (fix_asn1_decoded['oid'] != fix_asn1_oid_string): + # ASN.1 OBJECT IDENTIFIER Decoding Issue: + # + # In ASN.1 BER/DER encoding, the first two arcs of an OBJECT IDENTIFIER are + # combined into a single value: (40 * arc0) + arc1. This is encoded as a base-128 + # variable-length quantity (and commonly known as VLQ or base-128 encoding) + # as specified in ITU-T X.690 §8.19, it can span multiple bytes if + # the value is large. + # + # For arc0 = 0 or 1, arc1 must be in [0, 39]. For arc0 = 2, arc1 can be any non-negative integer. + # All subsequent arcs (arc2, arc3, ...) are each encoded as a separate base-128 VLQ. + # + # The decoding bug occurs when the decoder does not properly split the first + # subidentifier for arc0 = 2 and arc1 >= 40. Instead of decoding: + # - arc0 = 2 + # - arc1 = (first_subidentifier - 80) + # it may incorrectly interpret the first_subidentifier as arc0 = (first_subidentifier // 40), + # arc1 = (first_subidentifier % 40), which is only valid for arc1 < 40. + # + # This patch handles it properly for all valid OBJECT IDENTIFIERs + # with large second arcs, by applying the ASN.1 rules: + # - if first_subidentifier < 40: arc0 = 0, arc1 = first_subidentifier + # - elif first_subidentifier < 80: arc0 = 1, arc1 = first_subidentifier - 40 + # - else: arc0 = 2, arc1 = first_subidentifier - 80 + # + # This problem is not uncommon, see for example https://github.com/randombit/botan/issues/4023 -import logging + def fixed_decode_object_identifier(data, offset, end_offset): + """Decode ASN.1 OBJECT IDENTIFIER from bytes to dotted string, fixing large second arc handling.""" + def read_subidentifier(data, offset): + value = 0 + while True: + b = data[offset] + value = (value << 7) | (b & 0x7F) + offset += 1 + if not (b & 0x80): + break + return value, offset + + subid, offset = read_subidentifier(data, offset) + if subid < 40: + first = 0 + second = subid + elif subid < 80: + first = 1 + second = subid - 40 + else: + first = 2 + second = subid - 80 + arcs = [first, second] + + while offset < end_offset: + subid, offset = read_subidentifier(data, offset) + arcs.append(subid) + + return '.'.join(str(x) for x in arcs) + + asn1tools.codecs.ber.decode_object_identifier = fixed_decode_object_identifier + asn1tools.codecs.der.decode_object_identifier = fixed_decode_object_identifier + + # test our patch + asn1 = asn1tools.compile_string(fix_asn1_schema, codec='der') + decoded = asn1.decode('TestOid', fix_asn1_encoded)['oid'] + assert fix_asn1_oid_string == str(decoded) + +fix_asn1_oid_decoding() + +from cryptography.hazmat.primitives.asymmetric.utils import encode_dss_signature # noqa: E402 +from cryptography import x509 # noqa: E402 +from cryptography.exceptions import InvalidSignature # noqa: E402 +from cryptography.hazmat.primitives import hashes # noqa: E402 +from cryptography.hazmat.primitives.asymmetric import ec, dh # noqa: E402 +from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat, PrivateFormat, NoEncryption, ParameterFormat # noqa: E402 +from pathlib import Path # noqa: E402 +import json # noqa: E402 +import sys # noqa: E402 +import argparse # noqa: E402 +import uuid # noqa: E402 +import os # noqa: E402 +import functools # noqa: E402 +from typing import Optional, Dict, List # noqa: E402 +from pprint import pprint as pp # noqa: E402 + +import base64 # noqa: E402 +from base64 import b64decode # noqa: E402 +from klein import Klein # noqa: E402 +from twisted.web.iweb import IRequest # noqa: E402 + +from osmocom.utils import h2b, b2h, swap_nibbles # noqa: E402 + +import pySim.esim.rsp as rsp # noqa: E402 +from pySim.esim import saip, PMO # noqa: E402 +from pySim.esim.es8p import ProfileMetadata,UnprotectedProfilePackage,ProtectedProfilePackage,BoundProfilePackage,BspInstance # noqa: E402 +from pySim.esim.x509_cert import oid, cert_policy_has_oid, cert_get_auth_key_id # noqa: E402 +from pySim.esim.x509_cert import CertAndPrivkey, CertificateSet, cert_get_subject_key_id, VerifyError # noqa: E402 + +import logging # noqa: E402 logger = logging.getLogger(__name__) # HACK: make this configurable @@ -138,8 +224,6 @@ if isinstance(ext_der, bytes): try: - import asn1tools - permitted_eins_schema = """ PermittedEins DEFINITIONS ::= BEGIN PermittedEins ::= SEQUENCE OF PrintableString -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40526?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Ic678e6c4a4c1a01de87a8dce26f4a5e452e8562a Gerrit-Change-Number: 40526 Gerrit-PatchSet: 7 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 month

1
0
0 0

[M] Change in pysim[master]: memory backed ephermal session store for easy concurrent runs

by laforge

laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/40527?usp=email ) Change subject: memory backed ephermal session store for easy concurrent runs ...................................................................... memory backed ephermal session store for easy concurrent runs Change-Id: I05bfd6ff471ccf1c8c2b5f2b748b9d4125ddd4f7 --- M osmo-smdpp.py M pySim/esim/rsp.py 2 files changed, 63 insertions(+), 7 deletions(-) Approvals: laforge: Looks good to me, approved Jenkins Builder: Verified diff --git a/osmo-smdpp.py b/osmo-smdpp.py index 27923d4..94e64e8 100755 --- a/osmo-smdpp.py +++ b/osmo-smdpp.py @@ -405,7 +405,7 @@ continue return False - def __init__(self, server_hostname: str, ci_certs_path: str, common_cert_path: str, use_brainpool: bool = False): + def __init__(self, server_hostname: str, ci_certs_path: str, common_cert_path: str, use_brainpool: bool = False, in_memory: bool = False): self.server_hostname = server_hostname self.upp_dir = os.path.realpath(os.path.join(DATA_DIR, 'upp')) self.ci_certs = self.load_certs_from_path(ci_certs_path) @@ -426,9 +426,15 @@ else: self.dp_pb.cert_from_der_file(os.path.join(cert_dir, 'DPpb', 'CERT_S_SM_DPpb_ECDSA_NIST.der')) self.dp_pb.privkey_from_pem_file(os.path.join(cert_dir, 'DPpb', 'SK_S_SM_DPpb_ECDSA_NIST.pem')) - # Use different session database files for BRP and NIST to avoid file locking during concurrent runs - session_db_suffix = "BRP" if use_brainpool else "NIST" - self.rss = rsp.RspSessionStore(os.path.join(DATA_DIR, f"sm-dp-sessions-{session_db_suffix}")) + if in_memory: + self.rss = rsp.RspSessionStore(in_memory=True) + logger.info("Using in-memory session storage") + else: + # Use different session database files for BRP and NIST to avoid file locking during concurrent runs + session_db_suffix = "BRP" if use_brainpool else "NIST" + db_path = os.path.join(DATA_DIR, f"sm-dp-sessions-{session_db_suffix}") + self.rss = rsp.RspSessionStore(filename=db_path, in_memory=False) + logger.info(f"Using file-based session storage: {db_path}") @app.handle_errors(ApiError) def handle_apierror(self, request: IRequest, failure): @@ -861,6 +867,8 @@ parser.add_argument("-v", "--verbose", help="dump more raw info", action='store_true', default=False) parser.add_argument("-b", "--brainpool", help="Use Brainpool curves instead of NIST", action='store_true', default=False) + parser.add_argument("-m", "--in-memory", help="Use ephermal in-memory session storage (for concurrent runs)", + action='store_true', default=False) args = parser.parse_args() logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING) diff --git a/pySim/esim/rsp.py b/pySim/esim/rsp.py index 3dccf2d..6ab3fa3 100644 --- a/pySim/esim/rsp.py +++ b/pySim/esim/rsp.py @@ -94,9 +94,57 @@ self.__dict__.update(state) -class RspSessionStore(shelve.DbfilenameShelf): - """A derived class as wrapper around the database-backed non-volatile storage 'shelve', in case we might - need to extend it in the future. We use it to store RspSessionState objects indexed by transactionId.""" +class RspSessionStore: + """A wrapper around the database-backed storage 'shelve' for storing RspSessionState objects. + Can be configured to use either file-based storage or in-memory storage. + We use it to store RspSessionState objects indexed by transactionId.""" + + def __init__(self, filename: Optional[str] = None, in_memory: bool = False): + self._in_memory = in_memory + + if in_memory: + self._shelf = shelve.Shelf(dict()) + else: + if filename is None: + raise ValueError("filename is required for file-based session store") + self._shelf = shelve.open(filename) + + # dunder magic + def __getitem__(self, key): + return self._shelf[key] + + def __setitem__(self, key, value): + self._shelf[key] = value + + def __delitem__(self, key): + del self._shelf[key] + + def __contains__(self, key): + return key in self._shelf + + def __iter__(self): + return iter(self._shelf) + + def __len__(self): + return len(self._shelf) + + # everything else + def __getattr__(self, name): + """Delegate attribute access to the underlying shelf object.""" + return getattr(self._shelf, name) + + def close(self): + """Close the session store.""" + if hasattr(self._shelf, 'close'): + self._shelf.close() + if self._in_memory: + # For in-memory store, clear the reference + self._shelf = None + + def sync(self): + """Synchronize the cache with the underlying storage.""" + if hasattr(self._shelf, 'sync'): + self._shelf.sync() def extract_euiccSigned1(authenticateServerResponse: bytes) -> bytes: """Extract the raw, DER-encoded binary euiccSigned1 field from the given AuthenticateServerResponse. This -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40527?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I05bfd6ff471ccf1c8c2b5f2b748b9d4125ddd4f7 Gerrit-Change-Number: 40527 Gerrit-PatchSet: 7 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org>

1 month

1
0
0 0

[L] Change in pysim[master]: smdpp: cc support, proper checks, proper check order, test mode

by laforge

Attention is currently required from: Hoernchen. laforge has posted comments on this change by Hoernchen. ( https://gerrit.osmocom.org/c/pysim/+/40880?usp=email ) Change subject: smdpp: cc support, proper checks, proper check order, test mode ...................................................................... Patch Set 1: Code-Review-1 (1 comment) Patchset: PS1: there are too many different logical changes for one patch. test mode should be a separate patch. Cc support another one, ... -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40880?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I7b22fe4eecb41d0f98e28a1d9390cd7fc8c5784c Gerrit-Change-Number: 40880 Gerrit-PatchSet: 1 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: Hoernchen <ewild(a)sysmocom.de> Gerrit-Comment-Date: Thu, 21 Aug 2025 14:53:59 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: Yes

1 month

1
0
0 0

[M] Change in pysim[master]: memory backed ephermal session store for easy concurrent runs

by laforge

Attention is currently required from: Hoernchen. laforge has posted comments on this change by Hoernchen. ( https://gerrit.osmocom.org/c/pysim/+/40527?usp=email ) Change subject: memory backed ephermal session store for easy concurrent runs ...................................................................... Patch Set 7: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40527?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I05bfd6ff471ccf1c8c2b5f2b748b9d4125ddd4f7 Gerrit-Change-Number: 40527 Gerrit-PatchSet: 7 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: Hoernchen <ewild(a)sysmocom.de> Gerrit-Comment-Date: Thu, 21 Aug 2025 14:51:40 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes

1 month

1
0
0 0

[M] Change in pysim[master]: smdpp: fix asn1tool OBJECT IDENTIFIER decoding

by laforge

Attention is currently required from: Hoernchen. laforge has posted comments on this change by Hoernchen. ( https://gerrit.osmocom.org/c/pysim/+/40526?usp=email ) Change subject: smdpp: fix asn1tool OBJECT IDENTIFIER decoding ...................................................................... Patch Set 7: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40526?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Ic678e6c4a4c1a01de87a8dce26f4a5e452e8562a Gerrit-Change-Number: 40526 Gerrit-PatchSet: 7 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: Hoernchen <ewild(a)sysmocom.de> Gerrit-Comment-Date: Thu, 21 Aug 2025 14:51:22 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes

1 month

1
0
0 0

[S] Change in pysim[master]: smdpp: add proper brp cert support

by laforge

Attention is currently required from: Hoernchen. laforge has posted comments on this change by Hoernchen. ( https://gerrit.osmocom.org/c/pysim/+/40525?usp=email ) Change subject: smdpp: add proper brp cert support ...................................................................... Patch Set 7: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40525?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I6906732f7d193a9c2234075f4a82df5e0ed46100 Gerrit-Change-Number: 40525 Gerrit-PatchSet: 7 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: Hoernchen <ewild(a)sysmocom.de> Gerrit-Comment-Date: Thu, 21 Aug 2025 14:51:00 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes

1 month

1
0
0 0

[M] Change in pysim[master]: smdpp: less verbose by default

by laforge

Attention is currently required from: Hoernchen, dexter. laforge has posted comments on this change by Hoernchen. ( https://gerrit.osmocom.org/c/pysim/+/40468?usp=email ) Change subject: smdpp: less verbose by default ...................................................................... Patch Set 12: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/40468?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I04a72b8f52417862d4dcba1f0743700dd942ef49 Gerrit-Change-Number: 40468 Gerrit-PatchSet: 12 Gerrit-Owner: Hoernchen <ewild(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-CC: lynxis lazus <lynxis(a)fe80.eu> Gerrit-Attention: Hoernchen <ewild(a)sysmocom.de> Gerrit-Attention: dexter <pmaier(a)sysmocom.de> Gerrit-Comment-Date: Thu, 21 Aug 2025 14:50:52 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes

1 month

1
0
0 0

2025

2024

2023

2022

gerrit-log August 2025