November 2025 - gerrit-log - lists.osmocom.org

[XS] Change in libosmo-sigtran[master]: ss7_as_vty: Forbid configuring RKM-dynamically allocated AS
by pespin 25 Nov '25

25 Nov '25

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/libosmo-sigtran/+/41511?usp=email ) Change subject: ss7_as_vty: Forbid configuring RKM-dynamically allocated AS ...................................................................... ss7_as_vty: Forbid configuring RKM-dynamically allocated AS Change-Id: Ia0c3ce39337de3a10ff0215ccec72c65fab13029 --- M src/ss7_as_vty.c 1 file changed, 5 insertions(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/libosmo-sigtran refs/changes/11/41511/1 diff --git a/src/ss7_as_vty.c b/src/ss7_as_vty.c index 87ea26b..d88c9b4 100644 --- a/src/ss7_as_vty.c +++ b/src/ss7_as_vty.c @@ -71,10 +71,14 @@ as = osmo_ss7_as_find_or_create(inst, name, protocol); if (!as) { - vty_out(vty, "cannot create AS '%s'%s", name, VTY_NEWLINE); + vty_out(vty, "Cannot create AS '%s'%s", name, VTY_NEWLINE); return CMD_WARNING; } + if (as->rkm_dyn_allocated) { + vty_out(vty, "Cannot configure RKM-dynamically allocated AS '%s'%s", name, VTY_NEWLINE); + return CMD_WARNING; + } vty->node = L_CS7_AS_NODE; vty->index = as; -- To view, visit https://gerrit.osmocom.org/c/libosmo-sigtran/+/41511?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: libosmo-sigtran Gerrit-Branch: master Gerrit-Change-Id: Ia0c3ce39337de3a10ff0215ccec72c65fab13029 Gerrit-Change-Number: 41511 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de>

1 0

[XL] Change in libosmo-sigtran[master]: Add TCAP based loadsharing/routing
by daniel 25 Nov '25

25 Nov '25

Attention is currently required from: laforge, lynxis lazus, pespin. daniel has posted comments on this change by lynxis lazus. ( https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309?usp=email ) Change subject: Add TCAP based loadsharing/routing ...................................................................... Patch Set 11: (13 comments) File src/ipa.c: https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/7e879692_e8bb0… : PS8, Line 51: #include "ss7_as_loadshare_tcap.h" > #ifdef WITH_TCAP_LOADSHARING Done https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/e017bfa1_63569… : PS8, Line 315: /* update TCAP Transaction Tracking (Rx) */ > no, not needed. Done https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/b8024c49_a656a… : PS8, Line 368: if (hh_ext->proto == IPAC_PROTO_EXT_TCAP_ROUTING) { > Should be ok with the ifdef encompassing the whole case. Done File src/ss7_as.h: https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/f931cda8_bfcb6… : PS8, Line 6: #include <osmocom/core/hashtable.h> > Does the include hurt? I would try to prevent cluttering #ifdefs everywhere. Here I would agree. File src/ss7_as.c: https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/75f526fe_33bc5… : PS8, Line 246: tcap_disable(as); > #ifdef WITH_TCAP_LOADSHARING Done File src/ss7_as_loadshare_tcap.h: https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/9e30b12d_26a6b… : PS8, Line 14: #ifdef WITH_TCAP_LOADSHARING > No, I want to keep it here because of the #else branch. Done File src/ss7_as_loadshare_tcap.c: https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/5e99493a_92a6d… : PS8, Line 379: /* FIXME: use UTDS */ > This needs to be fixed? It's used in asp_loadshare_tcap_sccp. @lynxis@fe80.eu this comment can be removed? https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/0b931d51_b4796… : PS8, Line 752: LOGPASP(asp, DLSS7, LOGL_ERROR, "TCAP routing message is too small\n"); > I think we really want to add a new log category "TCAP" here. ACK File src/ss7_as_vty.c: https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/8f450e8c_ad0e5… : PS8, Line 39: #include "ss7_as_loadshare_tcap.h" > No Done https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/c6f40028_8ae8b… : PS8, Line 158: tcap_disable(as); > No, not needed. Done https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/dd2d7ab1_6a562… : PS8, Line 165: "traffic-mode (loadshare|loadshare-tcap) [bindings] [sls] [opc-sls] [opc-shift] [<0-2>]", > I'm not 100% familiar with the tcap_enable/disable hooks, but wouldn't you need to call tcap_disable […] Done https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/fef869f3_7afc2… : PS8, Line 186: vty_out(vty, "tcap loadsharing is not supported in this build.\n"); > This is wrong, will fix it in v2 Done File src/ss7_asp.c: https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309/comment/ea3225b4_fabd5… : PS8, Line 53: #include "ss7_as_loadshare_tcap.h" > #ifdef WITH_TCAP_LOADSHARING Done -- To view, visit https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: libosmo-sigtran Gerrit-Branch: master Gerrit-Change-Id: Ibcb48aa0e515ad346f59ddd84b24c6e2c026144d Gerrit-Change-Number: 41309 Gerrit-PatchSet: 11 Gerrit-Owner: lynxis lazus <lynxis(a)fe80.eu> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-CC: daniel <dwillmann(a)sysmocom.de> Gerrit-CC: laforge <laforge(a)osmocom.org> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Attention: lynxis lazus <lynxis(a)fe80.eu> Gerrit-Comment-Date: Tue, 25 Nov 2025 17:51:49 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: pespin <pespin(a)sysmocom.de> Comment-In-Reply-To: daniel <dwillmann(a)sysmocom.de> Comment-In-Reply-To: lynxis lazus <lynxis(a)fe80.eu>

1 0

[XL] Change in libosmo-sigtran[master]: Add TCAP based loadsharing/routing
by daniel 25 Nov '25

25 Nov '25

Attention is currently required from: laforge, lynxis lazus, pespin. daniel has uploaded a new patch set (#11) to the change originally created by lynxis lazus. ( https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309?usp=email ) The following approvals got outdated and were removed: Verified-1 by Jenkins Builder Change subject: Add TCAP based loadsharing/routing ...................................................................... Add TCAP based loadsharing/routing TCAP based loadsharing will share the load based on the TCAP oTID and dTID. Because TCAP are session based, a TCAP session based tracking is implemented, to allow following traffic to forwarded to the same ASP. This TCAP session tracking is similar to IP connection tracking. ASPs within an AS can use the new IPA TCAP ROUTING protocol to register for specific TCAP ranges. TCAP sessions initiated by a peer (traffic towards such loadsharing AS/ASP), will use the oTID of the `TCAP Begin` to loadshare and select a ASP. Further if the TCAP session was initiated by a `loadshared` ASP, the oTID will be added to the session tracking. Co-authored-by: Harald Welte <laforge(a)osmocom.org> Co-authored-by: Daniel Willmann <dwillmann(a)sysmocom.de> Related: SYS#5423 Change-Id: Ibcb48aa0e515ad346f59ddd84b24c6e2c026144d --- M configure.ac M src/Makefile.am M src/ipa.c M src/ss7_as.c M src/ss7_as.h A src/ss7_as_loadshare_tcap.c A src/ss7_as_loadshare_tcap.h M src/ss7_as_vty.c M src/ss7_asp.c M src/ss7_asp.h A src/tcap_transaction_tracking.c A src/tcap_transaction_tracking.h M tests/Makefile.am A tests/tcap/Makefile.am A tests/tcap/tcap_transaction_tracking_test.c A tests/tcap/tcap_transaction_tracking_test.ok M tests/testsuite.at M tests/vty/Makefile.am M tests/vty/osmo_stp_test.vty A tests/vty/osmo_stp_test_tcap.vty 20 files changed, 2,190 insertions(+), 6 deletions(-) git pull ssh://gerrit.osmocom.org:29418/libosmo-sigtran refs/changes/09/41309/11 -- To view, visit https://gerrit.osmocom.org/c/libosmo-sigtran/+/41309?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newpatchset Gerrit-Project: libosmo-sigtran Gerrit-Branch: master Gerrit-Change-Id: Ibcb48aa0e515ad346f59ddd84b24c6e2c026144d Gerrit-Change-Number: 41309 Gerrit-PatchSet: 11 Gerrit-Owner: lynxis lazus <lynxis(a)fe80.eu> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: pespin <pespin(a)sysmocom.de> Gerrit-CC: daniel <dwillmann(a)sysmocom.de> Gerrit-CC: laforge <laforge(a)osmocom.org> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Attention: lynxis lazus <lynxis(a)fe80.eu>

1 0

[M] Change in libosmo-sigtran[master]: Add VTY command to list TCAP ranges for an AS
by daniel 25 Nov '25

25 Nov '25

Attention is currently required from: daniel, lynxis lazus. daniel has uploaded a new patch set (#11) to the change originally created by lynxis lazus. ( https://gerrit.osmocom.org/c/libosmo-sigtran/+/41311?usp=email ) The following approvals got outdated and were removed: Verified-1 by Jenkins Builder Change subject: Add VTY command to list TCAP ranges for an AS ...................................................................... Add VTY command to list TCAP ranges for an AS Show the list of all tcap ranges and ASP for an AS: OsmoSTP# show cs7 instance 0 as name as-ipa-loadshare-0 tcap-ranges Tid Min Tid Max SSN PC ASP Name ------- ------- --- ------------- ------------ 300 599 0 (no PC) asp-ipa-loadshare-0-1 0 299 0 (no PC) asp-ipa-loadshare-0-0 Show the tcap range and ASP for specific TCAP Id: OsmoSTP# show cs7 instance 0 as name as-ipa-loadshare-0 tcap-ranges tid 15 Tid Min Tid Max SSN PC ASP Name ------- ------- --- ------------- ------------ 0 299 0 (no PC) asp-ipa-loadshare-0-0 Related: SYS#5423 Change-Id: I8026248d9897aadab5bf13a425269ec1e948dfdf --- M src/Makefile.am A src/ss7_as_loadshare_tcap_vty.c A src/ss7_as_loadshare_tcap_vty.h M src/ss7_vty.c M tests/vty/osmo_stp_test.vty 5 files changed, 151 insertions(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/libosmo-sigtran refs/changes/11/41311/11 -- To view, visit https://gerrit.osmocom.org/c/libosmo-sigtran/+/41311?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newpatchset Gerrit-Project: libosmo-sigtran Gerrit-Branch: master Gerrit-Change-Id: I8026248d9897aadab5bf13a425269ec1e948dfdf Gerrit-Change-Number: 41311 Gerrit-PatchSet: 11 Gerrit-Owner: lynxis lazus <lynxis(a)fe80.eu> Gerrit-Reviewer: Jenkins Builder Gerrit-CC: daniel <dwillmann(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Attention: daniel <dwillmann(a)sysmocom.de> Gerrit-Attention: lynxis lazus <lynxis(a)fe80.eu>

1 0

[S] Change in libosmo-sigtran[master]: Avoid removing ASP from all AS during osmo_ss7_as_del_asp()
by daniel 25 Nov '25

25 Nov '25

daniel has uploaded this change for review. ( https://gerrit.osmocom.org/c/libosmo-sigtran/+/41509?usp=email ) Change subject: Avoid removing ASP from all AS during osmo_ss7_as_del_asp() ...................................................................... Avoid removing ASP from all AS during osmo_ss7_as_del_asp() The osmo_ss7_as_del_asp() -> tcap_asp_down() is weird. In practice it won't hurt because in IPA we only have 1 ASP per AS, per conceptually is wrong. In osmo_ss7_as_del_asp(), you remove ASP from AS. But there you call asp_down(), which actually removes the ASP from *all* its AS. So that asp_down() needs to be split and then inside osmo_ss7_as_del_asp() only remove the ASP from that specific AS. Change-Id: I37093f5a2fae17a6ef117ac880436bb813bcb2b1 --- M src/ss7_as.c M src/ss7_as_loadshare_tcap.c M src/ss7_as_loadshare_tcap.h 3 files changed, 10 insertions(+), 5 deletions(-) git pull ssh://gerrit.osmocom.org:29418/libosmo-sigtran refs/changes/09/41509/1 diff --git a/src/ss7_as.c b/src/ss7_as.c index 5bca3f9..f15d6a5 100644 --- a/src/ss7_as.c +++ b/src/ss7_as.c @@ -228,7 +228,7 @@ } #ifdef WITH_TCAP_LOADSHARING - tcap_asp_down(asp); + tcap_as_del_asp(as, asp); #endif /* WITH_TCAP_LOADSHARING */ for (i = 0; i < ARRAY_SIZE(as->cfg.asps); i++) { diff --git a/src/ss7_as_loadshare_tcap.c b/src/ss7_as_loadshare_tcap.c index ffff526..bdbfcea 100644 --- a/src/ss7_as_loadshare_tcap.c +++ b/src/ss7_as_loadshare_tcap.c @@ -932,6 +932,13 @@ } } +void tcap_as_del_asp(struct osmo_ss7_as *as, struct osmo_ss7_asp *asp) +{ + _tcap_range_asp_down(as, asp); + if (as->tcap.contains_pc || as->tcap.contains_ssn) + tcap_range_as_update_pc_ssn(as); +} + /** Called when the ASP is going down or free'd * * @param[in] asp the asp which is going to be destroyed @@ -949,10 +956,7 @@ llist_for_each_entry(as, &inst->as_list, list) { if (!osmo_ss7_as_has_asp(as, asp)) continue; - - _tcap_range_asp_down(as, asp); - if (as->tcap.contains_pc || as->tcap.contains_ssn) - tcap_range_as_update_pc_ssn(as); + tcap_as_del_asp(as, asp); } } diff --git a/src/ss7_as_loadshare_tcap.h b/src/ss7_as_loadshare_tcap.h index abba5b7..a939c82 100644 --- a/src/ss7_as_loadshare_tcap.h +++ b/src/ss7_as_loadshare_tcap.h @@ -36,6 +36,7 @@ int ss7_as_select_asp_loadshare_tcap(struct osmo_ss7_asp **asp, struct osmo_ss7_as *as, const struct xua_msg *xua); /* When the ASP got removed */ +void tcap_as_del_asp(struct osmo_ss7_as *as, struct osmo_ss7_asp *asp); void tcap_asp_down(struct osmo_ss7_asp *asp); void tcap_enable(struct osmo_ss7_as *as); -- To view, visit https://gerrit.osmocom.org/c/libosmo-sigtran/+/41509?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: libosmo-sigtran Gerrit-Branch: master Gerrit-Change-Id: I37093f5a2fae17a6ef117ac880436bb813bcb2b1 Gerrit-Change-Number: 41509 Gerrit-PatchSet: 1 Gerrit-Owner: daniel <dwillmann(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de>

1 0

[L] Change in pysim[master]: card_key_provider: add PostgreSQL support
by Jenkins Builder 25 Nov '25

25 Nov '25

Jenkins Builder has posted comments on this change by dexter. ( https://gerrit.osmocom.org/c/pysim/+/41508?usp=email ) Change subject: card_key_provider: add PostgreSQL support ...................................................................... Patch Set 1: (3 comments) File contrib/csv-to-pgsql.py: Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/pysim/+/41508/comment/dbdc53e1_0538a54e?usp=em… : PS1, Line 129: Get a list of all columns availabe in the current table scheme. 'availabe' may be misspelled - perhaps 'available'? File docs/card-key-provider.rst: Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/pysim/+/41508/comment/c65fb3b8_cfe1c7d5?usp=em… : PS1, Line 144: assume that you didn't get the Global Plattform keys from your card vendor for 'Plattform' may be misspelled - perhaps 'Platform'? Robot Comment from checkpatch (run ID ): https://gerrit.osmocom.org/c/pysim/+/41508/comment/af57878f_3376d5e9?usp=em… : PS1, Line 176: trailing whitespace -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41508?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Icba625c02a60d7e1f519b506a46bda5ded0537d3 Gerrit-Change-Number: 41508 Gerrit-PatchSet: 1 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-CC: Jenkins Builder Gerrit-Comment-Date: Tue, 25 Nov 2025 16:30:13 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No

1 0

[XS] Change in pysim[master]: card_key_provider: add missing type annotation
by dexter 25 Nov '25

25 Nov '25

dexter has uploaded this change for review. ( https://gerrit.osmocom.org/c/pysim/+/41506?usp=email ) Change subject: card_key_provider: add missing type annotation ...................................................................... card_key_provider: add missing type annotation Related: SYS#7725 Change-Id: I45751d2b31976c04c03006d8baa195eef2576b4f --- M pySim/card_key_provider.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/06/41506/1 diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index 494eed2..f614edc 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -200,7 +200,7 @@ provider_list.append(provider) -def card_key_provider_get(fields, key: str, value: str, provider_list=card_key_providers) -> Dict[str, str]: +def card_key_provider_get(fields: list[str], key: str, value: str, provider_list=card_key_providers) -> Dict[str, str]: """Query all registered card data providers for card-individual [key] data. Args: -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41506?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I45751d2b31976c04c03006d8baa195eef2576b4f Gerrit-Change-Number: 41506 Gerrit-PatchSet: 1 Gerrit-Owner: dexter <pmaier(a)sysmocom.de>

1 0

[XS] Change in pysim[master]: docs/conf.py: update copyright year
by dexter 25 Nov '25

25 Nov '25

dexter has uploaded this change for review. ( https://gerrit.osmocom.org/c/pysim/+/41507?usp=email ) Change subject: docs/conf.py: update copyright year ...................................................................... docs/conf.py: update copyright year The copyright year of the docs is still at 2023, let's update it to the current year. Change-Id: Icf64670847d090a250f732d94d18e780e483239b --- M docs/conf.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/07/41507/1 diff --git a/docs/conf.py b/docs/conf.py index 9091878..ef40070 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -18,7 +18,7 @@ # -- Project information ----------------------------------------------------- project = 'osmopysim-usermanual' -copyright = '2009-2023 by Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle, Merlin Chlosta' +copyright = '2009-2025 by Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle, Merlin Chlosta' author = 'Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle, Merlin Chlosta' # PDF: Avoid that the authors list exceeds the page by inserting '\and' -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41507?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Icf64670847d090a250f732d94d18e780e483239b Gerrit-Change-Number: 41507 Gerrit-PatchSet: 1 Gerrit-Owner: dexter <pmaier(a)sysmocom.de>

1 0

[L] Change in pysim[master]: card_key_provider: add PostgreSQL support
by dexter 25 Nov '25

25 Nov '25

dexter has uploaded this change for review. ( https://gerrit.osmocom.org/c/pysim/+/41508?usp=email ) Change subject: card_key_provider: add PostgreSQL support ...................................................................... card_key_provider: add PostgreSQL support The Card Key Provider currently only has support for CSV files as input. Unfortunately using CSV files does not scale very well when the card inventory is very large and continously updated. In this case a centralized storage in the form of a database is the more suitable approach. This patch adds a PostgreSQL support next to the existing CSV file support. It also adds an importer tool to import existing CSV files into the database. Change-Id: Icba625c02a60d7e1f519b506a46bda5ded0537d3 Related: SYS#7725 --- A contrib/csv-to-pgsql.py M docs/card-key-provider.rst M pySim-shell.py M pySim/card_key_provider.py 4 files changed, 589 insertions(+), 10 deletions(-) git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/08/41508/1 diff --git a/contrib/csv-to-pgsql.py b/contrib/csv-to-pgsql.py new file mode 100755 index 0000000..b735082 --- /dev/null +++ b/contrib/csv-to-pgsql.py @@ -0,0 +1,304 @@ +#!/usr/bin/env python3 + +import argparse +import logging +import csv +import psycopg2 +import sys +import yaml +from pathlib import Path +from pySim.log import PySimLogger +from cmd2 import __version__, style +from packaging import version + +log = PySimLogger.get("CSV2PGQSL") + +# cmd2 >= 2.3.0 has deprecated the bg/fg in favor of Bg/Fg :( +if version.parse(__version__) < version.parse("2.3.0"): + from cmd2 import fg, bg # pylint: disable=no-name-in-module + RED = fg.red + YELLOW = fg.yellow + LIGHT_RED = fg.bright_red + LIGHT_GREEN = fg.bright_green +else: + from cmd2 import Fg, Bg # pylint: disable=no-name-in-module + RED = Fg.RED + YELLOW = Fg.YELLOW + LIGHT_RED = Fg.LIGHT_RED + LIGHT_GREEN = Fg.LIGHT_GREEN + +class CardKeyDatabase: + def __init__(self, config_filename: str, table_name: str, create_table: bool = False, admin: bool = False): + """ + Initialize database connection and set the table which shall be used as storage for the card key data. + In case the specified table does not exist yet it can be created using the create_table_type parameter. + + New tables are always minimal tables which follow a pre-defined table scheme. The user may extend the table + with additional columns using the add_cols() later. + + Args: + tablename : name of the database table to create. + create_table_type : type of the table to create ('UICC' or 'EUICC') + """ + + def user_from_config_file(config, role: str) -> tuple[str, str]: + db_users = config.get('db_users') + user = db_users.get(role) + if user is None: + raise ValueError("user for role '%s' not set up in config file." % role) + return user.get('name'), user.get('pass') + + log = PySimLogger.get("PQSQL") + self.table = table_name + self.cols = None + + # Depending on the table type, the table name must contain either the substring "uicc_keys" or "euicc_keys". + # This convention will allow us to deduct the table type from the table name. + if "euicc_keys" not in table_name and "uicc_keys" not in table_name: + raise ValueError("Table name (%s) should contain the substring \"uicc_keys\" or \"euicc_keys\"" % table_name) + + # Read config file + log.info("Using config file: %s" % config_filename) + with open(config_filename, "r") as cfg: + config = yaml.load(cfg, Loader=yaml.FullLoader) + host = config.get('host') + log.info("Database host: %s" % host) + db_name = config.get('db_name') + log.info("Database name: %s" % db_name) + table_names = config.get('table_names') + username_admin, password_admin = user_from_config_file(config, 'admin') + username_importer, password_importer = user_from_config_file(config, 'importer') + username_reader, _ = user_from_config_file(config, 'reader') + + # Switch between admin and importer user + if admin: + username, password = username_admin, password_admin + else: + username, password = username_importer, password_importer + + # Create database connection + log.info("Database user: %s" % username) + self.conn = psycopg2.connect(dbname=db_name, user=username, password=password, host=host) + self.cur = self.conn.cursor() + + # In the context of this tool it is not relevant if the table name is present in the config file. However, + # pySim-shell.py will require the table name to be configured properly to access the database table. + if self.table not in table_names: + log.warning("Specified table name (%s) is not yet present in config file (required for access from pySim-shell.py)", + self.table) + + # Create a new minimal database table of the specified table type. + if create_table: + if not admin: + raise ValueError("creation of new table refused, use option --admin and try again.") + if "euicc_keys" in self.table: + self.__create_table(username_reader, username_importer, ['EID']) + elif "uicc_keys" in self.table: + self.__create_table(username_reader, username_importer, ['ICCID', 'IMSI']) + + # Ensure a table with the specified name exists + log.info("Database table: %s" % self.table) + if self.get_cols() == []: + raise ValueError("Table name (%s) does not exist yet" % self.table) + log.info("Database table columns: %s" % str(self.get_cols())) + + def __create_table(self, user_reader:str, user_importer:str, cols:list[str]): + """ + Initialize a new table. New tables are always minimal tables with one primary key and additional index columns. + Non index-columns may be added later using method _update_cols(). + """ + + # Create table columns with primary key + secondary_col_string = "" + for c in cols[1:]: + secondary_col_string += ", %s VARCHAR" % c.lower() + self.cur.execute("CREATE TABLE %s (%s VARCHAR PRIMARY KEY%s);" % + (self.table, cols[0], secondary_col_string)) + + # Create indexes for all other columns + for c in cols[1:]: + self.cur.execute("CREATE INDEX %s ON %s(%s);" % (c.lower(), self.table, c.lower())); + + # Set permissions + self.cur.execute("GRANT INSERT ON %s TO %s;" % (self.table, user_importer)); + self.cur.execute("GRANT SELECT ON %s TO %s;" % (self.table, user_reader)); + log.info("New database table created: %s" % str(self.table)) + + def get_cols(self) -> list[str]: + """ + Get a list of all columns availabe in the current table scheme. + + Returns: + list with column names (in uppercase) of the database table + """ + + # Return cached col list if present + if self.cols: + return self.cols + + # Request a list of current cols from the database + self.cur.execute("SELECT column_name FROM information_schema.columns where table_name = '%s';" % self.table) + cols_result = self.cur.fetchall() + cols = [] + for c in cols_result: + cols.append(c[0].upper()) + self.cols = cols + return cols + + def get_missing_cols(self, cols_expected:list[str]) -> list[str]: + """ + Check if the current table scheme lacks any of the given expected columns. + + Returns: + list with the missing columns. + """ + + cols_present = self.get_cols() + return list(set(cols_expected) - set(cols_present)) + + def add_cols(self, cols:list[str]): + """ + Update the current table scheme with additional columns. In case the updated columns are already exist, the + table schema is not changed. + + Args: + table : name of the database table to alter + cols : list with updated colum names to add + """ + + cols_missing = self.get_missing_cols(cols) + + # Depending on the table type (see constructor), we either have a primary key 'ICCID' (for UICC data), or 'EID' + # (for eUICC data). Both table formats different types of data and have rather differen columns also. Let's + # prevent the excidentally mixing of both types. + if 'ICCID' in cols_missing: + raise ValueError("Table %s stores eUCCC key material, refusing to add UICC specific column 'ICCID'" % self.table) + if 'EID' in cols_missing: + raise ValueError("Table %s stores UCCC key material, refusing to add eUICC specific column 'EID'" % self.table) + + # Add the missing columns to the table + self.cols = None + for c in cols_missing: + self.cur.execute("ALTER TABLE %s ADD %s VARCHAR;" % (self.table, c.lower())); + + def insert_row(self, row:dict[str, str]): + """ + Insert a new row into the database table. + + Args: + row : dictionary with the colum names and their designated values + """ + + # Check if the row is compatible with the current table scheme + cols_expected = list(row.keys()) + cols_missing = self.get_missing_cols(cols_expected) + if cols_missing != []: + raise ValueError("table %s has incompatible format, the row %s contains unknown cols %s" % + (self.table, str(row), str(cols_missing))) + + # Insert row into datbase table + column_names = "" + for k in row.keys(): + column_names += k.lower() + ", " + column_names = column_names[:-2] + + values = "" + for v in row.values(): + values += "'" + v + "', " + values = values[:-2] + + self.cur.execute("INSERT INTO %s (%s) VALUES (%s)" % (self.table, column_names, values)); + + def commit(self): + self.conn.commit() + log.info("Changes to table %s committed!" % self.table) + + + + + + + + + +def open_csv(opts: argparse.Namespace): + log.info("CSV file: %s" % opts.csv) + csv_file = open(opts.csv, 'r') + if not csv_file: + raise RuntimeError("Could not open CSV file '%s'" % opts.csv) + + cr = csv.DictReader(csv_file) + if not cr: + raise RuntimeError( + "could not open DictReader for CSV-File '%s'" % opts.csv) + cr.fieldnames = [field.upper() for field in cr.fieldnames] + log.info("CSV file columns: %s" % str(cr.fieldnames)) + return cr + +def open_db(cr: csv.DictReader, opts: argparse.Namespace) -> CardKeyDatabase: + try: + db = CardKeyDatabase(opts.pqsql, opts.table_name, opts.create_table, opts.admin) + + # Check CSV format against table schema, add missing columns + cols_missing = db.get_missing_cols(cr.fieldnames) + if cols_missing != [] and (opts.update_columns or opts.create_table): + log.info("Adding missing columns: " + str(cols_missing)) + db.add_cols(cols_missing) + cols_missing = db.get_missing_cols(cr.fieldnames) + + # Make sure the table schema has no missing columns + if cols_missing != []: + log.error("Database table lacks CSV file columns: %s -- import aborted!" % cols_missing) + sys.exit(2) + except Exception as e: + log.error(str(e).strip()) + log.error("Database initialization aborted due to error!") + sys.exit(2) + + return db + +def import_from_csv(db: CardKeyDatabase, cr: csv.DictReader): + count = 0 + for row in cr: + try: + db.insert_row(row) + count+=1 + if count % 100 == 0: + log.info("CSV file import in progress, %d rows imported..." % count) + except Exception as e: + log.error(str(e).strip()) + log.error("CSV file import aborted due to error, no datasets committed!") + sys.exit(2) + log.info("CSV file import done, %d rows imported" % count) + +if __name__ == '__main__': + option_parser = argparse.ArgumentParser(description='CSV importer for pySim-shell\'s PostgreSQL Card Key Provider', + formatter_class=argparse.ArgumentDefaultsHelpFormatter) + option_parser.add_argument("--verbose", help="Enable verbose logging", action='store_true', default=False) + option_parser.add_argument('--pqsql', metavar='FILE', + default=str(Path.home()) + "/.osmocom/pysim/card_data_pqsql.cfg", + help='Read card data from PostgreSQL database (config file)') + option_parser.add_argument('--csv', metavar='FILE', help='input CSV file with card data', required=True) + option_parser.add_argument("--table-name", help="name of the card key table", type=str, required=True) + option_parser.add_argument("--update-columns", help="add missing table columns", action='store_true', default=False) + option_parser.add_argument("--create-table", action='store_true', help="create new card key table", default=False) + option_parser.add_argument("--admin", action='store_true', help="perform action as admin", default=False) + opts = option_parser.parse_args() + + PySimLogger.setup(print, {logging.WARN: YELLOW}) + if (opts.verbose): + PySimLogger.set_verbose(True) + PySimLogger.set_level(logging.DEBUG) + + # Open CSV file + cr = open_csv(opts) + + # Open database, create initial table, update column scheme + db = open_db(cr, opts) + + # Progress with import + if not opts.admin: + import_from_csv(db, cr) + + # Commit changes to the database + db.commit() diff --git a/docs/card-key-provider.rst b/docs/card-key-provider.rst index 390a7c7..bda33a5 100644 --- a/docs/card-key-provider.rst +++ b/docs/card-key-provider.rst @@ -1,4 +1,4 @@ -Retrieving card-individual keys via CardKeyProvider +Retrieving card-individual keys via CardKeyProvider =================================================== When working with a batch of cards, or more than one card in general, it @@ -20,9 +20,11 @@ database for the key material, or that uses a key derivation function to derive card-specific key material from a global master key. -The only actual CardKeyProvider implementation included in pySim is the -`CardKeyProviderCsv` which retrieves the key material from a -[potentially encrypted] CSV file. +pySim already includes two CardKeyProvider implementations. One to retrieve +key material from a CSV file (`CardKeyProviderCsv`) and a second one that allows +to retrieve the key material from a PostgreSQL database (`CardKeyProviderCsv`). +Both implementations equally implement a column encryption scheme that allows +to protect sensitive columns using a *transport key* The CardKeyProviderCsv @@ -40,11 +42,215 @@ open a CSV file from the default location at `~/.osmocom/pysim/card_data.csv`, and use that, if it exists. +The `CardKeyProviderCsv` is suitable to manage small amounts of key material +locally. However, if your card inventory is very large and the key material +must be made available on multiple sites, the `CardKeyProviderPgsql` is the +better option. + + +The CardKeyProviderPqsql +------------------------ + +With the `CardKeyProviderPsql` you can use a PostgreSQL database as storage +medium. The implementation comes with a CSV importer tool that consumes the +same CSV files you would normally use with the `CardKeyProviderCsv`, so you +can just use your existing CSV files and import them into the database. + + +Setting up the database +^^^^^^^^^^^^^^^^^^^^^^^ + +From the perspective of the database, the `CardKeyProviderPsql` has only +minimal requirements. You do not have to create any tables in advance. An empty +database and at least one user that may create, alter and insert into tables is +sufficient. However, for increased reliability and as a protection against +incorrect operation, the `CardKeyProviderPsql` supports a hierarchical model +with three users (or roles): + +* **admin**: + This should be the owner of the database. It is intended to be used for + administrative tasks like adding new tables or adding new columns to existing + tables. This user should not be used to insert new data into tables or to access + data from within pySim-shell using the `CardKeyProviderPsql` + +* **importer**: + This user is used when feeding new data into an existing table. It should only + be able to insert new rows into existing tables. It should not be used for + administrative tasks or to access data from within pySim-shell using the + `CardKeyProviderPsql` + +* **reader**: + To access data from within pySim shell using the `CardKeyProviderPsql` the + reader user is the correct one to use. This user should have no write access + to the database or any of the tables. + + +Creating a config file +^^^^^^^^^^^^^^^^^^^^^^ + +The default location for the config file is `~/.osmocom/pysim/card_data_pqsql.cfg` +The file uses `yaml` syntax and should look like the example below: + +:: + + host: "127.0.0.1" + db_name: "my_database" + table_names: + - "uicc_keys" + - "euicc_keys" + db_users: + admin: + name: "my_admin_user" + pass: "my_admin_password" + importer: + name: "my_importer_user" + pass: "my_importer_password" + reader: + name: "my_reader_user" + pass: "my_reader_password" + +This file is used by pySim-shell and by the importer tool. Both expect the file +in the aforementioned location. In case you want to store the file in a +different location you may use the `--pgsql` commandline option to provide a +custom config file path. + +The hostname and the database name for the PostgreSQL database is set with the +`host` and `db_name` fields. The field `db_users` sets the user names and +passwords for each of the aforementioned users (or roles). In case only a single +admin user is used, all three entries may be populated with the same user name +and password (not recommended) + +The field `table_names` sets the tables that the `CardKeyProviderPsql` shall +use to query to locate card key data. You can set up as many tables as you +want, `CardKeyProviderPsql` will query them in order, one by one until a +matching entry is found. + +NOTE: In case you do not want to disclose the admin and the importer credentials +to pySim-shell you may remove those lines. pySim-shell will only require the +`reader` entry under `db_users`. + + +Using the Importer +^^^^^^^^^^^^^^^^^^ + +Before data can be imported, you must first create a database table. Tables +are created with the provided importer tool, which can be found under +`contrib/csv-to-pgsql.py`. This tool is used to create the database table and +read the data from the provided CSV file into the database. + +As mentioned before, all CSV file formats that work with `CardKeyProviderCsv` +ma be used. To demonstrate how the import process works, let's assume you want +to import a CSV file format that looks like the following example. Let's also +assume that you didn't get the Global Plattform keys from your card vendor for +this batch of UICC cards, so your CSV file lacks the columns for those fields. + +:: + + "id","imsi","iccid","acc","pin1","puk1","pin2","puk2","ki","opc","adm1" + "card1","999700000000001","8900000000000000001","0001","1111","11111111","0101","01010101","11111111111111111111111111111111","11111111111111111111111111111111","11111111" + "card2","999700000000002","8900000000000000002","0002","2222","22222222","0202","02020202","22222222222222222222222222222222","22222222222222222222222222222222","22222222" + "card3","999700000000003","8900000000000000003","0003","3333","22222222","0303","03030303","33333333333333333333333333333333","33333333333333333333333333333333","33333333" + +Since this is your first import, the database still lacks the table. To +instruct the importer to create a new table, you may use the `--create-table` +option. You also have to pick an appropriate name for the table. Any name may +be chosen as long as it contains the string `uicc_keys` or `euicc_keys`, +depending on the type of data (`UICC` or `eUICC`) you intend to store in the +table. The creation of the table is an administrative task and can only be done +with the `admin` user. The `admin` user is selected using the `--admin` switch. + +:: + + $ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_01.csv --table-name uicc_keys --create-table --admin + INFO: CSV file: ./csv-to-pgsql_example_01.csv + INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1'] + INFO: Using config file: /home/user/.osmocom/pysim/card_data_pqsql.cfg + INFO: Database host: 127.0.0.1 + INFO: Database name: my_database + INFO: Database user: my_admin_user + INFO: New database table created: uicc_keys + INFO: Database table: uicc_keys + INFO: Database table columns: ['ICCID', 'IMSI'] + INFO: Adding missing columns: ['PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC'] + INFO: Changes to table uicc_keys committed! + +The importer has created a new table with the name `uicc_keys`. The table is +now ready to be filled with data. + +:: + + $ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_01.csv --table-name uicc_keys + INFO: CSV file: ./csv-to-pgsql_example_01.csv + INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1'] + INFO: Using config file: /home/user/.osmocom/pysim/card_data_pqsql.cfg + INFO: Database host: 127.0.0.1 + INFO: Database name: my_database + INFO: Database user: my_importer_user + INFO: Database table: uicc_keys + INFO: Database table columns: ['ICCID', 'IMSI', 'PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC'] + INFO: CSV file import done, 3 rows imported + INFO: Changes to table uicc_keys committed! + +A quick `SELECT * FROM uicc_keys;` at the PostgreSQL console should now display +the contents of the CSV file you have fed into the importer. + +Let's now assume that with your next batch of UICC cards your vendor includes +the global platform keys so your CSV format changes. It may now look like this: + +:: + + "id","imsi","iccid","acc","pin1","puk1","pin2","puk2","ki","opc","adm1","scp02_dek_1","scp02_enc_1","scp02_mac_1" + "card4","999700000000004","8900000000000000004","0004","4444","44444444","0404","04040404","44444444444444444444444444444444","44444444444444444444444444444444","44444444","44444444444444444444444444444444","44444444444444444444444444444444","44444444444444444444444444444444" + "card5","999700000000005","8900000000000000005","0005","4444","55555555","0505","05050505","55555555555555555555555555555555","55555555555555555555555555555555","55555555","55555555555555555555555555555555","55555555555555555555555555555555","55555555555555555555555555555555" + "card6","999700000000006","8900000000000000006","0006","4444","66666666","0606","06060606","66666666666666666666666666666666","66666666666666666666666666666666","66666666","66666666666666666666666666666666","66666666666666666666666666666666","66666666666666666666666666666666" + +When importing data from an updated CSV format the database table also has +to be updated. This is done using the `--update-columns` switch. Like when +creating new tables, this operation also requires admin privileges, so the +`--admin` switch is required again. + +:: + + $ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_02.csv --table-name uicc_keys --update-columns --admin + INFO: CSV file: ./csv-to-pgsql_example_02.csv + INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1', 'SCP02_DEK_1', 'SCP02_ENC_1', 'SCP02_MAC_1'] + INFO: Using config file: /home/user/.osmocom/pysim/card_data_pqsql.cfg + INFO: Database host: 127.0.0.1 + INFO: Database name: my_database + INFO: Database user: my_admin_user + INFO: Database table: uicc_keys + INFO: Database table columns: ['ICCID', 'IMSI', 'PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC'] + INFO: Adding missing columns: ['SCP02_ENC_1', 'SCP02_MAC_1', 'SCP02_DEK_1'] + INFO: Changes to table uicc_keys committed! + +When the new table columns are added, the import may be continued like the +first one: + +:: + + $ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_02.csv --table-name uicc_keys + INFO: CSV file: ./csv-to-pgsql_example_02.csv + INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1', 'SCP02_DEK_1', 'SCP02_ENC_1', 'SCP02_MAC_1'] + INFO: Using config file: /home/user/.osmocom/pysim/card_data_pqsql.cfg + INFO: Database host: 127.0.0.1 + INFO: Database name: my_database + INFO: Database user: my_importer_user + INFO: Database table: uicc_keys + INFO: Database table columns: ['ICCID', 'IMSI', 'PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC', 'SCP02_ENC_1', 'SCP02_MAC_1', 'SCP02_DEK_1'] + INFO: CSV file import done, 3 rows imported + INFO: Changes to table uicc_keys committed! + +On the PostgreSQL console a `SELECT * FROM uicc_keys;` should now show the +imported data with the added columns. All important data should now also be +available from within pySim-shell via the `CardKeyProviderCsv`. + + Column-Level CSV encryption -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +--------------------------- pySim supports column-level CSV encryption. This feature will make sure -that your key material is not stored in plaintext in the CSV file. +that your key material is not stored in plaintext in the CSV file (or +database). The encryption mechanism uses AES in CBC mode. You can use any key length permitted by AES (128/192/256 bit). @@ -72,6 +278,8 @@ * `SCP03_ISDA` is a group alias for `SCP03_ENC_ISDA`, `SCP03_MAC_ISDA`, `SCP03_DEK_ISDA` * `SCP03_ISDR` is a group alias for `SCP03_ENC_ISDR`, `SCP03_MAC_ISDR`, `SCP03_DEK_ISDR` +NOTE: When using `CardKeyProviderPqsl`, the input CSV files must be encrypted +before import. Field naming ------------ @@ -82,9 +290,9 @@ * For look-up of eUICC specific key material (like SCP03 keys for the ISD-R, ECASD), pySim uses the `EID` field as lookup key. -As soon as the CardKeyProviderCsv finds a line (row) in your CSV where -the ICCID or EID match, it looks for the column containing the requested -data. +As soon as the CardKeyProviderCsv finds a line (row) in your CSV file +(or database) where the ICCID or EID match, it looks for the column containing +the requested data. ADM PIN diff --git a/pySim-shell.py b/pySim-shell.py index a8b15d5..74a518d 100755 --- a/pySim-shell.py +++ b/pySim-shell.py @@ -69,7 +69,7 @@ from pySim.gsm_r import DF_EIRENE from pySim.cat import ProactiveCommand -from pySim.card_key_provider import CardKeyProviderCsv +from pySim.card_key_provider import CardKeyProviderCsv, CardKeyProviderPgsql from pySim.card_key_provider import card_key_provider_register, card_key_provider_get_field, card_key_provider_get from pySim.app import init_card @@ -1140,6 +1140,9 @@ card_key_group.add_argument('--csv', metavar='FILE', default=str(Path.home()) + "/.osmocom/pysim/card_data.csv", help='Read card data from CSV file') +card_key_group.add_argument('--pqsql', metavar='FILE', + default=str(Path.home()) + "/.osmocom/pysim/card_data_pqsql.cfg", + help='Read card data from PostgreSQL database (config file)') card_key_group.add_argument('--csv-column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append', help=argparse.SUPPRESS, dest='column_key') card_key_group.add_argument('--column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append', @@ -1179,6 +1182,8 @@ column_keys[name] = key if os.path.isfile(opts.csv): card_key_provider_register(CardKeyProviderCsv(opts.csv, column_keys)) + if os.path.isfile(opts.pqsql): + card_key_provider_register(CardKeyProviderPgsql(opts.pqsql, column_keys)) # Init card reader driver sl = init_reader(opts, proactive_handler = Proact()) diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index f614edc..e297f46 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -36,6 +36,8 @@ import abc import csv import logging +import yaml +import psycopg2 log = PySimLogger.get("CARDKEY") @@ -159,6 +161,7 @@ csv_filename : file name (path) of CSV file containing card-individual key/data transport_keys : (see class CardKeyFieldCryptor) """ + log.info("Using CSV file as card key data source: %s" % csv_filename) self.csv_file = open(csv_filename, 'r') if not self.csv_file: raise RuntimeError("Could not open CSV file '%s'" % csv_filename) @@ -186,6 +189,65 @@ return None return return_dict +class CardKeyProviderPgsql(CardKeyProvider): + """Card key provider implementation that allows to query against a specified PostgreSQL database table.""" + + def __init__(self, config_filename: str, transport_keys: dict): + """ + Args: + config_filename : file name (path) of CSV file containing card-individual key/data + transport_keys : (see class CardKeyFieldCryptor) + """ + log.info("Using SQL database as card key data source: %s" % config_filename) + with open(config_filename, "r") as cfg: + config = yaml.load(cfg, Loader=yaml.FullLoader) + log.info("Card key database name: %s" % config.get('db_name')) + db_users = config.get('db_users') + user = db_users.get('reader') + if user is None: + raise ValueError("user for role 'reader' not set up in config file.") + self.conn = psycopg2.connect(dbname=config.get('db_name'), + user=user.get('name'), + password=user.get('pass'), + host=config.get('host')) + self.tables = config.get('table_names') + log.info("Card key database tables: %s" % str(self.tables)) + self.crypt = CardKeyFieldCryptor(transport_keys) + + def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]: + column_names = "" + for f in fields: + column_names += f.lower() + ", " + column_names = column_names[:-2] + + db_result = None + for t in self.tables: + self.conn.rollback() + cur = self.conn.cursor() + + # Make sure that the database table and the key column actually exists. If not, move on to the next table + cur.execute("SELECT column_name FROM information_schema.columns where table_name = '%s';" % t) + cols_result = cur.fetchall() + if cols_result == []: + log.warning("Card Key database seems to lack table %s, check config file!" % t) + continue + if (key.lower(),) not in cols_result: + continue + + # Query requested columns from database table + cur.execute("SELECT %s FROM %s WHERE %s = '%s' LIMIT 1;" % (column_names, t, key.lower(), value)); + db_result = cur.fetchone() + cur.close() + if db_result: + break; + + if db_result is None: + return None + result = dict(zip(fields, db_result)) + + for k in result.keys(): + result[k] = self.crypt.decrypt_field(k, result.get(k)) + return result def card_key_provider_register(provider: CardKeyProvider, provider_list=card_key_providers): -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41508?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Icba625c02a60d7e1f519b506a46bda5ded0537d3 Gerrit-Change-Number: 41508 Gerrit-PatchSet: 1 Gerrit-Owner: dexter <pmaier(a)sysmocom.de>

1 0

[M] Change in pysim[master]: card_key_provider: refactor code and optimize out get_field method
by dexter 25 Nov '25

25 Nov '25

Attention is currently required from: dexter, laforge. Hello Jenkins Builder, laforge, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/c/pysim/+/41481?usp=email to look at the new patch set (#2). The following approvals got outdated and were removed: Code-Review+1 by laforge, Verified+1 by Jenkins Builder Change subject: card_key_provider: refactor code and optimize out get_field method ...................................................................... card_key_provider: refactor code and optimize out get_field method The method get_field in the base class can be optimized out. This also allows us to remove code dup in the card_key_provider_get_field function. Let's also fix the return code behavior. A get method in a CardKeyProvider implementation should always return None in case nothing is found. Also it should not crash in that case. This will allow the card_key_provider_get function to move on to the next CardKeyProvider. In case no CardKeyProvider yields any results, an exception is appropriate since it is pointless to continue execution with "None" as key material. To make the debugging of problems easier, let's also print some debug messages that inform the user what key/value pair and which CardKeyProvider was queried. This will make it easier to investigate in case an expected result was not found. Related: SYS#7725 Change-Id: I4d6367b8eb057e7b2c06c8625094d8a1e4c8eef9 --- M pySim/card_key_provider.py 1 file changed, 26 insertions(+), 25 deletions(-) git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/81/41481/2 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/41481?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newpatchset Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I4d6367b8eb057e7b2c06c8625094d8a1e4c8eef9 Gerrit-Change-Number: 41481 Gerrit-PatchSet: 2 Gerrit-Owner: dexter <pmaier(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: dexter <pmaier(a)sysmocom.de>

1 0

2026

2025

2024

2023

2022

gerrit-log November 2025