April 2024 - gerrit-log - lists.osmocom.org

[S] Change in osmo-hnbgw[master]: jenkins.sh: add NFTABLES axis
by osmith 08 Apr '24

08 Apr '24

Attention is currently required from: neels. osmith has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-hnbgw/+/36537?usp=email ) Change subject: jenkins.sh: add NFTABLES axis ...................................................................... Patch Set 3: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/osmo-hnbgw/+/36537?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hnbgw Gerrit-Branch: master Gerrit-Change-Id: I331cce7b187cf427a5cbffb3aedb17054918bcc8 Gerrit-Change-Number: 36537 Gerrit-PatchSet: 3 Gerrit-Owner: neels <nhofmeyr(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: neels <nhofmeyr(a)sysmocom.de> Gerrit-Comment-Date: Mon, 08 Apr 2024 09:20:01 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

[M] Change in libosmocore[master]: add osmo_stats_report_lock api
by fixeria 08 Apr '24

08 Apr '24

Attention is currently required from: neels. fixeria has posted comments on this change. ( https://gerrit.osmocom.org/c/libosmocore/+/36538?usp=email ) Change subject: add osmo_stats_report_lock api ...................................................................... Patch Set 1: Code-Review+1 -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/36538?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: Ib335bea7d2a440ca284e6c439066f96456bf2c2d Gerrit-Change-Number: 36538 Gerrit-PatchSet: 1 Gerrit-Owner: neels <nhofmeyr(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Attention: neels <nhofmeyr(a)sysmocom.de> Gerrit-Comment-Date: Mon, 08 Apr 2024 06:18:18 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment

1 0

[S] Change in osmo-hnbgw[master]: nft_kpi: add rate_ctr gtpu:ue_bytes:ul,dl
by neels 08 Apr '24

08 Apr '24

neels has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-hnbgw/+/36539?usp=email ) Change subject: nft_kpi: add rate_ctr gtpu:ue_bytes:ul,dl ...................................................................... nft_kpi: add rate_ctr gtpu:ue_bytes:ul,dl So far we have the nftables based counters for total GTP-U bytes (UL, DL), as well as a packet count. Add another counter for the computed UE payload bytes: total_bytes - packets * (20 + 8 + 8) Related: SYS#6773 Change-Id: Ib2f0a9252715ea4b2fe9c367aa65f771357768ca --- M include/osmocom/hnbgw/hnbgw.h M include/osmocom/hnbgw/nft_kpi.h M src/osmo-hnbgw/hnbgw.c M src/osmo-hnbgw/nft_kpi.c 4 files changed, 46 insertions(+), 2 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-hnbgw refs/changes/39/36539/1 diff --git a/include/osmocom/hnbgw/hnbgw.h b/include/osmocom/hnbgw/hnbgw.h index d323bdf..c9142e9 100644 --- a/include/osmocom/hnbgw/hnbgw.h +++ b/include/osmocom/hnbgw/hnbgw.h @@ -139,8 +139,10 @@ HNB_CTR_GTPU_PACKETS_UL, HNB_CTR_GTPU_TOTAL_BYTES_UL, + HNB_CTR_GTPU_UE_BYTES_UL, HNB_CTR_GTPU_PACKETS_DL, HNB_CTR_GTPU_TOTAL_BYTES_DL, + HNB_CTR_GTPU_UE_BYTES_DL, }; enum hnb_stat { diff --git a/include/osmocom/hnbgw/nft_kpi.h b/include/osmocom/hnbgw/nft_kpi.h index 95304ce..7b3153d 100644 --- a/include/osmocom/hnbgw/nft_kpi.h +++ b/include/osmocom/hnbgw/nft_kpi.h @@ -6,7 +6,8 @@ struct nft_kpi_val { uint64_t packets; - uint64_t bytes; + uint64_t total_bytes; + uint64_t ue_bytes; bool handle_present; int64_t handle; diff --git a/src/osmo-hnbgw/hnbgw.c b/src/osmo-hnbgw/hnbgw.c index a3ea265..6340d10 100644 --- a/src/osmo-hnbgw/hnbgw.c +++ b/src/osmo-hnbgw/hnbgw.c @@ -468,6 +468,10 @@ "gtpu:total_bytes:ul", "Count of total GTP-U bytes received from the HNB, including the GTP-U/UDP/IP headers", }, + [HNB_CTR_GTPU_UE_BYTES_UL] = { + "gtpu:ue_bytes:ul", + "Assuming an IP header length of 20 bytes, GTP-U bytes received from the HNB, excluding the GTP-U/UDP/IP headers", + }, [HNB_CTR_GTPU_PACKETS_DL] = { "gtpu:packets:dl", "Count of GTP-U packets sent to the HNB", @@ -476,6 +480,10 @@ "gtpu:total_bytes:dl", "Count of total GTP-U bytes sent to the HNB, including the GTP-U/UDP/IP headers", }, + [HNB_CTR_GTPU_UE_BYTES_DL] = { + "gtpu:ue_bytes:dl", + "Assuming an IP header length of 20 bytes, GTP-U bytes sent to the HNB, excluding the GTP-U/UDP/IP headers", + }, }; diff --git a/src/osmo-hnbgw/nft_kpi.c b/src/osmo-hnbgw/nft_kpi.c index 2eb5496..f585415 100644 --- a/src/osmo-hnbgw/nft_kpi.c +++ b/src/osmo-hnbgw/nft_kpi.c @@ -268,7 +268,24 @@ &val->packets, packets); update_ctr(hnbp->ctrs, ul ? HNB_CTR_GTPU_TOTAL_BYTES_UL : HNB_CTR_GTPU_TOTAL_BYTES_DL, - &val->bytes, bytes); + &val->total_bytes, bytes); + + LOGP(DLGLOBAL, LOGL_ERROR, "XXX %s(): packets=%ld btes=%ld ueb=%ld\n", __func__, + packets, bytes, bytes - packets * 36); + + /* Assuming an IP header of 20 bytes, derive the GTP-U payload size: + * + * [...] \ \ + * [ UDP ][ TCP ] | UE payload | nft reports these bytes + * [ IP ] / | + * -- payload -- | + * [ GTP-U 8 bytes ] | \ + * [ UDP 8 bytes ] | | need to subtract these, ~20 + 8 + 8 + * [ IP 20 bytes ] / / + */ + update_ctr(hnbp->ctrs, + ul ? HNB_CTR_GTPU_UE_BYTES_UL : HNB_CTR_GTPU_UE_BYTES_DL, + &val->ue_bytes, bytes - OSMO_MIN(bytes, packets * (20 + 8 + 8))); } /* In the string section *pos .. end, find the first occurrence of after_str and return the following token, which ends -- To view, visit https://gerrit.osmocom.org/c/osmo-hnbgw/+/36539?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hnbgw Gerrit-Branch: master Gerrit-Change-Id: Ib2f0a9252715ea4b2fe9c367aa65f771357768ca Gerrit-Change-Number: 36539 Gerrit-PatchSet: 1 Gerrit-Owner: neels <nhofmeyr(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[M] Change in osmo-hnbgw[master]: nft_kpi: retrieve counters in a separate thread
by neels 08 Apr '24

08 Apr '24

neels has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-hnbgw/+/36540?usp=email ) Change subject: nft_kpi: retrieve counters in a separate thread ...................................................................... nft_kpi: retrieve counters in a separate thread Introduce an NFT thread which does: - periodically run nftables command to read all counters - parse the response - update rate_ctr values. The main thread still runs the rule addition / removal when a HNB registers or deregisters. See the comment added in nft_kpi.c, starting with "A more scalable solution...". This patch requires the new osmo_stats_report_lock(), see 'Depends'. Related: SYS#6773 Depends: libosmocore Ib335bea7d2a440ca284e6c439066f96456bf2c2d Change-Id: I9dc54e6bc94c553f45adfa71ae8ad70be4afbc8f --- M src/osmo-hnbgw/hnbgw.c M src/osmo-hnbgw/nft_kpi.c M src/osmo-hnbgw/osmo_hnbgw_main.c M src/osmo-hnbgw/tdefs.c 4 files changed, 171 insertions(+), 34 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-hnbgw refs/changes/40/36540/1 diff --git a/src/osmo-hnbgw/hnbgw.c b/src/osmo-hnbgw/hnbgw.c index 6340d10..fa516ea 100644 --- a/src/osmo-hnbgw/hnbgw.c +++ b/src/osmo-hnbgw/hnbgw.c @@ -513,6 +513,12 @@ if (!hnbp) return NULL; + /* nft_kpi.c updates hnb stats, and it locks osmo_stats_report_lock() while it does so. This here should run + * in the same thread as stats reporting, so there should be no conflict with stats. But to avoid a hnb in flux + * while nft_kpi.c looks up hnb, also obtain the osmo_stats_report_lock() while manipulating hnb records. */ + osmo_stats_report_lock(true); + /* { */ + hnbp->id = *id; hnbp->id_str = talloc_strdup(hnbp, umts_cell_id_name(id)); hnbp->ctrs = rate_ctr_group_alloc(hnbp, &hnb_ctrg_desc, 0); @@ -525,14 +531,21 @@ osmo_stat_item_group_set_name(hnbp->statg, hnbp->id_str); llist_add(&hnbp->list, &g_hnbgw->hnb_persistent_list); + /* success */ + goto out_unlock; - return hnbp; - + /* failure */ out_free_ctrs: rate_ctr_group_free(hnbp->ctrs); out_free: talloc_free(hnbp); - return NULL; + hnbp = NULL; + + /* for both success and failure: */ +out_unlock: + /* } */ + osmo_stats_report_lock(false); + return hnbp; } struct hnb_persistent *hnb_persistent_find_by_id(const struct umts_cell_id *id) diff --git a/src/osmo-hnbgw/nft_kpi.c b/src/osmo-hnbgw/nft_kpi.c index f585415..039c891 100644 --- a/src/osmo-hnbgw/nft_kpi.c +++ b/src/osmo-hnbgw/nft_kpi.c @@ -15,7 +15,11 @@ * GNU Affero General Public License for more details. */ +#include <pthread.h> +#include <unistd.h> + #include <osmocom/core/logging.h> +#include <osmocom/core/stats.h> #include <osmocom/hnbgw/hnbgw.h> #include "config.h" @@ -46,13 +50,68 @@ #include <osmocom/hnbgw/nft_kpi.h> #include <osmocom/hnbgw/tdefs.h> +/* Threading and locks in this implementation: + * + * - osmo_stats_report_lock() held while updating rate_ctr from nft results. + * - g_nft_kpi_state.lock held while running an nftables command buffer. + * + * contrived example: + * + * Main Thread + * | + * osmo_stats_report_use_lock(true) + * | + * nft_kpi_init() + * create nft ctx, create table + * | + * +---------------------------> NFT thread + * | | + * | sleep(X34) + * | | + * | LOCK(g_nft_kpi_state.lock) + * | | + * osmo_stats_report() query all nft counters + * LOCK(osmo_stats_report_lock) | + * | LOCK(osmo_stats_report_lock) + * collect stats : wait because libosmocore stats reporting is busy + * | : + * UNLOCK(osmo_stats_report_lock) LOCK------| + * send out stats for all hnbp: rate_ctr_add2() + * | | + * | UNLOCK(osmo_stats_report_lock) + * | | + * | UNLOCK(g_nft_kpi_state.lock) + * | | + * hnbgw_rx_hnb_register_req() sleep(X34) + * hnb_nft_kpi_start() | + * LOCK(g_nft_kpi_state.lock) ... + * | + * nftables: add new rule + * | + * UNLOCK(g_nft_kpi_state.lock) + * | + * ... + * + * So the NFT thread only retrieves counters. The main thread adds and removes NFT rules for counters. It is possible + * that a HNBAP HNB Register or HNB De-Register occurrs while the NFT thread holds the g_nft_kpi_state.lock, so that the + * main thread blocks until the NFT thread is done reading counters. Note, this happens only for HNB attach or detach. + * + * A more scalable solution is to move all NFT interaction to the thread. Instead of submitting rules from the main + * thread, we could submit instructions to an inter-thread queue that the NFT thread works off. This would add + * considerable complexity -- for now we accept the possible but rarely occurring short delay for HNB de/registration. + */ + struct nft_kpi_state { + /* lock this while modifying g_nft_kpi_state */ + pthread_mutex_t lock; + struct { struct nft_ctx *nft_ctx; char *table_name; bool table_created; } nft; - struct osmo_timer_list period; + + pthread_t thread; }; static struct nft_kpi_state g_nft_kpi_state = {}; @@ -98,15 +157,22 @@ return 0; } -static void nft_kpi_period_cb(void *data); +static void nft_kpi_period_cb(void); -static void nft_kpi_period_schedule(void) +void *nft_kpi_thread(void *arg) { - unsigned long period = osmo_tdef_get(hnbgw_T_defs, -34, OSMO_TDEF_S, 10); - if (period < 1) - period = 1; - osmo_timer_setup(&g_nft_kpi_state.period, nft_kpi_period_cb, NULL); - osmo_timer_schedule(&g_nft_kpi_state.period, period, 0); + OSMO_ASSERT(osmo_ctx_init(__func__) == 0); + while (1) { + /* Let's just hope that the unsigned long in the hnbgw_T_defs is not modified non-atomically while + * reading the timeout value. */ + unsigned long period = osmo_tdef_get(hnbgw_T_defs, -34, OSMO_TDEF_US, 1000000); + if (period < 1) + period = 1; + usleep(period); + + nft_kpi_period_cb(); + } + return NULL; } int nft_kpi_init(const char *table_name) @@ -145,7 +211,11 @@ return -EIO; s->nft.table_created = true; - nft_kpi_period_schedule(); + + /* Up to here, it was fine to dispatch nft without locking, because this is the initialization from the main + * thread. From now on, whoever wants to use the g_nft_ctx must lock this mutex first. */ + pthread_mutex_init(&s->lock, NULL); + pthread_create(&s->thread, NULL, nft_kpi_thread, NULL); return 0; } @@ -163,14 +233,19 @@ return -EINVAL; } + /* Manipulating nft state, obtain lock */ + pthread_mutex_lock(&s->lock); + /* { */ + if (!osmo_sockaddr_str_cmp(gtpu_remote, &hnbp->nft_kpi.addr_remote)) { /* The remote address is unchanged, no need to update the nft probe */ - return 0; + rc = 0; + goto unlock_return; } /* When there is no table created, it means nft is disabled. Do not attempt to set up counters. */ if (!s->nft.table_created) - return 0; + goto unlock_return; /* The remote address has changed. Cancel previous probe, if any, and start a new one. */ if (osmo_sockaddr_str_is_nonzero(&hnbp->nft_kpi.addr_remote)) @@ -196,6 +271,10 @@ /* There was an error running the rule, clear addr_remote to indicate that no rule exists. */ hnbp->nft_kpi.addr_remote = (struct osmo_sockaddr_str){}; } + +unlock_return: + /* } */ + pthread_mutex_unlock(&s->lock); return rc; } @@ -206,15 +285,19 @@ { struct nft_kpi_state *s = &g_nft_kpi_state; char *cmd; + int rc = 0; /* When there is no table created, neither can there be any rules to be deleted. * The rules get removed, but the table remains present for as long as osmo-hnbgw runs. */ if (!s->nft.table_created) return 0; + pthread_mutex_lock(&s->lock); + /* { */ + /* presence of addr_remote indicates whether an nft rule has been submitted and still needs to be removed */ if (!osmo_sockaddr_str_is_nonzero(&hnbp->nft_kpi.addr_remote)) - return 0; + goto unlock_return; if (!hnbp->nft_kpi.last.ul.handle_present || !hnbp->nft_kpi.last.dl.handle_present) { @@ -235,7 +318,12 @@ hnbp->nft_kpi.last.ul.handle, s->nft.table_name, hnbp->nft_kpi.last.dl.handle); - return nft_run_now(cmd); + rc = nft_run_now(cmd); + +unlock_return: + /* } */ + pthread_mutex_unlock(&s->lock); + return rc; } static void update_ctr(struct rate_ctr_group *cg, int cgidx, uint64_t *last_val, uint64_t new_val) @@ -383,6 +471,7 @@ LOGP(DNFT, LOGL_DEBUG, "read %d counters from nft table %s\n", count, s->nft.table_name); } +/* The caller must hold the g_nft_kpi_state.lock! */ static void nft_kpi_read_counters(void) { int rc; @@ -399,6 +488,12 @@ OSMO_STRBUF_PRINTF(sb, "list table inet %s", s->nft.table_name); OSMO_ASSERT(sb.chars_needed < sizeof(cmd)); + size_t l = strlen(cmd); + LOGP(DNFT, LOGL_DEBUG, "running nft request, %zu chars: \"%s%s\"\n", + l, + osmo_escape_cstr_c(OTC_SELECT, cmd, OSMO_MIN(logmax, l)), + l > logmax ? "..." : ""); + rc = nft_ctx_buffer_output(nft); if (rc) { LOGP(DNFT, LOGL_ERROR, "error: nft_ctx_buffer_output() returned failure: rc=%d cmd=%s\n", @@ -413,29 +508,29 @@ } output = nft_ctx_get_output_buffer(nft); - if (log_check_level(DNFT, LOGL_DEBUG)) { - size_t l = strlen(cmd); - LOGP(DNFT, LOGL_DEBUG, "ran nft request, %zu chars: \"%s%s\"\n", - l, - osmo_escape_cstr_c(OTC_SELECT, cmd, OSMO_MIN(logmax, l)), - l > logmax ? "..." : ""); - l = strlen(output); - LOGP(DNFT, LOGL_DEBUG, "got nft response, %zu chars: \"%s%s\"\n", - l, - osmo_escape_cstr_c(OTC_SELECT, output, OSMO_MIN(logmax, l)), - l > logmax ? "..." : ""); - } + l = strlen(output); + LOGP(DNFT, LOGL_DEBUG, "got nft response, %zu chars: \"%s%s\"\n", + l, + osmo_escape_cstr_c(OTC_SELECT, output, OSMO_MIN(logmax, l)), + l > logmax ? "..." : ""); + osmo_stats_report_lock(true); + /* { */ decode_nft_response(output); + /* } */ + osmo_stats_report_lock(false); unbuffer_and_exit: nft_ctx_unbuffer_output(nft); } -static void nft_kpi_period_cb(void *data) +static void nft_kpi_period_cb(void) { + pthread_mutex_lock(&g_nft_kpi_state.lock); + /* { */ nft_kpi_read_counters(); - nft_kpi_period_schedule(); + /* } */ + pthread_mutex_unlock(&g_nft_kpi_state.lock); } #endif // ENABLE_NFTABLES diff --git a/src/osmo-hnbgw/osmo_hnbgw_main.c b/src/osmo-hnbgw/osmo_hnbgw_main.c index 1052d18..36180ef 100644 --- a/src/osmo-hnbgw/osmo_hnbgw_main.c +++ b/src/osmo-hnbgw/osmo_hnbgw_main.c @@ -328,16 +328,23 @@ /* If UPF is configured, set up PFCP socket and send Association Setup Request to UPF */ hnbgw_pfcp_init(); #endif -#if ENABLE_NFTABLES + /* If nftables is enabled, initialize the nft table now or fail startup. This is important to immediately let * the user know if cap_net_admin privileges are missing, and not only when the first hNodeB connects. */ if (g_hnbgw->config.nft_kpi.enable) { +#if ENABLE_NFTABLES if (nft_kpi_init(g_hnbgw->config.nft_kpi.table_name)) { - perror("Failed to initialize nft KPI, probably missing cap_net_admin"); + perror("ERROR: Failed to initialize nft KPI, probably missing cap_net_admin"); exit(1); } - } + /* nft_kpi.c manipulates rate_ctr state directly. Enable the mutex lock around stats reporting, so + * nft_kpi.c can make use of it. */ + osmo_stats_report_use_lock(true); +#else + fprintf(stderr, "ERROR: Cannot enable nft KPI, this binary was built without nftables support\n"); + exit(1); #endif + } hnbgw_cnpool_start(&g_hnbgw->sccp.cnpool_iucs); hnbgw_cnpool_start(&g_hnbgw->sccp.cnpool_iups); diff --git a/src/osmo-hnbgw/tdefs.c b/src/osmo-hnbgw/tdefs.c index 3a7f0bc..cfcd4c2 100644 --- a/src/osmo-hnbgw/tdefs.c +++ b/src/osmo-hnbgw/tdefs.c @@ -35,8 +35,8 @@ {.T = 3113, .default_val = 15, .desc = "Time to keep Paging record, for CN pools with more than one link" }, {.T = 4, .default_val = 5, .desc = "Timeout to receive RANAP RESET ACKNOWLEDGE from an MSC/SGSN" }, {.T = -31, .default_val = 15, .desc = "Timeout for establishing and releasing context maps (RUA <-> SCCP)" }, + {.T = -34, .default_val = 1000, .unit = OSMO_TDEF_MS, .desc = "Period to query network traffic stats from netfilter" }, {.T = -1002, .default_val = 10, .desc = "Timeout for the HNB to respond to PS RAB Assignment Request" }, - {.T = -34, .default_val = 1, .desc = "Period to query network traffic stats from netfilter" }, { } }; -- To view, visit https://gerrit.osmocom.org/c/osmo-hnbgw/+/36540?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hnbgw Gerrit-Branch: master Gerrit-Change-Id: I9dc54e6bc94c553f45adfa71ae8ad70be4afbc8f Gerrit-Change-Number: 36540 Gerrit-PatchSet: 1 Gerrit-Owner: neels <nhofmeyr(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[M] Change in libosmocore[master]: add osmo_stats_report_lock api
by neels 08 Apr '24

08 Apr '24

neels has uploaded this change for review. ( https://gerrit.osmocom.org/c/libosmocore/+/36538?usp=email ) Change subject: add osmo_stats_report_lock api ...................................................................... add osmo_stats_report_lock api Allow multi-threaded access to reported stats: - enable use of a stats mutex with osmo_stats_report_use_lock(true). - lock/unlock externally with osmo_stats_report_lock(true/false). Rationale: In osmo-hnbgw, we would like to collect stats from nftables, and do so in a separate thead. The most efficient way is to write the parsing results from nft directly to the rate_ctr destination. But what if the main thread reports rate counters at exactly that time? - is writing to stats atomic on a data type level? - do applications need stats to be "atomic" as a whole? In osmo-hnbgw in particular, there are two counters, 'packets' and 'total_bytes'. These correspond, and it would skew statistics if we reported them out of sync to each other. The simplest way to ensure correctness in all cases is a mutex around the stats reporting. But this mutex isn't needed in most of our programs. To completely avoid any overhead the mutex may bring, make use of it optional with a global flag. This use case is likely to also show up in other programs that would like to collect and report stats from a separate thread. Related: SYS#6773 Related: osmo-hnbgw I9dc54e6bc94c553f45adfa71ae8ad70be4afbc8f Change-Id: Ib335bea7d2a440ca284e6c439066f96456bf2c2d --- M include/osmocom/core/stats.h M src/core/libosmocore.map M src/core/stats.c M src/vty/stats_vty.c 4 files changed, 96 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/38/36538/1 diff --git a/include/osmocom/core/stats.h b/include/osmocom/core/stats.h index a034a61..4ec448c 100644 --- a/include/osmocom/core/stats.h +++ b/include/osmocom/core/stats.h @@ -109,6 +109,8 @@ void osmo_stats_init(void *ctx); int osmo_stats_report(void); +void osmo_stats_report_use_lock(bool enable); +void osmo_stats_report_lock(bool lock); int osmo_stats_set_interval(int interval); diff --git a/src/core/libosmocore.map b/src/core/libosmocore.map index c5ab6e3..a3c1db6 100644 --- a/src/core/libosmocore.map +++ b/src/core/libosmocore.map @@ -488,6 +488,8 @@ osmo_stats_config; osmo_stats_init; osmo_stats_report; +osmo_stats_report_use_lock; +osmo_stats_report_lock; osmo_stats_reporter_alloc; osmo_stats_reporter_create_log; osmo_stats_reporter_create_statsd; diff --git a/src/core/stats.c b/src/core/stats.c index 16e6f62..21c7ddd 100644 --- a/src/core/stats.c +++ b/src/core/stats.c @@ -71,6 +71,7 @@ #include <stdio.h> #include <sys/types.h> #include <inttypes.h> +#include <pthread.h> #ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> @@ -787,13 +788,59 @@ } } +static pthread_mutex_t *g_report_lock = NULL; + +/*! Enable osmo_stats_report() mutex locking with osmo_stats_report_lock(). + * Calling osmo_stats_report_use_lock(true) */ +void osmo_stats_report_use_lock(bool enable) +{ + static pthread_mutex_t report_lock; + if (enable) { + /* enable locking */ + if (!g_report_lock) { + pthread_mutex_init(&report_lock, NULL); + g_report_lock = &report_lock; + } + return; + } + /* disable locking */ + if (g_report_lock) { + pthread_mutex_destroy(g_report_lock); + g_report_lock = NULL; + } +} + +/*! After osmo_stats_report_use_lock(true), lock a mutex to avoid osmo_stats_report() reading from stats. + * The caller must ensure to call osmo_stats_report_lock(false) as soon as possible, or the application will lock up. + * This function has no effect with osmo_stats_report_use_lock(false). + * Useful in multi-threaded applications to allow writing to stats/counters/rate_ctrs directly from various threads. + * Particularly useful to ensure that a given set of stats/counters updates from another thread are reported in sync. + */ +void osmo_stats_report_lock(bool lock) +{ + if (!g_report_lock) + return; + if (lock) + pthread_mutex_lock(g_report_lock); + else + pthread_mutex_unlock(g_report_lock); +} + int osmo_stats_report(void) { + pthread_mutex_t *lock = g_report_lock; + /* per group actions */ TRACE(LIBOSMOCORE_STATS_START()); + if (lock) + pthread_mutex_lock(lock); + /* { */ osmo_counters_for_each(handle_counter, NULL); rate_ctr_for_each_group(rate_ctr_group_handler, NULL); osmo_stat_item_for_each_group(osmo_stat_item_group_handler, NULL); + /* } */ + if (lock) + pthread_mutex_unlock(lock); /* global actions */ flush_all_reporters(); diff --git a/src/vty/stats_vty.c b/src/vty/stats_vty.c index f940018..4a62dc4 100644 --- a/src/vty/stats_vty.c +++ b/src/vty/stats_vty.c @@ -426,7 +426,9 @@ if (argc > 0) skip_zero = true; + osmo_stats_report_lock(true); vty_out_statistics_full2(vty, "", skip_zero); + osmo_stats_report_lock(false); return CMD_SUCCESS; } @@ -444,7 +446,9 @@ bool skip_zero = false; if (argc > 1) skip_zero = true; + osmo_stats_report_lock(true); vty_out_statistics_partial2(vty, "", level, skip_zero); + osmo_stats_report_lock(false); return CMD_SUCCESS; } @@ -632,7 +636,9 @@ struct rctr_vty_ctx rctx = { .vty = vty, .skip_zero = false }; if (argc > 0) rctx.skip_zero = true; + osmo_stats_report_lock(true); rate_ctr_for_each_group(rate_ctr_group_handler, &rctx); + osmo_stats_report_lock(false); return CMD_SUCCESS; } -- To view, visit https://gerrit.osmocom.org/c/libosmocore/+/36538?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: libosmocore Gerrit-Branch: master Gerrit-Change-Id: Ib335bea7d2a440ca284e6c439066f96456bf2c2d Gerrit-Change-Number: 36538 Gerrit-PatchSet: 1 Gerrit-Owner: neels <nhofmeyr(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[S] Change in osmo-hnbgw[master]: jenkins.sh: add NFTABLES axis
by neels 07 Apr '24

07 Apr '24

neels has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-hnbgw/+/36537?usp=email ) Change subject: jenkins.sh: add NFTABLES axis ...................................................................... jenkins.sh: add NFTABLES axis Related: osmo-ci I9828b70708dbe466c37df6ffb87b04362f14c71c Related: OS#6425 Change-Id: I331cce7b187cf427a5cbffb3aedb17054918bcc8 --- M contrib/jenkins.sh 1 file changed, 16 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-hnbgw refs/changes/37/36537/1 diff --git a/contrib/jenkins.sh b/contrib/jenkins.sh index dfbbbfe..9493488 100755 --- a/contrib/jenkins.sh +++ b/contrib/jenkins.sh @@ -4,9 +4,11 @@ # environment variables: # * PFCP: configure PFCP support if set to "1" (default) # * WITH_MANUALS: build manual PDFs if set to "1" +# * NFTABLES: configure nftables support if set to "1" (default) # * PUBLISH: upload manuals after building if set to "1" (ignored without WITH_MANUALS = "1") # PFCP=${PFCP:-1} +NFTABLES=${NFTABLES:-1} if ! [ -x "$(command -v osmo-build-dep.sh)" ]; then echo "Error: We need to have scripts/osmo-deps.sh from http://git.osmocom.org/osmo-ci/ in PATH !" @@ -45,6 +47,9 @@ osmo-build-dep.sh libosmo-pfcp CONFIG="$CONFIG --enable-pfcp" fi +if [ "$NFTABLES" = "1" ]; then + CONFIG="$CONFIG --enable-nftables" +fi if [ "$WITH_MANUALS" = "1" ]; then CONFIG="$CONFIG --enable-manuals" fi -- To view, visit https://gerrit.osmocom.org/c/osmo-hnbgw/+/36537?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hnbgw Gerrit-Branch: master Gerrit-Change-Id: I331cce7b187cf427a5cbffb3aedb17054918bcc8 Gerrit-Change-Number: 36537 Gerrit-PatchSet: 1 Gerrit-Owner: neels <nhofmeyr(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[S] Change in osmo-ttcn3-hacks[master]: WIP
by pespin 06 Apr '24

06 Apr '24

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36533?usp=email ) Change subject: WIP ...................................................................... WIP Change-Id: I12ce55a6ba20ec5d0e17d8e225a002b391dc5285 --- M pgw/PGW_Tests.ttcn 1 file changed, 10 insertions(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks refs/changes/33/36533/1 diff --git a/pgw/PGW_Tests.ttcn b/pgw/PGW_Tests.ttcn index a1850a0..5d7eaa0 100644 --- a/pgw/PGW_Tests.ttcn +++ b/pgw/PGW_Tests.ttcn @@ -270,7 +270,7 @@ vc_S6b.start(DIAMETER_Emulation.main(ops, pars, id)); f_diameter_wait_capability(Gx_UNIT); - f_diameter_wait_capability(Gy_UNIT); + //f_diameter_wait_capability(Gy_UNIT); f_diameter_wait_capability(S6b_UNIT); /* Give some time for our emulation to get out of SUSPECT list of SUT (3 watchdong ping-pongs): * RFC6733 sec 5.1 -- To view, visit https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36533?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-Change-Id: I12ce55a6ba20ec5d0e17d8e225a002b391dc5285 Gerrit-Change-Number: 36533 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[S] Change in osmo-ttcn3-hacks[master]: pgw: Always expect local TEIC when receiving DeleteSessionResponse
by pespin 06 Apr '24

06 Apr '24

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36534?usp=email ) Change subject: pgw: Always expect local TEIC when receiving DeleteSessionResponse ...................................................................... pgw: Always expect local TEIC when receiving DeleteSessionResponse open5gs-smfd is since recently acting properly and setting the proper TEIC when sending GTPv2C messages for cases where session is not found. Related: https://github.com/open5gs/open5gs/issues/3043 Change-Id: I2086b4576a10c7b4ba8217be6e49c8f5f389bc4b --- M pgw/PGW_Tests.ttcn 1 file changed, 18 insertions(+), 5 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks refs/changes/34/36534/1 diff --git a/pgw/PGW_Tests.ttcn b/pgw/PGW_Tests.ttcn index 5d7eaa0..7976660 100644 --- a/pgw/PGW_Tests.ttcn +++ b/pgw/PGW_Tests.ttcn @@ -833,7 +833,6 @@ /* delete the session from the PGW */ private function f_delete_session(template (omit) GTP2C_Cause tx_cause := omit, - template (present) OCT4 exp_teid, template (present) GTP2C_Cause exp_cause, boolean expect_diameter := true) runs on PGW_Session_CT { var template (value) FullyQualifiedTEID fteid_c_ie @@ -852,7 +851,7 @@ as_DIA_Gy_CCR(TERMINATION_REQUEST); } alt { - [] GTP2.receive(tr_GTP2C_DeleteSessionResp(d_teid := exp_teid, cause := exp_cause)) { + [] GTP2.receive(tr_GTP2C_DeleteSessionResp(d_teid := g_pars.teic_local, cause := exp_cause)) { setverdict(pass); } [] GTP2.receive(tr_GTP2C_DeleteSessionResp(?, ?)) { @@ -1021,7 +1020,7 @@ /* create a session, then delete it again */ private function f_TC_createSession_deleteSession() runs on PGW_Session_CT { f_s5s8_create_session(); - f_delete_session(omit, g_pars.teic_local, Request_accepted); + f_delete_session(omit, Request_accepted); setverdict(pass); } testcase TC_createSession_deleteSession() runs on PGW_Test_CT { @@ -1035,7 +1034,7 @@ /* send a DeleteSessionReq for an unknown/invalid TEID */ private function f_TC_deleteSession_unknown() runs on PGW_Session_CT { g_pars.teic_remote := f_rnd_octstring(4); - f_delete_session(omit, '00000000'O, Context_Not_Found, false); + f_delete_session(omit, Context_Not_Found, false); setverdict(pass); } testcase TC_deleteSession_unknown() runs on PGW_Test_CT { @@ -1066,7 +1065,7 @@ as_DIA_Gy_CCR(UPDATE_REQUEST); f_validate_gy_cc_report(g_rx_gy, VALIDITY_TIME, (3..4), ?, ?); - f_delete_session(omit, g_pars.teic_local, Request_accepted); + f_delete_session(omit, Request_accepted); f_validate_gy_cc_report(g_rx_gy, FINAL, (0..1), 0, 0); setverdict(pass); } -- To view, visit https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36534?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-Change-Id: I2086b4576a10c7b4ba8217be6e49c8f5f389bc4b Gerrit-Change-Number: 36534 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[S] Change in osmo-ttcn3-hacks[master]: asterisk: Add missing 0-digit prefix in user
by pespin 05 Apr '24

05 Apr '24

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36531?usp=email ) Change subject: asterisk: Add missing 0-digit prefix in user ...................................................................... asterisk: Add missing 0-digit prefix in user Change-Id: Ia12552cf70c4c0f2c71cd9d5b70c608c883fb170 --- M asterisk/Asterisk_Tests.ttcn 1 file changed, 10 insertions(+), 1 deletion(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks refs/changes/31/36531/1 diff --git a/asterisk/Asterisk_Tests.ttcn b/asterisk/Asterisk_Tests.ttcn index 5d9754e..9402f0d 100644 --- a/asterisk/Asterisk_Tests.ttcn +++ b/asterisk/Asterisk_Tests.ttcn @@ -79,7 +79,7 @@ } function f_init_ConnHdlrPars(integer idx := 1) runs on test_CT return ConnHdlrPars { - var ConnHdlrPars pars := valueof(t_Pars(int2str(500 + idx))); + var ConnHdlrPars pars := valueof(t_Pars("0" & int2str(500 + idx))); return pars; } -- To view, visit https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36531?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-Change-Id: Ia12552cf70c4c0f2c71cd9d5b70c608c883fb170 Gerrit-Change-Number: 36531 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

[L] Change in osmo-ttcn3-hacks[master]: asterisk: Implement and test SIP Digest Authorization
by pespin 05 Apr '24

05 Apr '24

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36532?usp=email ) Change subject: asterisk: Implement and test SIP Digest Authorization ...................................................................... asterisk: Implement and test SIP Digest Authorization Related: SYS#6782 Change-Id: Ib469f1906927a3f246876040086ff115fbf4c032 --- M asterisk/Asterisk_Tests.ttcn M asterisk/gen_links.sh M asterisk/regen_makefile.sh M library/SIP_Templates.ttcn M sip/SIP_Tests.ttcn M sip/gen_links.sh M sip/regen_makefile.sh 7 files changed, 609 insertions(+), 19 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks refs/changes/32/36532/1 diff --git a/asterisk/Asterisk_Tests.ttcn b/asterisk/Asterisk_Tests.ttcn index 9402f0d..a300bd1 100644 --- a/asterisk/Asterisk_Tests.ttcn +++ b/asterisk/Asterisk_Tests.ttcn @@ -11,6 +11,7 @@ * SPDX-License-Identifier: GPL-2.0-or-later */ +import from TCCOpenSecurity_Functions all; import from General_Types all; import from Osmocom_Types all; import from Native_Functions all; @@ -44,6 +45,7 @@ type record ConnHdlrPars { float t_guard, charstring user, + charstring password, SipUrl registrar_sip_url, SipAddr registrar_sip_record, CallidString registrar_sip_call_id, @@ -55,9 +57,11 @@ } template (value) ConnHdlrPars t_Pars(charstring user, - charstring displayname := "\"Anonymous\"") := { + charstring displayname := "\"Anonymous\"", + charstring password := "secret") := { t_guard := 30.0, user := user, + password := password, registrar_sip_url := valueof(ts_SipUrlHost(mp_remote_sip_host)), registrar_sip_record := ts_SipAddr(ts_HostPort(mp_remote_sip_host), ts_UserInfo(user), @@ -188,29 +192,81 @@ via_resp_params); } +private function f_tr_To_response(SipAddr to_req) return template (present) SipAddr { + template (present) SemicolonParam_List via_resp_params := ?; + return tr_SipAddr_from_val(to_req); +} + function f_SIP_register() runs on ConnHdlr return PDU_SIP_Response { var template (present) PDU_SIP_Response exp; + var Authorization authorization; + var Via via := g_pars.registrar_via; + var SipAddr from_sipaddr := g_pars.registrar_sip_record; + var charstring branch_value; + branch_value := f_sip_gen_branch(f_sip_SipAddr_to_str(g_pars.registrar_sip_record), + f_sip_SipAddr_to_str(g_pars.registrar_sip_record), + g_pars.registrar_sip_call_id, + g_pars.registrar_sip_seq_nr); + + via.viaBody[0].viaParams := f_sip_param_set(via.viaBody[0].viaParams, "branch", branch_value); + from_sipaddr.params := f_sip_param_set(from_sipaddr.params, "tag", f_sip_rand_tag()); SIP.send(ts_SIP_REGISTER(g_pars.registrar_sip_url, g_pars.registrar_sip_call_id, + from_sipaddr, g_pars.registrar_sip_record, - g_pars.registrar_sip_record, - g_pars.registrar_via, + via, g_pars.registrar_sip_seq_nr, g_pars.local_contact, ts_Expires("7200"))); exp := tr_SIP_Response_REGISTER_Unauthorized( g_pars.registrar_sip_call_id, - g_pars.registrar_sip_record, - g_pars.registrar_sip_record, - f_tr_Via_response(g_pars.registrar_via), + from_sipaddr, + f_tr_To_response(g_pars.registrar_sip_record), + f_tr_Via_response(via), *, + tr_WwwAuthenticate({tr_Challenge_digestCln(?)}), g_pars.registrar_sip_seq_nr); as_SIP_expect_resp(exp); - /* Do the registering after calculating the md5 hash, etc. */ + /* Digest Auth: RFC 2617 */ + authorization := f_sip_digest_gen_Authorization(g_rx_sip_resp.msgHeader.wwwAuthenticate, + g_pars.user, g_pars.password, + "REGISTER", "sip:" & mp_remote_sip_host) + + /* New transaction: */ + g_pars.registrar_sip_seq_nr := g_pars.registrar_sip_seq_nr + 1; + branch_value := f_sip_gen_branch(f_sip_SipAddr_to_str(g_pars.registrar_sip_record), + f_sip_SipAddr_to_str(g_pars.registrar_sip_record), + g_pars.registrar_sip_call_id, + g_pars.registrar_sip_seq_nr); + via.viaBody[0].viaParams := f_sip_param_set(via.viaBody[0].viaParams, "branch", branch_value); + + SIP.send(ts_SIP_REGISTER(g_pars.registrar_sip_url, + g_pars.registrar_sip_call_id, + from_sipaddr, + g_pars.registrar_sip_record, + via, + g_pars.registrar_sip_seq_nr, + g_pars.local_contact, + ts_Expires("7200"), + authorization := authorization)); + + /* Wait for OK answer */ + exp := tr_SIP_Response( + g_pars.registrar_sip_call_id, + from_sipaddr, + f_tr_To_response(g_pars.registrar_sip_record), + f_tr_Via_response(via), + *, + "REGISTER", 200, + g_pars.registrar_sip_seq_nr, "OK"); + as_SIP_expect_resp(exp); + + /* Prepare for next use: */ + g_pars.registrar_sip_seq_nr := g_pars.registrar_sip_seq_nr + 1; return g_rx_sip_resp; } @@ -233,6 +289,11 @@ vc_conn.done; } +testcase TC_selftest() runs on test_CT { + f_sip_digest_selftest(); + setverdict(pass); +} + control { execute( TC_internal_registration() ); } diff --git a/asterisk/gen_links.sh b/asterisk/gen_links.sh index 1fd6ecc..7394b64 100755 --- a/asterisk/gen_links.sh +++ b/asterisk/gen_links.sh @@ -5,7 +5,8 @@ . ../gen_links.sh.inc DIR=$BASEDIR/titan.Libraries.TCCUsefulFunctions/src -FILES="TCCInterface_Functions.ttcn TCCConversion_Functions.ttcn TCCConversion.cc TCCInterface.cc TCCInterface_ip.h" +FILES="TCCInterface_Functions.ttcn TCCConversion_Functions.ttcn TCCConversion.cc TCCInterface.cc TCCInterface_ip.h " +FILES+="TCCOpenSecurity_Functions.ttcn TCCOpenSecurity.cc TCCOpenSecurity_Functions.hh " gen_links $DIR $FILES DIR=$BASEDIR/titan.TestPorts.Common_Components.Socket-API/src diff --git a/asterisk/regen_makefile.sh b/asterisk/regen_makefile.sh index 3995b3d..3cd0a66 100755 --- a/asterisk/regen_makefile.sh +++ b/asterisk/regen_makefile.sh @@ -14,6 +14,7 @@ SIPmsg_PT.cc TCCConversion.cc TCCInterface.cc + TCCOpenSecurity.cc " ../regen-makefile.sh -e $NAME $FILES diff --git a/library/SIP_Templates.ttcn b/library/SIP_Templates.ttcn index fc8e23f..7cb9d68 100644 --- a/library/SIP_Templates.ttcn +++ b/library/SIP_Templates.ttcn @@ -1,7 +1,11 @@ module SIP_Templates { import from SIPmsg_Types all; +import from TCCConversion_Functions all; +import from TCCOpenSecurity_Functions all; +import from Native_Functions all; import from Osmocom_Types all; +import from Misc_Helpers all; /* wrapper type to encapsulate the Addr_Union + parameter list used in From, To. ... */ type record SipAddr { @@ -11,6 +15,26 @@ const charstring c_SIP_VERSION := "SIP/2.0"; +template (value) GenericParam ts_Param(template (value) charstring id, + template (omit) charstring paramValue := omit) := { + id := id, + paramValue := paramValue +} +template (present) GenericParam tr_Param(template (present) charstring id := ?, + template charstring paramValue := *) := { + id := id, + paramValue := paramValue +} +function f_ts_Param_omit(template (value) charstring id, + template (omit) charstring paramValue := omit) + return template (omit) GenericParam +{ + if (istemplatekind(paramValue, "omit")) { + return omit; + } + return ts_Param(id, paramValue); +} + template (value) SipUrl ts_SipUrl(template (value) HostPort host_port, template (omit) UserInfo user_info := omit) := { scheme := "sip", @@ -31,6 +55,49 @@ template (value) SipUrl ts_SipUrlHost(template (value) charstring host) := ts_SipUrl(ts_HostPort(host)); +template (value) Credentials ts_Credentials_DigestResponse(template (value) CommaParam_List digestResponse) := { + digestResponse := digestResponse +} + +template (value) Credentials ts_Credentials_DigestResponseMD5( + template (value) charstring username, + template (value) charstring realm, + template (value) charstring nonce, + template (value) charstring uri, + template (value) charstring response, + template (value) charstring opaque, + template (value) charstring algorithm := "MD5", + template (value) charstring qop := "auth", + template (omit) charstring cnonce := omit, + template (omit) charstring nc := omit + ) := { + digestResponse := { + // Already added by digestResponse automatically: + //ts_Param("Digest", omit), + ts_Param("username", f_sip_str_quote(username)), + ts_Param("realm", f_sip_str_quote(realm)), + ts_Param("nonce", f_sip_str_quote(nonce)), + ts_Param("uri", f_sip_str_quote(uri)), + ts_Param("response", f_sip_str_quote(response)), + ts_Param("opaque", f_sip_str_quote(opaque)), + ts_Param("algorithm", algorithm), + ts_Param("qop", qop), + // FIXME: If "omit" is passed, these below end up in; + // "Dynamic test case error: Performing a valueof or send operation on a non-specific template of type @SIPmsg_Types.GenericParam" + f_ts_Param_omit("cnonce", f_sip_str_quote(cnonce)), + f_ts_Param_omit("nc", nc) + } +} + +template (value) Credentials ts_Credentials_OtherAuth(template (value) OtherAuth otherResponse) := { + otherResponse := otherResponse +} + +template (value) Authorization ts_Authorization(template (value) Credentials body) := { + fieldName := AUTHORIZATION_E, + body := body +} + // [20.10] template (present) NameAddr tr_NameAddr(template (present) SipUrl addrSpec := ?, template charstring displayName := *) := { @@ -262,7 +329,53 @@ viaParams := viaParams } } - } +} + +template (present) OtherAuth +tr_OtherAuth(template (present) charstring authScheme := ?, + template (present) CommaParam_List authParams := ?) := { + authScheme := authScheme, + authParams := authParams +} + +template (value) OtherAuth +ts_OtherAuth(template (value) charstring authScheme, + template (value) CommaParam_List authParams) := { + authScheme := authScheme, + authParams := authParams +} + +template (present) Challenge +tr_Challenge_digestCln(template (present) CommaParam_List digestCln := ?) := { + digestCln := digestCln +} + +template (value) Challenge +ts_Challenge_digestCln(template (value) CommaParam_List digestCln) := { + digestCln := digestCln +} + +template (present) Challenge +tr_Challenge_otherChallenge(template (present) OtherAuth otherChallenge := ?) := { + otherChallenge := otherChallenge +} + +template (value) Challenge +ts_Challenge_otherChallenge(template (value) OtherAuth otherChallenge) := { + otherChallenge := otherChallenge +} + +template (present) WwwAuthenticate +tr_WwwAuthenticate(template (present) Challenge_list challenge := ?) := { + fieldName := WWW_AUTHENTICATE_E, + challenge := challenge +} + +template (value) WwwAuthenticate +ts_WwwAuthenticate(template (value) Challenge_list challenge) := { + fieldName := WWW_AUTHENTICATE_E, + challenge := challenge +} template (value) MessageHeader ts_SIP_msgHeader_empty := c_SIP_msgHeader_empty; template (value) MessageHeader @@ -274,6 +387,7 @@ template (value) integer seq_nr, template (value) Via via, template (omit) ContentType content_type := omit, + template (omit)Authorization authorization := omit, template (value) Method_List allow_methods := c_SIP_defaultMethods, template (omit) Expires expires := omit ) modifies ts_SIP_msgHeader_empty := { @@ -281,6 +395,7 @@ fieldName := ALLOW_E, methods := allow_methods }, + authorization := authorization, callId := { fieldName := CALL_ID_E, callid := call_id @@ -337,7 +452,8 @@ template ContentType content_type := *, template integer seq_nr := ?, template Method_List allow_methods := *, - template Expires expires := * + template Expires expires := *, + template WwwAuthenticate wwwAuthenticate := * ) modifies t_SIP_msgHeader_any := { allow := tr_AllowMethods(allow_methods), callId := { @@ -363,7 +479,8 @@ toParams := to_addr.params }, userAgent := *, - via := via + via := via, + wwwAuthenticate := wwwAuthenticate } @@ -376,11 +493,13 @@ integer seq_nr, template (omit) Contact contact, template (omit) Expires expires, + template (omit) Authorization authorization := omit, template (omit) charstring body := omit) := { requestLine := ts_SIP_ReqLine(REGISTER_E, sip_url_host_port), msgHeader := ts_SIP_msgh_std(call_id, from_addr, to_addr, contact, "REGISTER", seq_nr, via, f_ContentTypeOrOmit(ts_CT_SDP, body), + authorization := authorization, expires := expires), messageBody := body, payload := omit @@ -511,15 +630,16 @@ tr_SIP_Response(template CallidString call_id, template SipAddr from_addr, template SipAddr to_addr, + template (present) Via via := tr_Via_from(?), template Contact contact, template charstring method, template integer status_code, template integer seq_nr := ?, template charstring reason := ?, - template charstring body := ?) := { + template charstring body := *) := { statusLine := tr_SIP_StatusLine(status_code, reason), msgHeader := tr_SIP_msgh_std(call_id, from_addr, to_addr, contact, - tr_Via_from(tr_HostPort(from_addr.addr.nameAddr.addrSpec.hostPort)), + via, method, *, seq_nr), messageBody := body, payload := omit @@ -533,6 +653,7 @@ template SipAddr to_addr, template (present) Via via := tr_Via_from(?), template Contact contact := *, + template (present) WwwAuthenticate wwwAuthenticate := ?, template integer seq_nr := ?, template charstring method := "REGISTER", template integer status_code := 401, @@ -541,11 +662,333 @@ statusLine := tr_SIP_StatusLine(status_code, reason), msgHeader := tr_SIP_msgh_std(call_id, from_addr, to_addr, contact, via, - method, *, seq_nr), + method, *, seq_nr, + wwwAuthenticate := wwwAuthenticate), messageBody := body, payload := omit } +function f_sip_param_find(GenericParam_List li, + template (present) charstring id := ?) +return template (omit) GenericParam { + var integer i; + + for (i := 0; i < lengthof(li); i := i + 1) { + if (not ispresent(li[i])) { + continue; + } + if (match(li[i].id, id)) { + return li[i]; + } + } + return omit; +} + +function f_sip_param_find_or_fail(GenericParam_List li, + template (present) charstring id := ?) +return GenericParam { + var template (omit) GenericParam parameter; + parameter := f_sip_param_find(li, id); + if (istemplatekind(parameter, "omit")) { + Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, + log2str("Param ", id, " not found in ", li)); + } + return valueof(parameter); +} + +function f_sip_param_get_value(GenericParam_List li, + template (present) charstring id := ?) +return template (omit) charstring { + var template (omit) GenericParam parameter; + parameter := f_sip_param_find(li, id); + if (istemplatekind(parameter, "omit")) { + return omit; + } + return parameter.paramValue; +} + +function f_sip_param_get_value_or_fail(GenericParam_List li, + template (present) charstring id := ?) +return template (omit) charstring { + var GenericParam parameter; + parameter := f_sip_param_find_or_fail(li, id); + return parameter.paramValue; +} + +function f_sip_param_get_value_present_or_fail(GenericParam_List li, + template (present) charstring id := ?) +return charstring { + var GenericParam parameter; + parameter := f_sip_param_find_or_fail(li, id); + if (not ispresent(parameter.paramValue)) { + Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, + log2str("Param ", id, " value not present in ", li)); + } + return parameter.paramValue; +} + +function f_sip_param_match_value(GenericParam_List li, + template (present) charstring id := ?, + template charstring exp_paramValue := *) +return boolean { + var template (omit) charstring val; + val := f_sip_param_get_value_or_fail(li, id); + if (istemplatekind(val, "omit")) { + return istemplatekind(val, "omit") or istemplatekind(val, "*"); + } + return match(valueof(val), exp_paramValue); +} + +function f_sip_param_match_value_or_fail(GenericParam_List li, + template (present) charstring id := ?, + template charstring exp_paramValue := *) +{ + var template (omit) charstring val := f_sip_param_get_value_or_fail(li, id); + if (istemplatekind(val, "omit")) { + if (istemplatekind(val, "omit") or istemplatekind(val, "*")) { + return; + } else { + Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, + log2str("Param ", id, " match failed: val ", val, + " vs exp ", exp_paramValue)); + } + } + if (not match(valueof(val), exp_paramValue)) { + Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, + log2str("Param ", id, " match failed: val ", val, + " vs exp ", exp_paramValue)); + } +} + +function f_sip_param_remove(template (omit) GenericParam_List li_tpl, charstring id) +return GenericParam_List { + var integer i; + var GenericParam_List li; + var GenericParam_List new_li := {}; + + if (istemplatekind(li_tpl, "omit")) { + return {}; + } + + li := valueof(li_tpl); + for (i := 0; i < lengthof(li); i := i + 1) { + if (not ispresent(li[i]) or + not match(li[i].id, id)) { + new_li := new_li & {li[i]}; + } + } + return new_li; +} + +function f_sip_param_set(template (omit) GenericParam_List li_tpl, charstring id, charstring val) +return GenericParam_List { + var integer i; + var GenericParam_List li; + var GenericParam_List new_li := {}; + var boolean found := false; + + if (istemplatekind(li_tpl, "omit")) { + return { valueof(ts_Param(id, val)) }; + } + + li := valueof(li_tpl); + for (i := 0; i < lengthof(li); i := i + 1) { + if (not ispresent(li[i]) or + not match(li[i].id, id)) { + new_li := new_li & {li[i]}; + continue; + } + new_li := new_li & { valueof(ts_Param(li[i].id, val)) }; + found := true; + } + + if (not found) { + new_li := new_li & { valueof(ts_Param(id, val)) }; + } + return new_li; +} + +/* Make sure string is quoted. */ +function f_sip_str_quote(template (value) charstring val) return charstring { + var charstring str := valueof(val); + if (lengthof(str) == 0) { + return ""; + } + + if (str[0] != "\"") { + return "\"" & str & "\""; + } + return str; +} + +/* Make sure string is unquoted. + * Similar to unq() in RFC 2617 */ +function f_sip_str_unquote(template (value) charstring val) return charstring { + var charstring str := valueof(val); + var integer len := lengthof(str); + + if (len <= 1) { + return str; + } + + if (str[0] == "\"" and str[len - 1] == "\"") { + return substr(str, 1, len - 2); + } + return str; +} + +/* RFC 2617 3.2.2.2 A1 */ +function f_sip_digest_A1(charstring user, charstring realm, charstring password) return charstring { + + /* RFC 2617 3.2.2.2 A1 */ + var charstring A1 := f_sip_str_unquote(user) & ":" & + f_sip_str_unquote(realm) & ":" & + password; + var charstring digestA1 := f_str_tolower(f_calculateMD5(A1)); + log("A1: md5('", A1, "') = ", digestA1); + return digestA1; +} + +/* RFC 2617 3.2.2.2 A2 */ +function f_sip_digest_A2(charstring method, charstring uri) return charstring { + + var charstring A2 := method & ":" & uri + var charstring digestA2 := f_str_tolower(f_calculateMD5(A2)); + log("A2: md5('", A2, "') = ", digestA2); + return digestA2; +} + +/* RFC 2617 3.2.2.1 Request-Digest */ +function f_sip_digest_RequestDigest(charstring digestA1, charstring nonce, + charstring nc, charstring cnonce, + charstring qop, charstring digestA2) return charstring { + var charstring digest_data := f_sip_str_unquote(nonce) & ":" & + nc & ":" & + cnonce & ":" & + f_sip_str_unquote(qop) & ":" & + digestA2; + var charstring req_digest := f_sip_digest_KD(digestA1, digest_data); + log("Request-Digest: md5('", digestA1, ":", digest_data ,"') = ", req_digest); + return req_digest; +} + +/* RFC 2617 3.2.1 The WWW-Authenticate Response Header + * KD(secret, data) = H(concat(secret, ":", data)) + */ +function f_sip_digest_KD(charstring secret, charstring data) return charstring { + return f_str_tolower(f_calculateMD5(secret & ":" & data)); +} + +/* Digest Auth: RFC 2617 */ +function f_sip_digest_gen_Authorization(WwwAuthenticate www_authenticate, + charstring user, charstring password, + charstring method, charstring uri, + charstring cnonce := "0a4f113b", integer nc_int := 1) return Authorization { + var CommaParam_List digestCln; + var template (value) Authorization authorization; + var template (value) Credentials cred; + var template (omit) GenericParam rx_param; + + digestCln := www_authenticate.challenge[0].digestCln; + + var charstring algorithm; + rx_param := f_sip_param_find(digestCln, "algorithm"); + if (istemplatekind(rx_param, "omit")) { + /* Assume MD5 if not set */ + algorithm := "MD5" + } else { + algorithm := valueof(rx_param.paramValue); + if (f_strstr(algorithm, "MD5") == -1) { + Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, + log2str("Unexpected algorithm: ", algorithm)); + } + } + + var charstring realm := f_sip_param_get_value_present_or_fail(digestCln, "realm"); + var charstring nonce := f_sip_param_get_value_present_or_fail(digestCln, "nonce"); + var charstring opaque := f_sip_param_get_value_present_or_fail(digestCln, "opaque"); + var charstring qop := f_sip_param_get_value_present_or_fail(digestCln, "qop"); + + if (f_strstr(qop, "auth") == -1) { + Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, log2str("Unexpected qop: ", qop)); + } + var charstring selected_qop := "auth"; + + /* RFC 2617 3.2.2.2 A1 */ + var charstring digestA1 := f_sip_digest_A1(user, realm, password); + /* RFC 2617 3.2.2.3 A2 */ + var charstring digestA2 := f_sip_digest_A2(method, uri); + + /* RFC 2617 3.2.2.1 Request-Digest */ + var charstring nc := f_str_tolower(hex2str(int2hex(nc_int, 8))); + var charstring req_digest := f_sip_digest_RequestDigest(digestA1, nonce, + nc, cnonce, + selected_qop, digestA2); + + cred := ts_Credentials_DigestResponseMD5(user, realm, nonce, + uri, req_digest, + opaque, algorithm, selected_qop, cnonce, nc); + + authorization := ts_Authorization(cred); + return valueof(authorization); +} + +/* RFC 2617 3.5 Example */ +function f_sip_digest_selftest() { +/* +The following example assumes that an access-protected document is +being requested from the server via a GET request. The URI of the +document is "http://www.nowhere.org/dir/index.html". Both client and +server know that the username for this document is "Mufasa", and the +password is "Circle Of Life" (with one space between each of the +three words). + +HTTP/1.1 401 Unauthorized +WWW-Authenticate: Digest + realm="testrealm(a)host.com", + qop="auth,auth-int", + nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093", + opaque="5ccc069c403ebaf9f0171e9517f40e41" + +Authorization: Digest username="Mufasa", + realm="testrealm(a)host.com", + nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093", + uri="/dir/index.html", + qop=auth, + nc=00000001, + cnonce="0a4f113b", + response="6629fae49393a05397450978507c4ef1", + opaque="5ccc069c403ebaf9f0171e9517f40e41" +*/ + var template (value) CommaParam_List digestCln := { + ts_Param("realm", f_sip_str_quote("testrealm(a)host.com")), + ts_Param("qop", f_sip_str_quote("auth,auth-int")), + ts_Param("nonce", f_sip_str_quote("dcd98b7102dd2f0e8b11d0f600bfb0c093")), + ts_Param("opaque", f_sip_str_quote("5ccc069c403ebaf9f0171e9517f40e41")) + }; + var template (value) WwwAuthenticate www_authenticate := + ts_WwwAuthenticate( { ts_Challenge_digestCln(digestCln) } ) + + var Authorization authorization := + f_sip_digest_gen_Authorization(valueof(www_authenticate), + "Mufasa", + "Circle Of Life", + "GET", + "/dir/index.html", + cnonce := "0a4f113b", + nc_int := 1); + + var CommaParam_List digestResp := authorization.body.digestResponse; + f_sip_param_match_value_or_fail(digestResp, "realm", f_sip_str_quote("testrealm(a)host.com")); + f_sip_param_match_value_or_fail(digestResp, "nonce", f_sip_str_quote("dcd98b7102dd2f0e8b11d0f600bfb0c093")); + f_sip_param_match_value_or_fail(digestResp, "uri", f_sip_str_quote("/dir/index.html")); + f_sip_param_match_value_or_fail(digestResp, "qop", "auth"); + f_sip_param_match_value_or_fail(digestResp, "nc", "00000001"); + f_sip_param_match_value_or_fail(digestResp, "cnonce", f_sip_str_quote("0a4f113b")); + f_sip_param_match_value_or_fail(digestResp, "response", f_sip_str_quote("6629fae49393a05397450978507c4ef1")); + f_sip_param_match_value_or_fail(digestResp, "opaque", f_sip_str_quote("5ccc069c403ebaf9f0171e9517f40e41")); +} + /* RFC 3261 8.1.1.5: * "The sequence number value MUST be expressible as a 32-bit unsigned integer * and MUST be less than 2**31." @@ -555,4 +998,68 @@ return f_rnd_int(2147483648) } +/* Tags shall have at least 32 bit of randomness */ +function f_sip_rand_tag() return charstring { + var integer rnd_int := f_rnd_int(4294967296); + return hex2str(int2hex(rnd_int, 8)); +} + +/* Generate a "branch" tag value. + * RFC 3261 p.105 section 8: + * "A common way to create this value is to compute a + * cryptographic hash of the To tag, From tag, Call-ID header + * field, the Request-URI of the request received (before + * translation), the topmost Via header, and the sequence number + * from the CSeq header field, in addition to any Proxy-Require + * and Proxy-Authorization header fields that may be present. The + * algorithm used to compute the hash is implementation-dependent, + * but MD5 (RFC 1321 [35]),expressed in hexadecimal, is a reasonable + * choice." + * See also Section 8.1.1.7: + * "The branch ID inserted by an element compliant with this + * specification MUST always begin with the characters "z9hG4bK"." + */ +const charstring sip_magic_cookie := "z9hG4bK"; +function f_sip_gen_branch(charstring tag_to, + charstring tag_from, + charstring tag_call_id, + integer cseq) return charstring { + var charstring str := tag_to & tag_from & tag_call_id & int2str(cseq); + var charstring hash := f_calculateMD5(str); + var charstring branch := sip_magic_cookie & hash; + return branch; +} + +function f_sip_HostPort_to_str(HostPort host_port) return charstring { + var charstring str := ""; + if (ispresent(host_port.host)) { + str := host_port.host; + } + if (ispresent(host_port.portField)) { + str := str & ":" & int2str(host_port.portField); + } + return str; +} + +function f_sip_SipUrl_to_str(SipUrl uri) return charstring { + var charstring str := uri.scheme & f_sip_HostPort_to_str(uri.hostPort); + return str; +} + +function f_sip_NameAddr_to_str(NameAddr naddr) return charstring { + if (ispresent(naddr.displayName)) { + return naddr.displayName & " <" & f_sip_SipUrl_to_str(naddr.addrSpec) & ">"; + } else { + return f_sip_SipUrl_to_str(naddr.addrSpec); + } +} + +function f_sip_SipAddr_to_str(SipAddr sip_addr) return charstring { + if (ischosen(sip_addr.addr.nameAddr)) { + return f_sip_NameAddr_to_str(sip_addr.addr.nameAddr); + } else { + return f_sip_SipUrl_to_str(sip_addr.addr.addrSpecUnion); + } +} + } diff --git a/sip/SIP_Tests.ttcn b/sip/SIP_Tests.ttcn index aee1a82..4c6be1c 100644 --- a/sip/SIP_Tests.ttcn +++ b/sip/SIP_Tests.ttcn @@ -274,7 +274,9 @@ } /* OSC -> SIP */ - as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, *, + as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, + tr_Via_from(tr_HostPort(sip_addr_ext.addr.nameAddr.addrSpec.hostPort)), + *, "INVITE", 100, ?, "Trying", *)); alt { @@ -328,7 +330,9 @@ SIP.clear; /* 180 Ringing should not contain any SDP. */ - as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, *, + as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, + tr_Via_from(tr_HostPort(sip_addr_ext.addr.nameAddr.addrSpec.hostPort)), + *, "INVITE", 180, ?, "Ringing", omit)); /* MSC -> OSC: After MT user has picked up and sent CC CONNECT */ @@ -342,7 +346,9 @@ } /* OSC -> SIP: OSC confirms call establishment to SIP side */ - as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, contact := ?, + as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, + tr_Via_from(tr_HostPort(sip_addr_ext.addr.nameAddr.addrSpec.hostPort)), + contact := ?, method := "INVITE", status_code := 200, seq_nr := ?, reason := "OK", body := expect_sdp_to_sip)); @@ -506,7 +512,9 @@ /* MSC -> OSC: Indicate GSM side release */ MNCC.send(ts_MNCC_REL_ind(cp.mncc_call_id, ts_MNCC_cause(0))); /* OSC -> SIP: Confirmation to SIP side */ - as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, *, + as_SIP_expect_resp(tr_SIP_Response(cp.comp.sip_call_id, sip_addr_ext, sip_addr_gsm, + tr_Via_from(tr_HostPort(sip_addr_ext.addr.nameAddr.addrSpec.hostPort)), + *, "BYE", 200, cp.comp.sip_seq_nr, "OK", omit)); } diff --git a/sip/gen_links.sh b/sip/gen_links.sh index 81c1578..d12e23c 100755 --- a/sip/gen_links.sh +++ b/sip/gen_links.sh @@ -9,7 +9,8 @@ gen_links $DIR $FILES DIR=$BASEDIR/titan.Libraries.TCCUsefulFunctions/src -FILES="TCCInterface_Functions.ttcn TCCConversion_Functions.ttcn TCCConversion.cc TCCInterface.cc TCCInterface_ip.h" +FILES="TCCInterface_Functions.ttcn TCCConversion_Functions.ttcn TCCConversion.cc TCCInterface.cc TCCInterface_ip.h " +FILES+="TCCOpenSecurity_Functions.ttcn TCCOpenSecurity.cc TCCOpenSecurity_Functions.hh " gen_links $DIR $FILES DIR=$BASEDIR/titan.TestPorts.Common_Components.Socket-API/src diff --git a/sip/regen_makefile.sh b/sip/regen_makefile.sh index c7a645d..ecec62b 100755 --- a/sip/regen_makefile.sh +++ b/sip/regen_makefile.sh @@ -17,6 +17,7 @@ SIPmsg_PT.cc TCCConversion.cc TCCInterface.cc + TCCOpenSecurity.cc TELNETasp_PT.cc UD_PT.cc " -- To view, visit https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36532?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-Change-Id: Ib469f1906927a3f246876040086ff115fbf4c032 Gerrit-Change-Number: 36532 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-MessageType: newchange

1 0

2025

2024

2023

2022

gerrit-log April 2024