June 2022 - gerrit-log - lists.osmocom.org

Change in osmo-msc[master]: Add VTY command for SMS queue trigger holdoff

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28345 ) Change subject: Add VTY command for SMS queue trigger holdoff ...................................................................... Add VTY command for SMS queue trigger holdoff There are various places in the code that retrigger the SMS -> In Memory pending queue mechanism. The default was to trigger the queue in one second, which could result in running the queue every second. This parameter can ease the queue run interval in case one second is causing too much load. Sms _SHOULD_ be delivered anyway without these queue runs, due to for example, MS becoming available or due to being notified by the call back of a new SMS. Change-Id: I43d88342436d654afd6d955e304e7f85fbc4840f --- M include/osmocom/msc/sms_queue.h M src/libmsc/sms_queue.c M src/libmsc/smsc_vty.c 3 files changed, 24 insertions(+), 5 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/45/28345/1 diff --git a/include/osmocom/msc/sms_queue.h b/include/osmocom/msc/sms_queue.h index e4b2a12..9ada525 100644 --- a/include/osmocom/msc/sms_queue.h +++ b/include/osmocom/msc/sms_queue.h @@ -32,6 +32,7 @@ bool delete_expired; /* delete expired SMS from DB? */ unsigned int minimum_validity_mins; /* minimum validity period in minutes */ unsigned int default_validity_mins; /* default validity period in minutes */ + unsigned int trigger_holdoff; /* How often can the queue be re-triggered? */ }; struct sms_queue_config *sms_queue_cfg_alloc(void *ctx); diff --git a/src/libmsc/sms_queue.c b/src/libmsc/sms_queue.c index b6f92e2..eb0118d 100644 --- a/src/libmsc/sms_queue.c +++ b/src/libmsc/sms_queue.c @@ -441,14 +441,21 @@ sms_submit_pending(net->sms_queue); } -/* Trigger a call to sms_submit_pending() in one second */ +/* Trigger a call to sms_submit_pending() in x seconds */ int sms_queue_trigger(struct gsm_sms_queue *smsq) { - LOGP(DLSMS, LOGL_DEBUG, "Triggering SMS queue\n"); - if (osmo_timer_pending(&smsq->push_queue)) - return 0; + struct timeval tv; - osmo_timer_schedule(&smsq->push_queue, 1, 0); + LOGP(DLSMS, LOGL_DEBUG, "Triggering SMS queue\n"); + if (osmo_timer_pending(&smsq->push_queue)) { + if (osmo_timer_remaining(&smsq->push_queue, NULL, &tv) == -1) + return 0; + LOGP(DLSMS, LOGL_INFO, "SMS queue already set to run in %lu seconds.\n", + tv.tv_sec); + return 0; + } + + osmo_timer_schedule(&smsq->push_queue, smsq->cfg->trigger_holdoff, 0); return 0; } @@ -464,6 +471,7 @@ sqcfg->delete_expired = true; sqcfg->default_validity_mins = 7 * 24 * 60; /* 7 days */ sqcfg->minimum_validity_mins = 1; + sqcfg->trigger_holdoff = 1; sqcfg->db_file_path = talloc_strdup(ctx, SMS_DEFAULT_DB_FILE_PATH); return sqcfg; diff --git a/src/libmsc/smsc_vty.c b/src/libmsc/smsc_vty.c index e99b236..0cc4af4 100644 --- a/src/libmsc/smsc_vty.c +++ b/src/libmsc/smsc_vty.c @@ -60,6 +60,14 @@ return CMD_SUCCESS; } +DEFUN(cfg_sms_trigger_holdoff, cfg_sms_trigger_holdoff_cmd, + "queue trigger-holdoff <1-600>", + "SMS Queue\n" "How often can the Queue be re-triggered.\n" "Seconds\n") +{ + smqcfg->trigger_holdoff = atoi(argv[0]); + return CMD_SUCCESS; +} + DEFUN(cfg_sms_queue_max, cfg_sms_queue_max_cmd, "queue max-pending <1-500>", "SMS Queue\n" "SMS to deliver in parallel\n" "Amount\n") @@ -175,6 +183,7 @@ if (smqcfg->db_file_path && strcmp(smqcfg->db_file_path, SMS_DEFAULT_DB_FILE_PATH)) vty_out(vty, " database %s%s", smqcfg->db_file_path, VTY_NEWLINE); + vty_out(vty, " queue trigger-holdoff %u%s", smqcfg->trigger_holdoff, VTY_NEWLINE); vty_out(vty, " queue max-pending %u%s", smqcfg->max_pending, VTY_NEWLINE); vty_out(vty, " queue max-failure %u%s", smqcfg->max_fail, VTY_NEWLINE); @@ -197,6 +206,7 @@ install_element(CONFIG_NODE, &cfg_smsc_cmd); install_node(&smsc_node, config_write_smsc); install_element(SMSC_NODE, &cfg_sms_database_cmd); + install_element(SMSC_NODE, &cfg_sms_trigger_holdoff_cmd); install_element(SMSC_NODE, &cfg_sms_queue_max_cmd); install_element(SMSC_NODE, &cfg_sms_queue_fail_cmd); install_element(SMSC_NODE, &cfg_sms_db_del_delivered_cmd); -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28345 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I43d88342436d654afd6d955e304e7f85fbc4840f Gerrit-Change-Number: 28345 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-msc[master]: Try to send all pending SMS for subscriber in sub_ready_for_sms()

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28343 ) Change subject: Try to send all pending SMS for subscriber in sub_ready_for_sms() ...................................................................... Try to send all pending SMS for subscriber in sub_ready_for_sms() Change-Id: I5a1360a7a18312836b96b6be761a53b7d0b17d02 --- M src/libmsc/sms_queue.c 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/43/28343/1 diff --git a/src/libmsc/sms_queue.c b/src/libmsc/sms_queue.c index 8d4217b..b6f92e2 100644 --- a/src/libmsc/sms_queue.c +++ b/src/libmsc/sms_queue.c @@ -555,6 +555,8 @@ return -1; _gsm411_send_sms(net, vsub, sms); + if (!sms_subscriber_is_pending(net->sms_queue, vsub)) + sms_send_next(vsub); return 0; } -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28343 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I5a1360a7a18312836b96b6be761a53b7d0b17d02 Gerrit-Change-Number: 28343 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-msc[master]: Don't let this osmo-msc operate on a libdbi database

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28344 ) Change subject: Don't let this osmo-msc operate on a libdbi database ...................................................................... Don't let this osmo-msc operate on a libdbi database The Binary format changed when libdbi was removed. If we let osmo-msc run on an unconverted database, the results are unpredictable, certainly undesirable. Change-Id: I887b6a4374b1c83684f4007e9791ae58bba4e8c1 --- M src/libmsc/db.c 1 file changed, 7 insertions(+), 7 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/44/28344/1 diff --git a/src/libmsc/db.c b/src/libmsc/db.c index b310fae..5e5260f 100644 --- a/src/libmsc/db.c +++ b/src/libmsc/db.c @@ -75,7 +75,7 @@ * DATABASE SCHEMA AND MIGRATION ***********************************************************************/ -#define SCHEMA_REVISION "5" +#define SCHEMA_REVISION "6" enum { SCHEMA_META, @@ -497,14 +497,14 @@ LOGP(DDB, LOGL_FATAL, "You must use osmo-msc 1.1.0 to 1.8.0 to upgrade database " "schema from '%u' to '5', sorry\n", db_rev); break; -#if 0 case 5: - if (update_db_revision_5()) - goto error; - - /* The end of waterfall */ + LOGP(DDB, LOGL_FATAL, "The storage format of BINARY data in the database " + "has changed. In order to deliver any pending SMS in your database, " + "you must manually convert your database from " + "'%u' to '6'. Alternatively you can use a fresh, blank database " + "with this version of osmo-msc, sorry.\n", db_rev); + return -1; break; -#endif default: LOGP(DDB, LOGL_FATAL, "Invalid database schema revision '%d'.\n", db_rev); return -EINVAL; -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28344 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I887b6a4374b1c83684f4007e9791ae58bba4e8c1 Gerrit-Change-Number: 28344 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-msc[master]: After RX of an SMPP Submit, send the SMS we just received.

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28342 ) Change subject: After RX of an SMPP Submit, send the SMS we just received. ...................................................................... After RX of an SMPP Submit, send the SMS we just received. Previously, after we have stored the new SMS in the db, we then signalled the queue to run, but this is not good enough. The queue may not pick up this new SMS on the next run, if there are sufficient older messages in the queue. So let's specifically send the message that was just submitted. Change-Id: I9af51ef0d9c2e6c5acc5128efd6195df881b680c --- M include/osmocom/msc/signal.h M src/libmsc/smpp_openbsc.c M src/libmsc/sms_queue.c 3 files changed, 42 insertions(+), 3 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/42/28342/1 diff --git a/include/osmocom/msc/signal.h b/include/osmocom/msc/signal.h index 9153131..77a9bcf 100644 --- a/include/osmocom/msc/signal.h +++ b/include/osmocom/msc/signal.h @@ -86,4 +86,6 @@ struct gsm_sms *sms; /* true when paging was successful */ bool paging_result; + /* the id of the last inserted SMS */ + unsigned long long id; }; diff --git a/src/libmsc/smpp_openbsc.c b/src/libmsc/smpp_openbsc.c index 91666a9..19e3914 100644 --- a/src/libmsc/smpp_openbsc.c +++ b/src/libmsc/smpp_openbsc.c @@ -301,6 +301,8 @@ case 1: /* datagram */ case 3: /* store-and-forward */ rc = db_sms_store(sms); + memset(&sig, 0, sizeof(sig)); + sig.id = sms->id; sms_free(sms); sms = NULL; if (rc < 0) { @@ -312,7 +314,6 @@ strcpy((char *)submit_r->message_id, "msg_id_not_implemented"); LOGP(DLSMS, LOGL_INFO, "SMPP SUBMIT-SM: Stored in DB\n"); - memset(&sig, 0, sizeof(sig)); osmo_signal_dispatch(SS_SMS, S_SMS_SUBMITTED, &sig); rc = 0; break; diff --git a/src/libmsc/sms_queue.c b/src/libmsc/sms_queue.c index 9b42024..8d4217b 100644 --- a/src/libmsc/sms_queue.c +++ b/src/libmsc/sms_queue.c @@ -580,10 +580,46 @@ struct sms_signal_data *sig_sms = signal_data; struct gsm_sms_pending *pending; struct vlr_subscr *vsub; + struct gsm_sms *sms; /* We got a new SMS and maybe should launch the queue again. */ - if (signal == S_SMS_SUBMITTED || signal == S_SMS_SMMA) { - /* TODO: For SMMA we might want to re-use the radio connection. */ + if (signal == S_SMS_SUBMITTED) { + if (sig_sms->id) { + LOGP(DLSMS, LOGL_INFO, "Got Signal for new Sms Submitted with ID [%llu]\n", + sig_sms->id); + sms = db_sms_get(network, sig_sms->id); + if (!sms) + return -1; + if (!sms->receiver || !sms->receiver->lu_complete) { + LOGP(DLSMS, LOGL_DEBUG, + "Subscriber %s%s is not attached, skipping SMS.\n", + sms->receiver ? "" : "MSISDN-", + sms->receiver ? vlr_subscr_msisdn_or_name(sms->receiver) : sms->dst.addr); + return -1; + } + + /* Check somehow it's not already in the pending list */ + if (sms_queue_sms_is_pending(smq, sms->id)) { + sms_free(sms); + return 0; + } + /* Or that this sub is not already pending */ + if (sms_subscriber_is_pending(smq, sms->receiver)) { + sms_free(sms); + return 0; + } + /* Now add this SMS to the Queue for immediate sending. */ + pending = sms_pending_from(smq, sms); + sms_free(sms); + /* Schedule the timer to send this SMS */ + sms_pending_resend(pending); + return 0; + } + } + + if (signal == S_SMS_SMMA) { + /* TODO: For SMMA we might want to re-use the radio connection. + */ sms_queue_trigger(smq); return 0; } -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28342 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I9af51ef0d9c2e6c5acc5128efd6195df881b680c Gerrit-Change-Number: 28342 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-msc[master]: Use unix timestamps in the SMS db

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28339 ) Change subject: Use unix timestamps in the SMS db ...................................................................... Use unix timestamps in the SMS db There's really no need to convert to/from human readable dateformat (YYYY-MM-DD HH:MM:SS) everytime we access a record in the SMS database. sqlite3 has pseudo column types. In fact this means the column type in the database schema is meaningless and anything will be accepted and stored. See (Flexible Typing) [1] It appears that more processing than might be expected is happening internally on such things as date conversions, as the database engine has to actually do verifications on the data. (Is this INT really an INT etc...) [1] https://www.sqlite.org/quirks.html Change-Id: I599a57666da22adf806b01ff095c8672d523a737 --- M src/libmsc/db.c 1 file changed, 6 insertions(+), 6 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/39/28339/1 diff --git a/src/libmsc/db.c b/src/libmsc/db.c index 68cc038..6fe8803 100644 --- a/src/libmsc/db.c +++ b/src/libmsc/db.c @@ -219,10 +219,10 @@ * database query results! */ #define SEL_COLUMNS \ "id," \ - "strftime('%s',created)," \ + "created," \ "sent," \ "deliver_attempts," \ - "strftime('%s', valid_until)," \ + "valid_until," \ "reply_path_req," \ "status_rep_req," \ "is_report," \ @@ -271,7 +271,7 @@ " msg_ref, protocol_id, data_coding_scheme, ud_hdr_ind, user_data, text, " " dest_addr, dest_ton, dest_npi, src_addr, src_ton, src_npi) " "VALUES " - "(datetime($created, 'unixepoch'), datetime($valid_until, 'unixepoch'), " + "($created, $valid_until, " "$reply_path_req, $status_rep_req, $is_report, " "$msg_ref, $protocol_id, $data_coding_scheme, $ud_hdr_ind, $user_data, $text, " "$dest_addr, $dest_ton, $dest_npi, $src_addr, $src_ton, $src_npi)", @@ -302,7 +302,7 @@ " ORDER BY dest_addr, id LIMIT 1", [DB_STMT_SMS_MARK_DELIVERED] = "UPDATE SMS " - " SET sent = datetime('now') " + " SET sent = strftime('%s') " " WHERE id = $id", [DB_STMT_SMS_INC_DELIVER_ATTEMPTS] = "UPDATE SMS " @@ -315,9 +315,9 @@ [DB_STMT_SMS_DEL_EXPIRED] = "DELETE FROM SMS WHERE id = $id", [DB_STMT_SMS_GET_VALID_UNTIL_BY_ID] = - "SELECT strftime('%s', valid_until) FROM SMS WHERE id = $id", + "SELECT valid_until FROM SMS WHERE id = $id", [DB_STMT_SMS_GET_OLDEST_EXPIRED] = - "SELECT id, strftime('%s', valid_until) FROM SMS ORDER BY valid_until LIMIT 1", + "SELECT id, valid_until FROM SMS ORDER BY valid_until LIMIT 1", [DB_STMT_VLR_INSERT] = "REPLACE INTO t_vlr (msisdn) VALUES ($msisdn)", }; -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28339 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I599a57666da22adf806b01ff095c8672d523a737 Gerrit-Change-Number: 28339 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-msc[master]: Turn off secure_delete in sqlite

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28341 ) Change subject: Turn off secure_delete in sqlite ...................................................................... Turn off secure_delete in sqlite libsqlite3 that ships with some distributions may have secure_delete activated by default. This means all database records are overwritten with zeros on DELETE. We don't needs this extra overhead. Change-Id: I9da6499a38096c8df2025bb9d35ec789864b7c5e --- M src/libmsc/db.c 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/41/28341/1 diff --git a/src/libmsc/db.c b/src/libmsc/db.c index ea0fef3..b310fae 100644 --- a/src/libmsc/db.c +++ b/src/libmsc/db.c @@ -585,6 +585,12 @@ sqlite3_free(err_msg); /* non-fatal */ } + rc = sqlite3_exec(g_dbc->db, "PRAGMA secure_delete=0;", 0, 0, &err_msg); + if (rc != SQLITE_OK) { + LOGP(DDB, LOGL_ERROR, "Unable to disable SECURE_DELETE: %s\n", err_msg); + sqlite3_free(err_msg); + /* non-fatal */ + } rc = sqlite3_exec(g_dbc->db, "PRAGMA temp_store=MEMORY;", 0, 0, &err_msg); if (rc != SQLITE_OK) { LOGP(DDB, LOGL_ERROR, "Unable to set TEMP_STORE: %s\n", err_msg); -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28341 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I9da6499a38096c8df2025bb9d35ec789864b7c5e Gerrit-Change-Number: 28341 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-msc[master]: Avoid UPDATE immediately followed by DELETE

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28340 ) Change subject: Avoid UPDATE immediately followed by DELETE ...................................................................... Avoid UPDATE immediately followed by DELETE There is no need to mark an SMS as sent before deleting it. Avoid the extra database overhead involved in doing this. Change-Id: I777155c0f818b979c636bb59953719e472771603 --- M include/osmocom/msc/db.h M include/osmocom/msc/sms_queue.h M src/libmsc/db.c M src/libmsc/gsm_04_11.c M src/libmsc/sms_queue.c 5 files changed, 54 insertions(+), 23 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/40/28340/1 diff --git a/include/osmocom/msc/db.h b/include/osmocom/msc/db.h index a67c050..4294b9f 100644 --- a/include/osmocom/msc/db.h +++ b/include/osmocom/msc/db.h @@ -50,6 +50,7 @@ int db_sms_mark_delivered(struct gsm_sms *sms); int db_sms_inc_deliver_attempts(struct gsm_sms *sms); int db_sms_delete_by_msisdn(const char *msisdn); +int db_sms_delete_message_by_id(unsigned long long sms_id); int db_sms_delete_sent_message_by_id(unsigned long long sms_id); int db_sms_delete_expired_message_by_id(unsigned long long sms_id); void db_sms_delete_oldest_expired_message(void); diff --git a/include/osmocom/msc/sms_queue.h b/include/osmocom/msc/sms_queue.h index a6e6aeb..e4b2a12 100644 --- a/include/osmocom/msc/sms_queue.h +++ b/include/osmocom/msc/sms_queue.h @@ -2,9 +2,26 @@ #define SMS_QUEUE_H #include <stdbool.h> +#include <osmocom/core/timer.h> +#include <osmocom/msc/gsm_subscriber.h> struct gsm_network; -struct gsm_sms_queue; +/* (global) state of the SMS queue. */ +struct gsm_sms_queue { + struct osmo_timer_list resend_pending; /* timer triggering sms_resend_pending() */ + struct osmo_timer_list push_queue; /* timer triggering sms_submit_pending() */ + struct gsm_network *network; + struct llist_head pending_sms; /* list of gsm_sms_pending */ + struct sms_queue_config *cfg; + int pending; /* current number of gsm_sms_pending in RAM */ + + /* last MSISDN for which we read SMS from the database and created gsm_sms_pending records */ + char last_msisdn[GSM23003_MSISDN_MAX_DIGITS+1]; + + /* statistics / counters */ + struct osmo_stat_item_group *statg; + struct rate_ctr_group *ctrg; +}; struct vty; struct sms_queue_config { diff --git a/src/libmsc/db.c b/src/libmsc/db.c index 6fe8803..ea0fef3 100644 --- a/src/libmsc/db.c +++ b/src/libmsc/db.c @@ -54,6 +54,7 @@ DB_STMT_SMS_INC_DELIVER_ATTEMPTS, DB_STMT_SMS_DEL_BY_MSISDN, DB_STMT_SMS_DEL_BY_ID, + DB_STMT_SMS_DEL_SENT_BY_ID, DB_STMT_SMS_DEL_EXPIRED, DB_STMT_SMS_GET_VALID_UNTIL_BY_ID, DB_STMT_SMS_GET_OLDEST_EXPIRED, @@ -310,8 +311,10 @@ " WHERE id = $id", [DB_STMT_SMS_DEL_BY_MSISDN] = "DELETE FROM SMS WHERE src_addr=$src_addr OR dest_addr=$dest_addr", - [DB_STMT_SMS_DEL_BY_ID] = + [DB_STMT_SMS_DEL_SENT_BY_ID] = "DELETE FROM SMS WHERE id = $id AND sent is NOT NULL", + [DB_STMT_SMS_DEL_BY_ID] = + "DELETE FROM SMS WHERE id = $id LIMIT 1", [DB_STMT_SMS_DEL_EXPIRED] = "DELETE FROM SMS WHERE id = $id", [DB_STMT_SMS_GET_VALID_UNTIL_BY_ID] = @@ -986,7 +989,7 @@ return 0; } -int db_sms_delete_sent_message_by_id(unsigned long long sms_id) +int db_sms_delete_message_by_id(unsigned long long sms_id) { OSMO_ASSERT(g_dbc); sqlite3_stmt *stmt = g_dbc->stmt[DB_STMT_SMS_DEL_BY_ID]; @@ -1005,6 +1008,25 @@ return 0; } +int db_sms_delete_sent_message_by_id(unsigned long long sms_id) +{ + OSMO_ASSERT(g_dbc); + sqlite3_stmt *stmt = g_dbc->stmt[DB_STMT_SMS_DEL_SENT_BY_ID]; + int rc; + + db_bind_int64(stmt, "$id", sms_id); + + rc = sqlite3_step(stmt); + if (rc != SQLITE_DONE) { + db_remove_reset(stmt); + LOGP(DDB, LOGL_ERROR, "Failed to delete SMS %llu.\n", sms_id); + return 1; + } + + db_remove_reset(stmt); + return 0; +} + static int delete_expired_sms(unsigned long long sms_id, time_t validity_timestamp) { OSMO_ASSERT(g_dbc); diff --git a/src/libmsc/gsm_04_11.c b/src/libmsc/gsm_04_11.c index 743e272..a9d5bd6 100644 --- a/src/libmsc/gsm_04_11.c +++ b/src/libmsc/gsm_04_11.c @@ -56,6 +56,7 @@ #include <osmocom/msc/msub.h> #include <osmocom/msc/msc_a.h> #include <osmocom/msc/paging.h> +#include <osmocom/msc/sms_queue.h> #ifdef BUILD_SMPP #include "smpp_smsc.h" @@ -845,6 +846,7 @@ struct gsm411_rp_hdr *rph) { struct gsm_sms *sms = trans->sms.sms; + struct gsm_sms_queue *smq = trans->net->sms_queue; /* Acnkowledgement to MT RP_DATA, i.e. the MS confirms it * successfully received a SMS. We can now safely mark it as @@ -861,8 +863,14 @@ GSM411_RP_CAUSE_PROTOCOL_ERR); } - /* mark this SMS as sent in database */ - db_sms_mark_delivered(sms); + /* If we are deleting delivered SMS, then this SMS will very soon be deleted + * in the signal callback. + * In this case, let's avoid the extra database overhead involved in doing + * an UPDATE followed by an immediate DELETE */ + + /* msc_vlr tests will not pass any sms_queue, hence need to check smq != NULL */ + if (smq && smq->cfg->delete_delivered == 0) + db_sms_mark_delivered(sms); send_signal(S_SMS_DELIVERED, trans, sms, 0); diff --git a/src/libmsc/sms_queue.c b/src/libmsc/sms_queue.c index a1cc465..9b42024 100644 --- a/src/libmsc/sms_queue.c +++ b/src/libmsc/sms_queue.c @@ -119,23 +119,6 @@ int resend; /* should we try re-sending it (now) ? */ }; -/* (global) state of the SMS queue. */ -struct gsm_sms_queue { - struct osmo_timer_list resend_pending; /* timer triggering sms_resend_pending() */ - struct osmo_timer_list push_queue; /* timer triggering sms_submit_pending() */ - struct gsm_network *network; - struct llist_head pending_sms; /* list of gsm_sms_pending */ - struct sms_queue_config *cfg; - int pending; /* current number of gsm_sms_pending in RAM */ - - /* last MSISDN for which we read SMS from the database and created gsm_sms_pending records */ - char last_msisdn[GSM23003_MSISDN_MAX_DIGITS+1]; - - /* statistics / counters */ - struct osmo_stat_item_group *statg; - struct rate_ctr_group *ctrg; -}; - /* private wrapper function to make sure we count all SMS delivery attempts */ static void _gsm411_send_sms(struct gsm_network *net, struct vlr_subscr *vsub, struct gsm_sms *sms) { @@ -625,7 +608,7 @@ vsub = pending->vsub; vlr_subscr_get(vsub, __func__); if (smq->cfg->delete_delivered) - db_sms_delete_sent_message_by_id(pending->sms_id); + db_sms_delete_message_by_id(pending->sms_id); sms_pending_free(smq, pending); /* Attempt to send another SMS to this subscriber */ sms_send_next(vsub); -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28340 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I777155c0f818b979c636bb59953719e472771603 Gerrit-Change-Number: 28340 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-msc[master]: Refactor smsq_take_next_sms()

by keith

keith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-msc/+/28338 ) Change subject: Refactor smsq_take_next_sms() ...................................................................... Refactor smsq_take_next_sms() The logic in sms_submit_pending() was: Loop up to 1,000 times around smsq_take_next_sms(): In smsq_take_next_sms(), loop up to 100 times around a DB query where this query will get one SMS record: We then allocate memory for the SMS, calling sqlite3_column_XXX() on each column and also check if the destination addr is attached in the VLR. If not, we discard (free) this SMS and continue the loop. A problem with the above occurs when there are more than 100 SMS in the database for subscribers that are not attached. In this case smsq_take_next_sms() reaches the "sanity" limit of 100 iterations and returns NULL. sms_submit_pending() consequently breaks out of its loop of 1000 iterations and no SMS is added to the pending (in-memory) queue. However, other activity will very likely soon trigger sms_submit_pending() again and we will repepat the above, pointlessly. It appears that these 100 iterations over a sqlite3 SELECT + subsequent sqlite3_column_XXXX() processing is causing osmo-msc to spend a significant of time in libsqlite3. So, rather than getting many SMS from the DB only to then discard them as the subscriber is not attached, let's add a WHERE clause to the query in order to only select SMS for attached subscribers. This way we should never have more than one iteration in smsq_take_next_sms() and we do not do any needless processing in libsqlite3 only to immediately free() the result. Change-Id: I06c418d4919dd8d28c643b7e0a735bc74d66212c --- M include/osmocom/msc/db.h M src/libmsc/db.c M src/libmsc/sms_queue.c M tests/sms_queue/sms_queue_test.c M tests/sms_queue/sms_queue_test.ok 5 files changed, 66 insertions(+), 39 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-msc refs/changes/38/28338/1 diff --git a/include/osmocom/msc/db.h b/include/osmocom/msc/db.h index fc1781b..a67c050 100644 --- a/include/osmocom/msc/db.h +++ b/include/osmocom/msc/db.h @@ -36,6 +36,7 @@ int db_fini(void); /* SMS store-and-forward */ +int db_save_to_temp_vlr(char *msisdn); int db_sms_store(struct gsm_sms *sms); struct gsm_sms *db_sms_get(struct gsm_network *net, unsigned long long id); struct gsm_sms *db_sms_get_next_unsent(struct gsm_network *net, diff --git a/src/libmsc/db.c b/src/libmsc/db.c index 07081d5..68cc038 100644 --- a/src/libmsc/db.c +++ b/src/libmsc/db.c @@ -46,6 +46,7 @@ enum stmt_idx { DB_STMT_SMS_STORE, DB_STMT_SMS_GET, + DB_STMT_SMS_GET_NEXT_UNSENT_RR_FOR_ATTACHED, DB_STMT_SMS_GET_NEXT_UNSENT, DB_STMT_SMS_GET_UNSENT_FOR_SUBSCR, DB_STMT_SMS_GET_NEXT_UNSENT_RR_MSISDN, @@ -56,6 +57,7 @@ DB_STMT_SMS_DEL_EXPIRED, DB_STMT_SMS_GET_VALID_UNTIL_BY_ID, DB_STMT_SMS_GET_OLDEST_EXPIRED, + DB_STMT_VLR_INSERT, _NUM_DB_STMT }; @@ -88,6 +90,7 @@ SCHEMA_RATE, SCHEMA_AUTHKEY, SCHEMA_AUTHLAST, + SCHEMA_TEMP_VLR, }; static const char *create_stmts[] = { @@ -203,6 +206,9 @@ "sres BLOB NOT NULL, " "kc BLOB NOT NULL " ")", + [SCHEMA_TEMP_VLR] = "CREATE TEMPORARY TABLE IF NOT EXISTS t_vlr (" + "msisdn TEXT NOT NULL" + ")" }; /*********************************************************************** @@ -270,6 +276,12 @@ "$msg_ref, $protocol_id, $data_coding_scheme, $ud_hdr_ind, $user_data, $text, " "$dest_addr, $dest_ton, $dest_npi, $src_addr, $src_ton, $src_npi)", [DB_STMT_SMS_GET] = "SELECT " SEL_COLUMNS " FROM SMS WHERE SMS.id = $id", + [DB_STMT_SMS_GET_NEXT_UNSENT_RR_FOR_ATTACHED] = + "SELECT " SEL_COLUMNS " FROM SMS " + "WHERE sent IS NULL " + "AND dest_addr in (SELECT msisdn FROM t_vlr WHERE msisdn > $dest_addr) " + " AND deliver_attempts <= $attempts" + " ORDER BY dest_addr, id LIMIT 1", [DB_STMT_SMS_GET_NEXT_UNSENT] = "SELECT " SEL_COLUMNS " FROM SMS" " WHERE sent IS NULL" @@ -306,6 +318,8 @@ "SELECT strftime('%s', valid_until) FROM SMS WHERE id = $id", [DB_STMT_SMS_GET_OLDEST_EXPIRED] = "SELECT id, strftime('%s', valid_until) FROM SMS ORDER BY valid_until LIMIT 1", + [DB_STMT_VLR_INSERT] = + "REPLACE INTO t_vlr (msisdn) VALUES ($msisdn)", }; /*********************************************************************** @@ -568,6 +582,12 @@ sqlite3_free(err_msg); /* non-fatal */ } + rc = sqlite3_exec(g_dbc->db, "PRAGMA temp_store=MEMORY;", 0, 0, &err_msg); + if (rc != SQLITE_OK) { + LOGP(DDB, LOGL_ERROR, "Unable to set TEMP_STORE: %s\n", err_msg); + sqlite3_free(err_msg); + /* non-fatal */ + } return 0; } @@ -648,6 +668,22 @@ return 0; } +/* Add a connected subscriber MSISDN to the temporary table */ +int db_save_to_temp_vlr(char *msisdn) +{ + OSMO_ASSERT(g_dbc); + sqlite3_stmt *stmt = g_dbc->stmt[DB_STMT_VLR_INSERT]; + int rc; + + db_bind_text(stmt, "$msisdn", (char *)msisdn); + rc = sqlite3_step(stmt); + db_remove_reset(stmt); + if (rc != SQLITE_DONE) { + LOGP(DDB, LOGL_ERROR, "Cannot add to VLR: SQL error: (%d) %s\n", rc, sqlite3_errmsg(g_dbc->db)); + return -EIO; + } + return 0; +} /* store an [unsent] SMS to the database */ int db_sms_store(struct gsm_sms *sms) { @@ -858,7 +894,7 @@ int max_failed) { OSMO_ASSERT(g_dbc); - sqlite3_stmt *stmt = g_dbc->stmt[DB_STMT_SMS_GET_NEXT_UNSENT_RR_MSISDN]; + sqlite3_stmt *stmt = g_dbc->stmt[DB_STMT_SMS_GET_NEXT_UNSENT_RR_FOR_ATTACHED]; struct gsm_sms *sms; int rc; diff --git a/src/libmsc/sms_queue.c b/src/libmsc/sms_queue.c index 9f18f4f..a1cc465 100644 --- a/src/libmsc/sms_queue.c +++ b/src/libmsc/sms_queue.c @@ -294,10 +294,26 @@ struct gsm_sms *sms; int wrapped = 0; int sanity = 100; + unsigned int count = 0; char started_with_msisdn[last_msisdn_buflen]; + struct vlr_subscr *vsub; OSMO_STRLCPY_ARRAY(started_with_msisdn, last_msisdn); + if (net != NULL) { /* db_sms_test passes NULL, so we need to be tolerant */ + llist_for_each_entry(vsub, &net->vlr->subscribers, list) { + if (vsub->msisdn[0] != '\0') { + count++; + DEBUGP(DLSMS, "SMS queue: %s is attached.\n", vsub->msisdn); + db_save_to_temp_vlr(vsub->msisdn); + } + } + if (count == 0) { + DEBUGP(DLSMS, "SMS queue: no subscribers attached.\n"); + return NULL; + } + } + while (wrapped < 2 && (--sanity)) { /* If we wrapped around and passed the first msisdn, we're * through the entire SMS DB; end it. */ @@ -305,27 +321,16 @@ break; sms = db_sms_get_next_unsent_rr_msisdn(net, last_msisdn, 9); + if (!sms) { last_msisdn[0] = '\0'; wrapped++; continue; } - /* Whatever happens, next time around service another recipient - */ + /* Whatever happens, next time around service another recipient */ osmo_strlcpy(last_msisdn, sms->dst.addr, last_msisdn_buflen); - /* Is the subscriber attached? If not, go to next SMS */ - if (!sms->receiver || !sms->receiver->lu_complete) { - LOGP(DLSMS, LOGL_DEBUG, - "Subscriber %s%s is not attached, skipping SMS %llu\n", - sms->receiver ? "" : "MSISDN-", - sms->receiver ? vlr_subscr_msisdn_or_name(sms->receiver) - : sms->dst.addr, sms->id); - sms_free(sms); - continue; - } - return sms; } diff --git a/tests/sms_queue/sms_queue_test.c b/tests/sms_queue/sms_queue_test.c index f681657..20c31eb 100644 --- a/tests/sms_queue/sms_queue_test.c +++ b/tests/sms_queue/sms_queue_test.c @@ -132,6 +132,10 @@ osmo_strlcpy(sms->text, fake_sms_db[i].msisdn, sizeof(sms->text)); if (fake_sms_db[i].vsub_attached) fake_sms_db[i].nr_of_sms--; + if (!sms->receiver) { + sms_free(sms); + return NULL; + } return sms; } diff --git a/tests/sms_queue/sms_queue_test.ok b/tests/sms_queue/sms_queue_test.ok index 146400d..6ccc95f 100644 --- a/tests/sms_queue/sms_queue_test.ok +++ b/tests/sms_queue/sms_queue_test.ok @@ -12,21 +12,17 @@ hitting database: looking for MSISDN > '2222', failed_attempts <= 9 #1: sending SMS to 3333 (last_msisdn='3333') hitting database: looking for MSISDN > '3333', failed_attempts <= 9 - hitting database: looking for MSISDN > '5555', failed_attempts <= 9 hitting database: looking for MSISDN > '', failed_attempts <= 9 #2: sending SMS to 2222 (last_msisdn='2222') hitting database: looking for MSISDN > '2222', failed_attempts <= 9 #3: sending SMS to 3333 (last_msisdn='3333') hitting database: looking for MSISDN > '3333', failed_attempts <= 9 - hitting database: looking for MSISDN > '5555', failed_attempts <= 9 hitting database: looking for MSISDN > '', failed_attempts <= 9 -#4: no SMS to send (last_msisdn='5555') - hitting database: looking for MSISDN > '5555', failed_attempts <= 9 +#4: no SMS to send (last_msisdn='') hitting database: looking for MSISDN > '', failed_attempts <= 9 -#5: no SMS to send (last_msisdn='5555') - hitting database: looking for MSISDN > '5555', failed_attempts <= 9 +#5: no SMS to send (last_msisdn='') hitting database: looking for MSISDN > '', failed_attempts <= 9 -#6: no SMS to send (last_msisdn='5555') +#6: no SMS to send (last_msisdn='') - SMS are pending at various nr failed attempts (cutoff at >= 10) 1111 has 1 SMS pending, 0 failed attempts @@ -60,27 +56,12 @@ 5555 (NOT attached) has 1 SMS pending, 0 failed attempts --> hitting database: looking for MSISDN > '2345', failed_attempts <= 9 - hitting database: looking for MSISDN > '3333', failed_attempts <= 9 - hitting database: looking for MSISDN > '4444', failed_attempts <= 9 - hitting database: looking for MSISDN > '5555', failed_attempts <= 9 hitting database: looking for MSISDN > '', failed_attempts <= 9 - hitting database: looking for MSISDN > '1111', failed_attempts <= 9 - hitting database: looking for MSISDN > '2222', failed_attempts <= 9 -#0: no SMS to send (last_msisdn='3333') - hitting database: looking for MSISDN > '3333', failed_attempts <= 9 - hitting database: looking for MSISDN > '4444', failed_attempts <= 9 - hitting database: looking for MSISDN > '5555', failed_attempts <= 9 +#0: no SMS to send (last_msisdn='') hitting database: looking for MSISDN > '', failed_attempts <= 9 - hitting database: looking for MSISDN > '1111', failed_attempts <= 9 - hitting database: looking for MSISDN > '2222', failed_attempts <= 9 -#1: no SMS to send (last_msisdn='3333') - hitting database: looking for MSISDN > '3333', failed_attempts <= 9 - hitting database: looking for MSISDN > '4444', failed_attempts <= 9 - hitting database: looking for MSISDN > '5555', failed_attempts <= 9 +#1: no SMS to send (last_msisdn='') hitting database: looking for MSISDN > '', failed_attempts <= 9 - hitting database: looking for MSISDN > '1111', failed_attempts <= 9 - hitting database: looking for MSISDN > '2222', failed_attempts <= 9 -#2: no SMS to send (last_msisdn='3333') +#2: no SMS to send (last_msisdn='') - there are no SMS in the DB 1111 has 0 SMS pending, 0 failed attempts -- To view, visit https://gerrit.osmocom.org/c/osmo-msc/+/28338 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-msc Gerrit-Branch: master Gerrit-Change-Id: I06c418d4919dd8d28c643b7e0a735bc74d66212c Gerrit-Change-Number: 28338 Gerrit-PatchSet: 1 Gerrit-Owner: keith <keith(a)rhizomatica.org> Gerrit-MessageType: newchange

3 years, 1 month

1
0
0 0

Change in osmo-bsc[master]: lchan_select: implement dynamic selection mode for assignment

by fixeria

Attention is currently required from: fixeria. Hello Jenkins Builder, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/c/osmo-bsc/+/28328 to look at the new patch set (#2). Change subject: lchan_select: implement dynamic selection mode for assignment ...................................................................... lchan_select: implement dynamic selection mode for assignment Change-Id: I1b7a0d706976b73cc5c30a8714b830811addfe8d Related: SYS#5460 --- M include/osmocom/bsc/bts.h M include/osmocom/bsc/lchan_select.h M src/osmo-bsc/abis_rsl.c M src/osmo-bsc/assignment_fsm.c M src/osmo-bsc/bsc_vty.c M src/osmo-bsc/bts.c M src/osmo-bsc/bts_vty.c M src/osmo-bsc/handover_decision_2.c M src/osmo-bsc/handover_fsm.c M src/osmo-bsc/lchan_select.c M tests/osmo-bsc.vty 11 files changed, 263 insertions(+), 30 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-bsc refs/changes/28/28328/2 -- To view, visit https://gerrit.osmocom.org/c/osmo-bsc/+/28328 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-bsc Gerrit-Branch: master Gerrit-Change-Id: I1b7a0d706976b73cc5c30a8714b830811addfe8d Gerrit-Change-Number: 28328 Gerrit-PatchSet: 2 Gerrit-Owner: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-MessageType: newpatchset

3 years, 1 month

1
0
0 0

Change in osmo-bsc[master]: lchan_select: prepare a list of timeslots once, iterate over it

by fixeria

Attention is currently required from: neels. fixeria has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-bsc/+/28326 ) Change subject: lchan_select: prepare a list of timeslots once, iterate over it ...................................................................... Patch Set 1: (1 comment) File src/osmo-bsc/lchan_select.c: https://gerrit.osmocom.org/c/osmo-bsc/+/28326/comment/3ff23657_d6febd73 PS1, Line 68: for (unsigned int tn = 0; tn < ts_list->num; tn++) { I just realized that this patch also improves interference aware allocation mode (SYS#5313), because now we can pick an lchan with the least interference within the scope of all TRXes. Without this patch we're limited to the scope of a single TRX here. On the other hand, this improvement nullifies the meaning of ascending/descending modes... -- To view, visit https://gerrit.osmocom.org/c/osmo-bsc/+/28326 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-bsc Gerrit-Branch: master Gerrit-Change-Id: I7ccc56856bfd40fd7c63b7437736de60c2b516ff Gerrit-Change-Number: 28326 Gerrit-PatchSet: 1 Gerrit-Owner: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: neels <nhofmeyr(a)sysmocom.de> Gerrit-Attention: neels <nhofmeyr(a)sysmocom.de> Gerrit-Comment-Date: Fri, 17 Jun 2022 22:18:13 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Gerrit-MessageType: comment

3 years, 1 month

1
0
0 0

2025

2024

2023

2022

gerrit-log June 2022