<p>ipse <strong>submitted</strong> this change.</p><p><a href="https://gerrit.osmocom.org/c/osmo-mgw/+/18050">View Change</a></p><div style="white-space:pre-wrap">Approvals:
Jenkins Builder: Verified
laforge: Looks good to me, but someone else must approve
pespin: Looks good to me, approved
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">counters: Implement more useful counters.<br><br>Right now a lot of errors with MGCP processing are invisible in rate<br>counters which makes them difficult to trace or even notice in<br>a production environment. E.g. reaching a limit of MGCP endpoints<br>is completely invisible even though it's a critical opertion alarm.<br><br>Change-Id: I6db68f044255c927dfd534fed880e405ec3ed4d6<br>---<br>M include/osmocom/mgcp/mgcp.h<br>M src/libosmo-mgcp/mgcp_protocol.c<br>M src/libosmo-mgcp/mgcp_vty.c<br>3 files changed, 59 insertions(+), 2 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/include/osmocom/mgcp/mgcp.h b/include/osmocom/mgcp/mgcp.h</span><br><span>index a479fbb..43f480e 100644</span><br><span>--- a/include/osmocom/mgcp/mgcp.h</span><br><span>+++ b/include/osmocom/mgcp/mgcp.h</span><br><span>@@ -119,6 +119,16 @@</span><br><span> #define MGCP_KEEPALIVE_ONCE (-1)</span><br><span> #define MGCP_KEEPALIVE_NEVER 0</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+/* Global MCGP general rate counters */</span><br><span style="color: hsl(120, 100%, 40%);">+enum {</span><br><span style="color: hsl(120, 100%, 40%);">+ MGCP_GENERAL_RX_MSGS_TOTAL,</span><br><span style="color: hsl(120, 100%, 40%);">+ MGCP_GENERAL_RX_MSGS_RETRANSMITTED,</span><br><span style="color: hsl(120, 100%, 40%);">+ MGCP_GENERAL_RX_MSGS_HANDLED,</span><br><span style="color: hsl(120, 100%, 40%);">+ MGCP_GENERAL_RX_MSGS_UNHANDLED,</span><br><span style="color: hsl(120, 100%, 40%);">+ MGCP_GENERAL_RX_FAIL_MSG_PARSE,</span><br><span style="color: hsl(120, 100%, 40%);">+ MGCP_GENERAL_RX_FAIL_NO_ENDPOINT,</span><br><span style="color: hsl(120, 100%, 40%);">+};</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> /* Global MCGP CRCX related rate counters */</span><br><span> enum {</span><br><span> MGCP_CRCX_SUCCESS,</span><br><span>@@ -207,6 +217,8 @@</span><br><span> int vty_number_endpoints;</span><br><span> struct mgcp_endpoint *endpoints;</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ /* Rate counter group which contains stats for generic MGCP events. */</span><br><span style="color: hsl(120, 100%, 40%);">+ struct rate_ctr_group *mgcp_general_ctr_group;</span><br><span> /* Rate counter group which contains stats for processed CRCX commands. */</span><br><span> struct rate_ctr_group *mgcp_crcx_ctr_group;</span><br><span> /* Rate counter group which contains stats for processed MDCX commands. */</span><br><span>diff --git a/src/libosmo-mgcp/mgcp_protocol.c b/src/libosmo-mgcp/mgcp_protocol.c</span><br><span>index 12fea1d..82c10aa 100644</span><br><span>--- a/src/libosmo-mgcp/mgcp_protocol.c</span><br><span>+++ b/src/libosmo-mgcp/mgcp_protocol.c</span><br><span>@@ -53,6 +53,24 @@</span><br><span> #define MGCP_REQUEST(NAME, REQ, DEBUG_NAME) \</span><br><span> { .name = NAME, .handle_request = REQ, .debug_name = DEBUG_NAME },</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+static const struct rate_ctr_desc mgcp_general_ctr_desc[] = {</span><br><span style="color: hsl(120, 100%, 40%);">+ /* rx_msgs = rx_msgs_retransmitted + rx_msgs_handled + rx_msgs_unhandled + err_rx_msg_parse + err_rx_no_endpoint */</span><br><span style="color: hsl(120, 100%, 40%);">+ [MGCP_GENERAL_RX_MSGS_TOTAL] = {"mgcp:rx_msgs", "total number of MGCP messages received."},</span><br><span style="color: hsl(120, 100%, 40%);">+ [MGCP_GENERAL_RX_MSGS_RETRANSMITTED] = {"mgcp:rx_msgs_retransmitted", "number of received retransmissions."},</span><br><span style="color: hsl(120, 100%, 40%);">+ [MGCP_GENERAL_RX_MSGS_HANDLED] = {"mgcp:rx_msgs_handled", "number of handled MGCP messages."},</span><br><span style="color: hsl(120, 100%, 40%);">+ [MGCP_GENERAL_RX_MSGS_UNHANDLED] = {"mgcp:rx_msgs_unhandled", "number of unhandled MGCP messages."},</span><br><span style="color: hsl(120, 100%, 40%);">+ [MGCP_GENERAL_RX_FAIL_MSG_PARSE] = {"mgcp:err_rx_msg_parse", "error parsing MGCP message."},</span><br><span style="color: hsl(120, 100%, 40%);">+ [MGCP_GENERAL_RX_FAIL_NO_ENDPOINT] = {"mgcp:err_rx_no_endpoint", "can't find MGCP endpoint, probably we've used all allocated endpoints."},</span><br><span style="color: hsl(120, 100%, 40%);">+};</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+const static struct rate_ctr_group_desc mgcp_general_ctr_group_desc = {</span><br><span style="color: hsl(120, 100%, 40%);">+ .group_name_prefix = "mgcp",</span><br><span style="color: hsl(120, 100%, 40%);">+ .group_description = "mgcp general statistics",</span><br><span style="color: hsl(120, 100%, 40%);">+ .class_id = OSMO_STATS_CLASS_GLOBAL,</span><br><span style="color: hsl(120, 100%, 40%);">+ .num_ctr = ARRAY_SIZE(mgcp_general_ctr_desc),</span><br><span style="color: hsl(120, 100%, 40%);">+ .ctr_desc = mgcp_general_ctr_desc</span><br><span style="color: hsl(120, 100%, 40%);">+};</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> static const struct rate_ctr_desc mgcp_crcx_ctr_desc[] = {</span><br><span> [MGCP_CRCX_SUCCESS] = {"crcx:success", "CRCX command processed successfully."},</span><br><span> [MGCP_CRCX_FAIL_BAD_ACTION] = {"crcx:bad_action", "bad action in CRCX command."},</span><br><span>@@ -361,24 +379,33 @@</span><br><span> * - or a response (three numbers, space, transaction id) */</span><br><span> struct msgb *mgcp_handle_message(struct mgcp_config *cfg, struct msgb *msg)</span><br><span> {</span><br><span style="color: hsl(120, 100%, 40%);">+ struct mgcp_trunk_config *tcfg = &cfg->trunk;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct rate_ctr_group *rate_ctrs = tcfg->mgcp_general_ctr_group;</span><br><span> struct mgcp_parse_data pdata;</span><br><span> int rc, i, code, handled = 0;</span><br><span> struct msgb *resp = NULL;</span><br><span> char *data;</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ /* Count all messages, even incorect ones */</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_MSGS_TOTAL]);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> if (msgb_l2len(msg) < 4) {</span><br><span> LOGP(DLMGCP, LOGL_ERROR, "msg too short: %d\n", msg->len);</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_FAIL_MSG_PARSE]);</span><br><span> return NULL;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (mgcp_msg_terminate_nul(msg))</span><br><span style="color: hsl(120, 100%, 40%);">+ if (mgcp_msg_terminate_nul(msg)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_FAIL_MSG_PARSE]);</span><br><span> return NULL;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> </span><br><span> mgcp_disp_msg(msg->l2h, msgb_l2len(msg), "Received message");</span><br><span> </span><br><span> /* attempt to treat it as a response */</span><br><span> if (sscanf((const char *)&msg->l2h[0], "%3d %*s", &code) == 1) {</span><br><span> LOGP(DLMGCP, LOGL_DEBUG, "Response: Code: %d\n", code);</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_FAIL_MSG_PARSE]);</span><br><span> return NULL;</span><br><span> }</span><br><span> </span><br><span>@@ -394,12 +421,14 @@</span><br><span> if (pdata.endp && pdata.trans</span><br><span> && pdata.endp->last_trans</span><br><span> && strcmp(pdata.endp->last_trans, pdata.trans) == 0) {</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_MSGS_RETRANSMITTED]);</span><br><span> return do_retransmission(pdata.endp);</span><br><span> }</span><br><span> </span><br><span> /* check for general parser failure */</span><br><span> if (rc < 0) {</span><br><span> LOGP(DLMGCP, LOGL_NOTICE, "%s: failed to find the endpoint\n", msg->l2h);</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_FAIL_NO_ENDPOINT]);</span><br><span> return create_err_response(NULL, -rc, (const char *) msg->l2h, pdata.trans);</span><br><span> }</span><br><span> </span><br><span>@@ -413,9 +442,13 @@</span><br><span> }</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (!handled)</span><br><span style="color: hsl(120, 100%, 40%);">+ if (handled) {</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_MSGS_HANDLED]);</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ rate_ctr_inc(&rate_ctrs->ctr[MGCP_GENERAL_RX_MSGS_UNHANDLED]);</span><br><span> LOGP(DLMGCP, LOGL_NOTICE, "MSG with type: '%.4s' not handled\n",</span><br><span> &msg->l2h[0]);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> </span><br><span> return resp;</span><br><span> }</span><br><span>@@ -1556,11 +1589,19 @@</span><br><span> /* FIXME: Each new rate counter group requires a unique index. At the</span><br><span> * moment we generate an index using a counter, but perhaps there is</span><br><span> * a better way of assigning indices? */</span><br><span style="color: hsl(120, 100%, 40%);">+ static unsigned int general_rate_ctr_index = 0;</span><br><span> static unsigned int crcx_rate_ctr_index = 0;</span><br><span> static unsigned int mdcx_rate_ctr_index = 0;</span><br><span> static unsigned int dlcx_rate_ctr_index = 0;</span><br><span> static unsigned int all_rtp_conn_rate_ctr_index = 0;</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ if (trunk->mgcp_general_ctr_group == NULL) {</span><br><span style="color: hsl(120, 100%, 40%);">+ trunk->mgcp_general_ctr_group = rate_ctr_group_alloc(ctx, &mgcp_general_ctr_group_desc, general_rate_ctr_index);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!trunk->mgcp_general_ctr_group)</span><br><span style="color: hsl(120, 100%, 40%);">+ return -1;</span><br><span style="color: hsl(120, 100%, 40%);">+ talloc_set_destructor(trunk->mgcp_general_ctr_group, free_rate_counter_group);</span><br><span style="color: hsl(120, 100%, 40%);">+ general_rate_ctr_index++;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> if (trunk->mgcp_crcx_ctr_group == NULL) {</span><br><span> trunk->mgcp_crcx_ctr_group = rate_ctr_group_alloc(ctx, &mgcp_crcx_ctr_group_desc, crcx_rate_ctr_index);</span><br><span> if (!trunk->mgcp_crcx_ctr_group)</span><br><span>diff --git a/src/libosmo-mgcp/mgcp_vty.c b/src/libosmo-mgcp/mgcp_vty.c</span><br><span>index fa175e1..76d674f 100644</span><br><span>--- a/src/libosmo-mgcp/mgcp_vty.c</span><br><span>+++ b/src/libosmo-mgcp/mgcp_vty.c</span><br><span>@@ -266,6 +266,10 @@</span><br><span> vty_out(vty, "%s", VTY_NEWLINE);</span><br><span> vty_out(vty, "Rate counters:%s", VTY_NEWLINE);</span><br><span> }</span><br><span style="color: hsl(120, 100%, 40%);">+ if (show_stats && cfg->mgcp_general_ctr_group) {</span><br><span style="color: hsl(120, 100%, 40%);">+ vty_out(vty, " %s:%s", cfg->mgcp_general_ctr_group->desc->group_description, VTY_NEWLINE);</span><br><span style="color: hsl(120, 100%, 40%);">+ vty_out_rate_ctr_group_fmt(vty, " %25n: %10c (%S/s %M/m %H/h %D/d) %d", cfg->mgcp_general_ctr_group);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> if (show_stats && cfg->mgcp_crcx_ctr_group) {</span><br><span> vty_out(vty, " %s:%s", cfg->mgcp_crcx_ctr_group->desc->group_description, VTY_NEWLINE);</span><br><span> vty_out_rate_ctr_group_fmt(vty, " %25n: %10c (%S/s %M/m %H/h %D/d) %d", cfg->mgcp_crcx_ctr_group);</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-mgw/+/18050">change 18050</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-mgw/+/18050"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-mgw </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I6db68f044255c927dfd534fed880e405ec3ed4d6 </div>
<div style="display:none"> Gerrit-Change-Number: 18050 </div>
<div style="display:none"> Gerrit-PatchSet: 5 </div>
<div style="display:none"> Gerrit-Owner: ipse <Alexander.Chemeris@gmail.com> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins Builder </div>
<div style="display:none"> Gerrit-Reviewer: ipse <Alexander.Chemeris@gmail.com> </div>
<div style="display:none"> Gerrit-Reviewer: laforge <laforge@osmocom.org> </div>
<div style="display:none"> Gerrit-Reviewer: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>