This is merely a historical archive of years 2008-2021, before the migration to mailman3.
A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.
dexter gerrit-no-reply at lists.osmocom.orgdexter has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-bsc/+/25123 )
Change subject: osmo_bsc_main: integrate MGW pooling into osmo-bsc
......................................................................
osmo_bsc_main: integrate MGW pooling into osmo-bsc
Since the libosmo-mgcp-client now supports MGW pooling, lets use this
feature in osmo-bsc. Large RAN installations may benefit from
distributing the RTP voice stream load on multiple media gateways.
Depends: osmo-mgw Icaaba0e470e916eefddfee750b83f5f65291a6b0
Change-Id: I8f33ab2cea04b545c403a6fe479aa963a0fc0d0d
Related: SYS#5091
---
A doc/manuals/chapters/mgwpool.adoc
M doc/manuals/chapters/running.adoc
M doc/manuals/osmobsc-usermanual.adoc
M include/osmocom/bsc/gsm_data.h
M include/osmocom/bsc/vty.h
M src/osmo-bsc/bsc_subscr_conn_fsm.c
M src/osmo-bsc/bsc_vty.c
M src/osmo-bsc/osmo_bsc_main.c
8 files changed, 326 insertions(+), 17 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-bsc refs/changes/23/25123/1
diff --git a/doc/manuals/chapters/mgwpool.adoc b/doc/manuals/chapters/mgwpool.adoc
new file mode 100644
index 0000000..d7f1b1c
--- /dev/null
+++ b/doc/manuals/chapters/mgwpool.adoc
@@ -0,0 +1,240 @@
+[[mgw_pooling]]
+== MGW Pooling
+
+OsmoBSC is able to use a pool of media gateway (MGW) instances since mid 2021.
+The aim of MGW pooling is to evenly distribute the RTP voice stream load across
+multiple MGW instances. This can help to get a better CPU utilization on
+multi-core CPUs.
+
+The load distribution is managed in such a way that when a new call is placed
+the pool will automatically assign the call to the MGW with the lowest load.
+
+MGW pooling is recommended for larger RAN installations. For small networks and
+lab installations the classic method with one MGW per BSC offers sufficient
+performance.
+
+=== VTY configuration
+
+In OsmoBSC the MGW is controlled via an MGCP-Client. The VTY commands to
+configure the MGCP-Client are part of the 'msc' node due to historical reasons.
+Unfortunately this concept does not allow to configure multiple MGCP-Client
+instances as required by MGW pooling. In order to support MGW pooling a new
+'mgw' node has been added under the 'network' node.
+
+=== Legacy configuration files
+
+legacy OsmoBSC configuration files will continue to work but as soon as an MGW
+pool is configured the 'mgw' settings under the 'msc' node will be ignored.
+
+Example configuration with only one MGCP-Client under the 'msc' node:
+----
+msc 0
+ mgw remote-ip 127.0.0.1
+ mgw remote-port 2428
+----
+
+=== MGW pool configuration
+
+to setup an MGW pool, the user must first install multiple OsmoMGW instances so
+that they won’t interfere each other. This can be done using different local
+host ip-addresses or different ports. When OsmoMGW is installed from packages,
+the systemd configuration may also require adjustment.
+
+The VTY settings under the 'mgw' node work the same way as the VTY settings for
+the MGW under the 'msc' node.
+
+Example configuration with two MGCP-Client instances in a pool:
+----
+ mgw 0
+ mgw remote-ip 127.0.0.1
+ mgw remote-port 2432
+ mgw local-ip 127.0.0.1
+ mgw local-port 2431
+ mgw endpoint-domain mgw0
+ mgw 1
+ mgw remote-ip 127.0.0.1
+ mgw remote-port 2430
+ mgw local-ip 127.0.0.1
+ mgw local-port 2429
+ mgw endpoint-domain mgw1
+----
+
+NOTE: When working with multiple MGW / MGCP-Client instances it is higly
+recommended to assign each MGW an individual domain name. Otherwise it can be
+difficult to distinguish the endpoint names in the log.
+
+=== MGW pool management
+
+While it was not possible to change the MGCP-Client configuration under the
+“msc” node at runtime, the pool is fully runtime-manageable. The only limitation
+is that an MGCP-Client can not be restarted or removed as long as it is serving
+calls.
+
+==== MGW pool status
+
+The VTY implenets a 'show mgw-pool' command that lists the currently configured
+MGW pool members, their status and call utilization.
+
+----
+OsmoBSC> show mgw-pool
+% MGW-Pool:
+% MGW 0
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 1
+% MGW 1
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 0
+----
+
+==== Adding an MGW / MGCP-Client to the MGW pool
+
+To add a new MGCP-Client to the pool, the 'mgw' node is used. Like with the
+'bts' or the 'msc' node a reference number is used that usually starts at 0.
+However it is still possible to assign any number from 0-255. The enumeration
+also may contain gaps.
+
+----
+OsmoBSC> enable
+OsmoBSC# configure terminal
+OsmoBSC(config)# network
+OsmoBSC(config-net)# mgw 2
+OsmoBSC(config-mgw)# mgw
+ local-ip local bind to connect to MGW from
+ local-port local port to connect to MGW from
+ remote-ip remote IP address to reach the MGW at
+ remote-port remote port to reach the MGW at
+ endpoint-domain Set the domain name to send in MGCP messages, e.g. the part 'foo' in 'rtpbridge/*@foo'.
+ reset-endpoint Add an endpoint name that should be reset (DLCX) on connect to the reset-endpoint list,e.g. 'rtpbridge/*'
+----
+
+The newly added MGW will immediately appear in the mgc-pool list but it won't
+be used until its configuration finished by restarting it.
+
+----
+% MGW-Pool:
+% MGW 0
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 2
+% MGW 1
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 3
+% MGW 2
+% mgcp-client: disconnected
+% service: unblocked
+% ongoing calls: 0
+----
+
+==== Reconnecting an MGW / MGCP-Client
+
+It may become necessary to reconnect an MGCP-Client. This is the case when the
+VTY configuration was changed at runtime. In order to make the changes effective
+the MGW configuration must be reloaded by reconnecting the MGW connection. Also
+newly created MGW instances require a reconnect once their configuration is
+done.
+
+To reconnect an MGCP-Client use the 'reconnect' VTY command:
+----
+OsmoBSC# mgw 2 reconnect
+----
+
+The mgcp-client status should immediately change to 'connected'. The MGW is now
+ready to be used for new calls.
+
+----
+OsmoBSC# show mgw-pool
+% MGW-Pool:
+% MGW 0
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 2
+% MGW 1
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 3
+% MGW 2
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 0
+----
+
+It should be noted that MGCP a protocol is used via UDP, the connect only
+happens locally to forward the UDP datagrams properly. Also (unless a reset
+endpoint is configured like in the example config above) there will be no
+immediate interaction with the MGW. However, the log should at least confirm
+the the connect worked and the MGCP client has been created successfully.
+
+----
+Mon Aug 2 17:15:00 2021 DLMGCP <0025> mgcp_client.c:788 MGCP client: using endpoint domain '@mgw'
+Mon Aug 2 17:15:00 2021 DLMGCP <0025> mgcp_client.c:908 MGCP GW connection: r=127.0.0.1:2427<->l=127.0.0.1:2727
+----
+
+It is strongly advised to check the activity on the related MGW and to follow
+the log in order to see that the communication between OsmoBSC and the MGW is
+working correctly.
+
+==== Blocking an MGW / MGCP-Client
+
+If it becomes apparent that an MGCP-Client must be restarted or removed from
+the config (maintenance) the operator can put that MGCP-Client into a blocked
+mode. A blocked MGCP-Client will still serve the ongoing calls but it will not
+be picked for the assignment of new calls.
+
+To block an MGCP-Client use the 'block' VTY command:
+----
+OsmoBSC# mgw 2 block
+OsmoBSC# show mgw-pool
+% MGW-Pool:
+% MGW 0
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 11
+% MGW 1
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 12
+% MGW 2
+% mgcp-client: connected
+% service: blocked
+% ongoing calls: 10
+----
+
+When the number of ongoing calls has tapered off, the MGW / MGCP-Client can be
+restarted or removed if necessary.
+
+----
+OsmoBSC# show mgw-pool
+% MGW-Pool:
+% MGW 0
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 15
+% MGW 1
+% mgcp-client: connected
+% service: unblocked
+% ongoing calls: 14
+% MGW 2
+% mgcp-client: connected
+% service: blocked
+% ongoing calls: 0
+----
+
+If the blockade should be reverted, the 'unblock' VTY command can be used in
+the same way to remove the blockade. (Reconnecting will not remove the
+blockade.)
+
+==== Removing an MGW / MGCP-Client
+
+An MGCP-Client is removed from the pool using the 'no mgw' command from the
+configure terminal. The MGCP-Client instance will automatically be terminated
+and the related resources are freed. The only requirement is that there are no
+ongoing calls on the selected instance.
+
+----
+OsmoBSC# configure terminal
+OsmoBSC(config)# network
+OsmoBSC(config-net)# no mgw 2
+----
diff --git a/doc/manuals/chapters/running.adoc b/doc/manuals/chapters/running.adoc
index 327e0bd..26d62f6 100644
--- a/doc/manuals/chapters/running.adoc
+++ b/doc/manuals/chapters/running.adoc
@@ -143,6 +143,9 @@
DLCX to the media gateway. This helps to clear lingering calls from the
media gateway when the OsmoBSC is restarted.
+NOTE: OsmoBSC is also able to handle a pool of media gateways for load
+distribution. See also See also <<mgw_pooling>>.
+
==== Configure Lb to connect to an SMLC
diff --git a/doc/manuals/osmobsc-usermanual.adoc b/doc/manuals/osmobsc-usermanual.adoc
index c7589cc..120fe2c 100644
--- a/doc/manuals/osmobsc-usermanual.adoc
+++ b/doc/manuals/osmobsc-usermanual.adoc
@@ -32,6 +32,8 @@
include::{srcdir}/chapters/mscpool.adoc[]
+include::{srcdir}/chapters/mgwpool.adoc[]
+
include::{srcdir}/chapters/smlc.adoc[]
include::./common/chapters/qos-dscp-pcp.adoc[]
diff --git a/include/osmocom/bsc/gsm_data.h b/include/osmocom/bsc/gsm_data.h
index 5a20e72..8fd88cb 100644
--- a/include/osmocom/bsc/gsm_data.h
+++ b/include/osmocom/bsc/gsm_data.h
@@ -1346,8 +1346,14 @@
struct osmo_timer_list bts_store_uptime_timer;
struct {
+ /* Legacy MGCP client configuration under msc node (also required for
+ * MGCP proxy when sccp-lite is used) */
struct mgcp_client_conf *conf;
- struct mgcp_client *client;
+
+ /* MGW pool, also contains the legacy MGCP client. */
+ struct mgcp_client_pool *mgw_pool;
+
+ /* Timer definitions, the same for all MGW pool members */
struct osmo_tdef *tdefs;
} mgw;
diff --git a/include/osmocom/bsc/vty.h b/include/osmocom/bsc/vty.h
index 9ea9835..70b973d 100644
--- a/include/osmocom/bsc/vty.h
+++ b/include/osmocom/bsc/vty.h
@@ -23,6 +23,7 @@
enum bsc_vty_node {
GSMNET_NODE = _LAST_OSMOVTY_NODE + 1,
+ MGW_NODE,
BTS_NODE,
TRX_NODE,
TS_NODE,
diff --git a/src/osmo-bsc/bsc_subscr_conn_fsm.c b/src/osmo-bsc/bsc_subscr_conn_fsm.c
index afb633d..2256487 100644
--- a/src/osmo-bsc/bsc_subscr_conn_fsm.c
+++ b/src/osmo-bsc/bsc_subscr_conn_fsm.c
@@ -45,6 +45,7 @@
#include <osmocom/bsc/assignment_fsm.h>
#include <osmocom/bsc/codec_pref.h>
#include <osmocom/mgcp_client/mgcp_client_endpoint_fsm.h>
+#include <osmocom/mgcp_client/mgcp_client_pool.h>
#include <osmocom/core/byteswap.h>
#include <osmocom/bsc/lb.h>
#include <osmocom/bsc/lcs_loc_req.h>
@@ -518,39 +519,54 @@
uint16_t msc_assigned_cic, struct gsm_lchan *for_lchan)
{
const char *epname;
+ struct mgcp_client *mgcp_client = NULL;
if (conn->user_plane.mgw_endpoint)
return conn->user_plane.mgw_endpoint;
if (gscon_is_sccplite(conn)) {
+ /* Get MGCP client from pool */
+ mgcp_client = mgcp_client_pool_get(conn->network->mgw.mgw_pool);
+ if (!mgcp_client) {
+ LOGPFSML(conn->fi, LOGL_ERROR,
+ "cannot ensure MGW endpoint -- no MGW configured, check configuration!\n");
+ conn->user_plane.mgw_endpoint = NULL;
+ return NULL;
+ }
+
/* derive endpoint name from CIC on A interface side */
conn->user_plane.mgw_endpoint =
osmo_mgcpc_ep_alloc(conn->fi, GSCON_EV_FORGET_MGW_ENDPOINT,
- conn->network->mgw.client,
+ mgcp_client,
conn->network->mgw.tdefs,
conn->fi->id,
"%x@%s", msc_assigned_cic,
- mgcp_client_endpoint_domain(conn->network->mgw.client));
+ mgcp_client_endpoint_domain(mgcp_client));
LOGPFSML(conn->fi, LOGL_DEBUG, "MGW endpoint name derived from CIC 0x%x: %s\n",
msc_assigned_cic, osmo_mgcpc_ep_name(conn->user_plane.mgw_endpoint));
} else if (gscon_is_aoip(conn)) {
+ /* Get MGCP client from pool */
+ mgcp_client = mgcp_client_pool_get(conn->network->mgw.mgw_pool);
+ if (!mgcp_client) {
+ LOGPFSML(conn->fi, LOGL_ERROR,
+ "cannot ensure MGW endpoint -- no MGW configured, check configuration!\n");
+ conn->user_plane.mgw_endpoint = NULL;
+ return NULL;
+ }
if (is_ipaccess_bts(for_lchan->ts->trx->bts))
/* use dynamic RTPBRIDGE endpoint allocation in MGW */
- epname = mgcp_client_rtpbridge_wildcard(conn->network->mgw.client);
+ epname = mgcp_client_rtpbridge_wildcard(mgcp_client);
else {
- epname = mgcp_client_e1_epname(conn, conn->network->mgw.client, for_lchan->ts->e1_link.e1_nr,
+ epname = mgcp_client_e1_epname(conn, mgcp_client, for_lchan->ts->e1_link.e1_nr,
for_lchan->ts->e1_link.e1_ts, 16,
for_lchan->ts->e1_link.e1_ts_ss*2);
}
conn->user_plane.mgw_endpoint =
- osmo_mgcpc_ep_alloc(conn->fi, GSCON_EV_FORGET_MGW_ENDPOINT,
- conn->network->mgw.client,
- conn->network->mgw.tdefs,
- conn->fi->id,
- "%s", epname);
+ osmo_mgcpc_ep_alloc(conn->fi, GSCON_EV_FORGET_MGW_ENDPOINT, mgcp_client,
+ conn->network->mgw.tdefs, conn->fi->id, "%s", epname);
} else {
LOGPFSML(conn->fi, LOGL_ERROR, "Conn is neither SCCPlite nor AoIP!?\n");
return NULL;
@@ -801,6 +817,12 @@
static void gscon_forget_mgw_endpoint(struct gsm_subscriber_connection *conn)
{
+ struct mgcp_client *mgcp_client;
+
+ /* Put MGCP client back into MGW pool */
+ mgcp_client = osmo_mgcpc_ep_client(conn->user_plane.mgw_endpoint);
+ mgcp_client_pool_put(mgcp_client);
+
conn->user_plane.mgw_endpoint = NULL;
conn->user_plane.mgw_endpoint_ci_msc = NULL;
conn->ho.created_ci_for_msc = NULL;
diff --git a/src/osmo-bsc/bsc_vty.c b/src/osmo-bsc/bsc_vty.c
index 6d10426..d3d769e 100644
--- a/src/osmo-bsc/bsc_vty.c
+++ b/src/osmo-bsc/bsc_vty.c
@@ -37,6 +37,7 @@
#include <osmocom/gsm/gsm0502.h>
#include <osmocom/mgcp_client/mgcp_client_endpoint_fsm.h>
+#include <osmocom/mgcp_client/mgcp_client_pool.h>
#include <osmocom/bsc/vty.h>
#include <osmocom/bsc/gsm_data.h>
@@ -3382,7 +3383,9 @@
install_element(GSMNET_NODE, &cfg_net_nri_null_add_cmd);
install_element(GSMNET_NODE, &cfg_net_nri_null_del_cmd);
+
bts_vty_init();
+ mgcp_client_vty_init_pool(GSMNET_NODE, MGW_NODE, " ", vty_global_gsm_network->mgw.mgw_pool);
install_element(ENABLE_NODE, &drop_bts_cmd);
install_element(ENABLE_NODE, &restart_bts_cmd);
diff --git a/src/osmo-bsc/osmo_bsc_main.c b/src/osmo-bsc/osmo_bsc_main.c
index b97c7a1..d44d3b7 100644
--- a/src/osmo-bsc/osmo_bsc_main.c
+++ b/src/osmo-bsc/osmo_bsc_main.c
@@ -57,6 +57,7 @@
#include <osmocom/vty/cpu_sched_vty.h>
#include <osmocom/mgcp_client/mgcp_client_endpoint_fsm.h>
+#include <osmocom/mgcp_client/mgcp_client_pool.h>
#include <osmocom/abis/abis.h>
#include <osmocom/bsc/abis_om2000.h>
@@ -69,6 +70,7 @@
#include <osmocom/bsc/bts.h>
#include <osmocom/mgcp_client/mgcp_client.h>
+#include <osmocom/mgcp_client/mgcp_client_pool.h>
#include <osmocom/sigtran/xua_msg.h>
@@ -866,6 +868,41 @@
extern void *tall_tqe_ctx;
extern void *tall_ctr_ctx;
+static int bsc_mgw_setup()
+{
+ struct mgcp_client *mgcp_client_legacy;
+ unsigned int pool_members_initalized;
+
+ /* Initalize MGW pool. This initalizes and connects all MGCP clients that are currently configured in
+ * the pool. Adding additional MGCP clients to the pool is possible but the user has to initalize and
+ * start them manually from the VTY. */
+ pool_members_initalized = mgcp_client_connect_pool(bsc_gsmnet->mgw.mgw_pool);
+ if (pool_members_initalized) {
+ LOGP(DNM, LOGL_NOTICE,
+ "MGW pool with %u pool members configured, (ignoring MGW configuration in VTY node 'msc').\n",
+ pool_members_initalized);
+ return 0;
+ }
+
+ /* Initalize and connect a legacy MGCP client. This MGCP client will appear as the one and only pool
+ * member if there is no MGW pool configured. */
+ LOGP(DNM, LOGL_NOTICE, "No MGW pool configured, using MGW configuration in VTY node 'msc'\n");
+ mgcp_client_legacy = mgcp_client_init(bsc_gsmnet, bsc_gsmnet->mgw.conf);
+ if (!mgcp_client_legacy) {
+ LOGP(DNM, LOGL_ERROR, "MGW (legacy) client initalization failed\n");
+ return -EINVAL;
+ }
+ if (mgcp_client_connect(mgcp_client_legacy)) {
+ LOGP(DNM, LOGL_ERROR, "MGW (legacy) connect failed at (%s:%u)\n",
+ bsc_gsmnet->mgw.conf->remote_addr,
+ bsc_gsmnet->mgw.conf->remote_port);
+ return -EINVAL;
+ }
+ mgcp_client_pool_register_legacy(bsc_gsmnet->mgw.mgw_pool, mgcp_client_legacy);
+
+ return 0;
+}
+
int main(int argc, char **argv)
{
struct bsc_msc_data *msc;
@@ -896,6 +933,7 @@
}
bsc_gsmnet->mgw.conf = talloc_zero(bsc_gsmnet, struct mgcp_client_conf);
+ bsc_gsmnet->mgw.mgw_pool = mgcp_client_alloc_pool(bsc_gsmnet);
mgcp_client_conf_init(bsc_gsmnet->mgw.conf);
bts_init();
@@ -999,14 +1037,8 @@
}
}
- bsc_gsmnet->mgw.client = mgcp_client_init(bsc_gsmnet, bsc_gsmnet->mgw.conf);
-
- if (mgcp_client_connect(bsc_gsmnet->mgw.client)) {
- LOGP(DNM, LOGL_ERROR, "MGW connect failed at (%s:%u)\n",
- bsc_gsmnet->mgw.conf->remote_addr,
- bsc_gsmnet->mgw.conf->remote_port);
+ if (bsc_mgw_setup() != 0)
exit(1);
- }
if (osmo_bsc_sigtran_init(&bsc_gsmnet->mscs) != 0) {
LOGP(DNM, LOGL_ERROR, "Failed to initialize sigtran backhaul.\n");
--
To view, visit https://gerrit.osmocom.org/c/osmo-bsc/+/25123
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings
Gerrit-Project: osmo-bsc
Gerrit-Branch: master
Gerrit-Change-Id: I8f33ab2cea04b545c403a6fe479aa963a0fc0d0d
Gerrit-Change-Number: 25123
Gerrit-PatchSet: 1
Gerrit-Owner: dexter <pmaier at sysmocom.de>
Gerrit-MessageType: newchange
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/gerrit-log/attachments/20210803/33116fbc/attachment.htm>