<p>fixeria has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/c/osmo-hlr/+/14939">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">src/db.c: integrate SQLite3 with talloc allocator<br><br>This change introduces an optional feature that allows to make<br>SQLite3 use talloc for all internal allocations. This would<br>facilitate finding memleaks. OsmoHLR needs to be configured<br>with '--enable-sqlite-talloc'.<br><br> full talloc report on 'OsmoHLR' (total 292168 bytes in 449 blocks)<br> struct osmo_gsup_server contains 162 bytes in 3 blocks (ref 0)<br> ...<br> struct db_context contains 288407 bytes in 420 blocks (ref 0)<br> hlr.db contains 7 bytes in 1 blocks (ref 0)<br> SQLite3 contains 288192 bytes in 418 blocks (ref 0)<br> db.c:95 contains 48 bytes in 1 blocks (ref 0)<br> db.c:95 contains 2 bytes in 1 blocks (ref 0)<br> ...<br><br>Unfortunately, old SQLite3 versions (such as 3.8.2) somehow become<br>unstable and fail to initialize the database with talloc. There is<br>a huge difference in heap usage footprint compared to malloc. At the<br>same time, the recent versions (at least 3.24.0), work just fine.<br><br>Change-Id: Icfe67ed0f063b63e6794f9516da3003d01cf20a7<br>---<br>M configure.ac<br>M src/db.c<br>2 files changed, 85 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-hlr refs/changes/39/14939/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/configure.ac b/configure.ac</span><br><span>index 6694f80..c59f40f 100644</span><br><span>--- a/configure.ac</span><br><span>+++ b/configure.ac</span><br><span>@@ -59,6 +59,20 @@</span><br><span> CPPFLAGS="$CPPFLAGS -fsanitize=address -fsanitize=undefined"</span><br><span> fi</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+AC_ARG_ENABLE([sqlite_talloc],</span><br><span style="color: hsl(120, 100%, 40%);">+ AC_HELP_STRING([--enable-sqlite-talloc],</span><br><span style="color: hsl(120, 100%, 40%);">+ [Configure SQLite3 to use talloc memory allocator [default=no]]),</span><br><span style="color: hsl(120, 100%, 40%);">+ [sqlite_talloc="$enableval"],[sqlite_talloc="no"])</span><br><span style="color: hsl(120, 100%, 40%);">+if test "x$sqlite_talloc" = "xyes" ; then</span><br><span style="color: hsl(120, 100%, 40%);">+ # Older versions of SQLite3 (at least 3.8.2) become unstable with talloc.</span><br><span style="color: hsl(120, 100%, 40%);">+ # Feel free to relax to 3.24.0 > VER > 3.8.2 if it works for you.</span><br><span style="color: hsl(120, 100%, 40%);">+ # FIXME: PKG_CHECK_MODULES() may return cached result here!</span><br><span style="color: hsl(120, 100%, 40%);">+ PKG_CHECK_MODULES(SQLITE3, sqlite3 >= 3.24.0)</span><br><span style="color: hsl(120, 100%, 40%);">+ AC_DEFINE([SQLITE_USE_TALLOC], 1, [Use talloc for SQLite3])</span><br><span style="color: hsl(120, 100%, 40%);">+fi</span><br><span style="color: hsl(120, 100%, 40%);">+AC_MSG_CHECKING([whether to use talloc for SQLite3])</span><br><span style="color: hsl(120, 100%, 40%);">+AC_MSG_RESULT([$sqlite_talloc])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> AC_ARG_ENABLE(werror,</span><br><span> [AS_HELP_STRING(</span><br><span> [--enable-werror],</span><br><span>diff --git a/src/db.c b/src/db.c</span><br><span>index 7de61a2..f0bac2f 100644</span><br><span>--- a/src/db.c</span><br><span>+++ b/src/db.c</span><br><span>@@ -18,6 +18,7 @@</span><br><span> */</span><br><span> </span><br><span> #include <osmocom/core/utils.h></span><br><span style="color: hsl(120, 100%, 40%);">+#include <osmocom/core/talloc.h></span><br><span> </span><br><span> #include <stdbool.h></span><br><span> #include <sqlite3.h></span><br><span>@@ -83,6 +84,63 @@</span><br><span> [DB_STMT_EXISTS_BY_MSISDN] = "SELECT 1 FROM subscriber WHERE msisdn = $msisdn",</span><br><span> };</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+/* Optional feature to make SQLite using talloc */</span><br><span style="color: hsl(120, 100%, 40%);">+#ifdef SQLITE_USE_TALLOC</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+/* Dedicated talloc context for SQLite */</span><br><span style="color: hsl(120, 100%, 40%);">+static void *db_sqlite_ctx = NULL;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+static void *tall_xMalloc(int size)</span><br><span style="color: hsl(120, 100%, 40%);">+{</span><br><span style="color: hsl(120, 100%, 40%);">+ return talloc_size(db_sqlite_ctx, size);</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+static void tall_xFree(void *ptr)</span><br><span style="color: hsl(120, 100%, 40%);">+{</span><br><span style="color: hsl(120, 100%, 40%);">+ talloc_free(ptr);</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+static void *tall_xRealloc(void *ptr, int size)</span><br><span style="color: hsl(120, 100%, 40%);">+{</span><br><span style="color: hsl(120, 100%, 40%);">+ return talloc_realloc_fn(db_sqlite_ctx, ptr, size);</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+static int tall_xSize(void *ptr)</span><br><span style="color: hsl(120, 100%, 40%);">+{</span><br><span style="color: hsl(120, 100%, 40%);">+ return talloc_total_size(ptr);</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+/* DUMMY: talloc doesn't round up the allocation size */</span><br><span style="color: hsl(120, 100%, 40%);">+static int tall_xRoundup(int size) { return size; }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+/* DUMMY: nothing to initialize */</span><br><span style="color: hsl(120, 100%, 40%);">+static int tall_xInit(void *data) { return 0; }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+/* DUMMY: nothing to deinitialize */</span><br><span style="color: hsl(120, 100%, 40%);">+static void tall_xShutdown(void *data) { }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+/* Interface between SQLite and talloc memory allocator */</span><br><span style="color: hsl(120, 100%, 40%);">+static const struct sqlite3_mem_methods tall_sqlite_if = {</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Memory allocation function */</span><br><span style="color: hsl(120, 100%, 40%);">+ .xMalloc = &tall_xMalloc,</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Free a prior allocation */</span><br><span style="color: hsl(120, 100%, 40%);">+ .xFree = &tall_xFree,</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Resize an allocation */</span><br><span style="color: hsl(120, 100%, 40%);">+ .xRealloc = &tall_xRealloc,</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Return the size of an allocation */</span><br><span style="color: hsl(120, 100%, 40%);">+ .xSize = &tall_xSize,</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Round up request size to allocation size */</span><br><span style="color: hsl(120, 100%, 40%);">+ .xRoundup = &tall_xRoundup,</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Initialize the memory allocator */</span><br><span style="color: hsl(120, 100%, 40%);">+ .xInit = &tall_xInit,</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Deinitialize the memory allocator */</span><br><span style="color: hsl(120, 100%, 40%);">+ .xShutdown = &tall_xShutdown,</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Argument to xInit() and xShutdown() */</span><br><span style="color: hsl(120, 100%, 40%);">+ .pAppData = NULL,</span><br><span style="color: hsl(120, 100%, 40%);">+};</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+#endif /* SQLITE_USE_TALLOC */</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> static void sql3_error_log_cb(void *arg, int err_code, const char *msg)</span><br><span> {</span><br><span> LOGP(DDB, LOGL_ERROR, "(%d) %s\n", err_code, msg);</span><br><span>@@ -365,6 +423,19 @@</span><br><span> LOGP(DDB, LOGL_INFO, "Compiled against SQLite3 lib version %s\n", SQLITE_VERSION);</span><br><span> LOGP(DDB, LOGL_INFO, "Running with SQLite3 lib version %s\n", sqlite3_libversion());</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+#ifdef SQLITE_USE_TALLOC</span><br><span style="color: hsl(120, 100%, 40%);">+ db_sqlite_ctx = talloc_named_const(dbc, 0, "SQLite3");</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ /* Configure SQLite3 to use talloc memory allocator */</span><br><span style="color: hsl(120, 100%, 40%);">+ rc = sqlite3_config(SQLITE_CONFIG_MALLOC, &tall_sqlite_if);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (rc == SQLITE_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ LOGP(DDB, LOGL_NOTICE, "SQLite3 is configured to use talloc\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ LOGP(DDB, LOGL_NOTICE, "Failed to configure SQLite3 "</span><br><span style="color: hsl(120, 100%, 40%);">+ "to use talloc, using default memory allocator\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+#endif</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> dbc->fname = talloc_strdup(dbc, fname);</span><br><span> </span><br><span> for (i = 0; i < 0xfffff; i++) {</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/osmo-hlr/+/14939">change 14939</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/osmo-hlr/+/14939"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-hlr </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: Icfe67ed0f063b63e6794f9516da3003d01cf20a7 </div>
<div style="display:none"> Gerrit-Change-Number: 14939 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: fixeria <axilirator@gmail.com> </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>