osmith has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-hlr/+/43010?usp=email )
Change subject: systemd: run db-upgrade script in ExecStartPre ......................................................................
systemd: run db-upgrade script in ExecStartPre
We have seen in CI jobs that calling the DB upgrade script from debian/postinst does not work reliably.
When doing a fresh install of osmo-hlr, it usually works as expected:
Setting up osmo-hlr (1.9.2) ... osmo-hlr-post-upgrade: nothing to do (no existing database)
But sometimes:
Setting up osmo-hlr (1.9.2) ... osmo-hlr-post-upgrade: database upgrade is required osmo-hlr-post-upgrade: stopping osmo-hlr service osmo-hlr-post-upgrade: creating backup: /var/lib/osmocom/hlr.db.20260706135658.bak osmo-hlr-post-upgrade: performing database upgrade <0001> db.c:600 using database: /var/lib/osmocom/hlr.db <0001> db.c:103 (283) recovered 4 frames from WAL file /var/lib/osmocom/hlr.db-wal <0001> db.c:679 Database '/var/lib/osmocom/hlr.db' has HLR DB schema version 0 <0001> db.c:103 (1) duplicate column name: last_lu_seen in "ALTER TABLE subscriber ADD COLUMN last_lu_seen TIMESTAMP default NULL" <0001> db.c:248 Unable to prepare SQL statement 'ALTER TABLE subscriber ADD COLUMN last_lu_seen TIMESTAMP default NULL' <0001> db.c:318 Unable to update HLR database schema to version 1 <0001> db.c:685 Failed to upgrade HLR DB schema to version 1: (rc=1) duplicate column name: last_lu_seen <0000> hlr_db_tool.c:509 Error opening database
It seems that osmo-hlr can get started before the osmo-hlr-post-upgrade script runs, then starts to create the database with the current schema, and gets stopped before it is done, leading to the failure above.
Rename the script to osmo-hlr-db-upgrade.sh, remove logic for stopping and starting the service and let systemd call it in ExecStartPre. That way we are sure that it runs *before* osmo-hlr starts up.
Related: SYS#8125 Change-Id: If13f4c8b49397e6630956e92a7825843c7d55ebd --- M contrib/Makefile.am R contrib/osmo-hlr-db-upgrade.sh M contrib/systemd/osmo-hlr.service M debian/osmo-hlr.install M debian/postinst 5 files changed, 6 insertions(+), 41 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-hlr refs/changes/10/43010/1
diff --git a/contrib/Makefile.am b/contrib/Makefile.am index 37c4aa4..fa3f1a2 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -3,12 +3,12 @@ dgsm \ $(NULL)
-EXTRA_DIST = osmo-hlr-post-upgrade.sh +EXTRA_DIST = osmo-hlr-db-upgrade.sh
install-data-hook: - install -Dm755 $(srcdir)/osmo-hlr-post-upgrade.sh \ + install -Dm755 $(srcdir)/osmo-hlr-db-upgrade.sh \ -t $(DESTDIR)$(datadir)/osmocom/
uninstall-hook: @$(PRE_UNINSTALL) - $(RM) $(DESTDIR)$(datadir)/osmocom/osmo-hlr-post-upgrade.sh + $(RM) $(DESTDIR)$(datadir)/osmocom/osmo-hlr-db-upgrade.sh diff --git a/contrib/osmo-hlr-post-upgrade.sh b/contrib/osmo-hlr-db-upgrade.sh similarity index 68% rename from contrib/osmo-hlr-post-upgrade.sh rename to contrib/osmo-hlr-db-upgrade.sh index eaef417..ff1eef8 100644 --- a/contrib/osmo-hlr-post-upgrade.sh +++ b/contrib/osmo-hlr-db-upgrade.sh @@ -6,10 +6,9 @@ # upgrade the database scheme if required.
DB="/var/lib/osmocom/hlr.db" -IS_ACTIVE=0
msg() { - echo "osmo-hlr-post-upgrade: $@" + echo "osmo-hlr-db-upgrade: $@" }
err() { @@ -36,27 +35,6 @@ msg "database upgrade is required" }
-stop_service() { - if systemctl is-active -q osmo-hlr; then - IS_ACTIVE=1 - msg "stopping osmo-hlr service" - systemctl stop osmo-hlr - - # Verify that it stopped - for i in $(seq 1 100); do - if ! systemctl is-active -q osmo-hlr; then - return - fi - sleep 0.1 - done - - err "failed to stop osmo-hlr service" - exit 1 - else - msg "osmo-hlr service is not running" - fi -} - create_backup() { backup="$DB.$(date +%Y%m%d%H%M%S).bak" msg "creating backup: $backup" @@ -81,15 +59,6 @@ msg "database upgrade successful" }
-start_service() { - if [ "$IS_ACTIVE" = "1" ]; then - msg "starting osmo-hlr service" - systemctl start osmo-hlr - fi -} - check_upgrade_required -stop_service create_backup upgrade -start_service diff --git a/contrib/systemd/osmo-hlr.service b/contrib/systemd/osmo-hlr.service index 2822f13..a28a989 100644 --- a/contrib/systemd/osmo-hlr.service +++ b/contrib/systemd/osmo-hlr.service @@ -11,6 +11,7 @@ WorkingDirectory=%S/osmocom User=osmocom Group=osmocom +ExecStartPre=/usr/share/osmocom/osmo-hlr-db-upgrade.sh ExecStart=/usr/bin/osmo-hlr -c /etc/osmocom/osmo-hlr.cfg -l /var/lib/osmocom/hlr.db RestartSec=2 ProtectHome=true diff --git a/debian/osmo-hlr.install b/debian/osmo-hlr.install index 7c8c398..bb13183 100644 --- a/debian/osmo-hlr.install +++ b/debian/osmo-hlr.install @@ -6,4 +6,4 @@ /usr/share/doc/osmo-hlr/sql/hlr_data.sql /usr/share/doc/osmo-hlr/examples/osmo-hlr.cfg /usr/share/doc/osmo-hlr/examples/osmo-hlr-dgsm.cfg -/usr/share/osmocom/osmo-hlr-post-upgrade.sh +/usr/share/osmocom/osmo-hlr-db-upgrade.sh diff --git a/debian/postinst b/debian/postinst index 34fb79d..327ba00 100755 --- a/debian/postinst +++ b/debian/postinst @@ -1,9 +1,4 @@ #!/bin/sh -e -# Debian's postinst script is called on both installation and upgrade. Call the -# post-upgrade script in both cases, it won't do anything if there is nothing -# to do. -/usr/share/osmocom/osmo-hlr-post-upgrade.sh - case "$1" in configure) # Create the osmocom group and user (if it doesn't exist yet)