osmith has uploaded this change for review.
testenv-coredump-helper: remove
After looking at this again, it makes more sense to keep
/proc/sys/kernel/core_pattern as "core" on build4 and build5, which we
use for jenkins jobs instead of installing systemd's coredumpctl and
making all coredumps go through that. The motivation for using
coredumpctl in testenv was that lots of developers may have it already
installed (though I'm not sure about that anymore), and that it gives a
nice API for retrieving the related executable name to a coredump
instead of having to parse the output of e.g. "file".
Using coredumpctl had the disadvantage that coredumps for all other
jobs, such as the master-builds are also caught by that and cannot be
easily placed in the workspace directory. I have started implementing
this by extending testenv-coredump-helper to communicate through a
socket and mounting that socket inside docker containers running
contrib/jenkins.sh (host -> lxc -> docker), and it works, but this
complexity is not useful here.
Instead the related patch will make testenv pick the core files when
coredumpctl is not available, and read the path from "file", making
testenv-coredump-helper obsolete.
Related: osmo-ttcn3-hacks Ia765b01432e4cb4cd36c45de874b966e3ebf55bc
Change-Id: I2959c6e6d97d5691ee2a4ae5d49a351eb5811f10
---
D ansible/roles/testenv-coredump-helper/README.md
D ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.py
D ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.service
D ansible/roles/testenv-coredump-helper/handlers/main.yml
D ansible/roles/testenv-coredump-helper/tasks/main.yml
M ansible/setup-build-host.yml
6 files changed, 0 insertions(+), 210 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/54/41154/1
diff --git a/ansible/roles/testenv-coredump-helper/README.md b/ansible/roles/testenv-coredump-helper/README.md
deleted file mode 100644
index d27f457..0000000
--- a/ansible/roles/testenv-coredump-helper/README.md
+++ /dev/null
@@ -1,49 +0,0 @@
-# testenv-coredump-helper
-
-A simple webserver to make Osmocom related coredumps available in LXCs.
-
-## Architecture
-
-```
-.-----------------------------------------------------------------------------.
-| build host (build4) .--------------------------.|
-| | LXC (deb12build-ansible) ||
-| | ||
-| shell | HTTP ||
-| coredumpctl --------- testenv-coredump-helper ------------- testenv ||
-| |__________________________||
-|_____________________________________________________________________________|
-```
-
-## What this script does
-
-This role installs a systemd service running the script in
-`files/testenv-coredump-helper.py`, which runs a HTTP server on port `8042` of
-the `lxcbr0`'s IP (e.g. `10.0.3.1`) on the build host. The IP is detected
-dynamically as it is random on each build host.
-
-The HTTP server provides one GET endpoint `/core`. When it is requested (by
-testenv running inside the LXC), the script runs `coredumpctl` with parameters
-to check for any coredump within the last three seconds that was created for
-any Osmocom specific program (starting with `osmo-*` or `open5gs-*`).
-
-* If no matching coredump was found, it returns HTTP status code `404`.
-
-* If a matching coredump was found, it returns HTTP status code `200`, sends
- the path to the executable in an `X-Executable-Path` header and sends the
- coredump itself as body.
-
-The coredump and path to the executable are retrieved from `coredumpctl`. The
-coredump is stored in a temporary file for the duration of the transfer.
-
-## Client implementation
-
-The clientside implementation is in `osmo-ttcn3-hacks.git`,
-`_testenv/testenv/coredump.py` in the `get_from_coredumpctl_lxc_host()`
-function.
-
-## Maximum coredump size
-
-The `testenv-coredump-helper` script does not limit the size of the coredump,
-however a maximum size that `systemd-coredump` accepts can be configured in
-`/etc/systemd/coredump.conf`.
diff --git a/ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.py b/ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.py
deleted file mode 100644
index a56bd0f..0000000
--- a/ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.py
+++ /dev/null
@@ -1,112 +0,0 @@
-#!/usr/bin/env python3
-# Copyright 2025 sysmocom - s.f.m.c. GmbH
-# SPDX-License-Identifier: GPL-3.0-or-later
-# Simple webserver to make Osmocom related coredumps available in LXCs. See
-# ../README.md and OS#6769 for details.
-import datetime
-import fnmatch
-import http.server
-import json
-import os
-import shutil
-import signal
-import socket
-import socketserver
-import subprocess
-import sys
-import tempfile
-
-
-NETDEV = "lxcbr0"
-IP_PATTERN = "10.0.*"
-PORT = 8042
-
-
-def find_lxc_ip():
- cmd = ["ip", "-j", "-o", "-4", "addr", "show", "dev", NETDEV]
- p = subprocess.run(cmd, capture_output=True, text=True, check=True)
- ret = json.loads(p.stdout)[0]["addr_info"][0]["local"]
- if not fnmatch.fnmatch(ret, IP_PATTERN):
- print(f"ERROR: IP doesn't match pattern {IP_PATTERN}: {ret}")
- sys.exit(1)
- return ret
-
-
-def executable_is_relevant(exe):
- basename = os.path.basename(exe)
- patterns = [
- "open5gs-*",
- "osmo-*",
- ]
-
- for pattern in patterns:
- if fnmatch.fnmatch(basename, pattern):
- return True
-
- return False
-
-
-class CustomRequestHandler(http.server.SimpleHTTPRequestHandler):
- def do_GET(self):
- if self.path == "/core":
- # Check for any coredump within last 3 seconds
- since = (datetime.datetime.now() - datetime.timedelta(seconds=3)).strftime("%Y-%m-%d %H:%M:%S")
- cmd = ["coredumpctl", "-q", "-S", since, "--json=short", "-n1"]
-
- p = subprocess.run(cmd, capture_output=True, text=True)
- if p.returncode != 0:
- self.send_error(404, "No coredump found")
- return None
-
- # Check if the coredump executable is from osmo-*, open5gs-*, etc.
- coredump = json.loads(p.stdout)[0]
- if not executable_is_relevant(coredump["exe"]):
- self.send_error(404, "No coredump found")
- return None
-
- # Put coredump into a temporary file and return it
- with tempfile.TemporaryDirectory() as tmpdirname:
- core_path = os.path.join(tmpdirname, "core")
- cmd = [
- "coredumpctl",
- "dump",
- "-q",
- "-S",
- since,
- "-o",
- core_path,
- str(coredump["pid"]),
- coredump["exe"],
- ]
- subprocess.run(cmd, stdout=subprocess.DEVNULL, check=True)
-
- with open(core_path, "rb") as f:
- self.send_response(200)
- self.send_header("X-Executable-Path", coredump["exe"])
- self.end_headers()
- self.wfile.write(f.read())
- else:
- self.send_error(404, "File Not Found")
-
-
-def signal_handler(sig, frame):
- sys.exit(0)
-
-
-def main():
- if not shutil.which("coredumpctl"):
- print("ERROR: coredumpctl not found!")
- sys.exit(1)
-
- ip = os.environ.get("LXC_HOST_IP") or find_lxc_ip()
- print(f"Listening on {ip}:{PORT}")
- signal.signal(signal.SIGINT, signal_handler)
- with socketserver.TCPServer((ip, PORT), CustomRequestHandler, False) as httpd:
- httpd.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
- httpd.server_bind()
- httpd.server_activate()
- httpd.serve_forever()
-
-
-if __name__ == "__main__":
- main()
diff --git a/ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.service b/ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.service
deleted file mode 100644
index ef5a851..0000000
--- a/ansible/roles/testenv-coredump-helper/files/testenv-coredump-helper.service
+++ /dev/null
@@ -1,12 +0,0 @@
-[Unit]
-Description=testenv coredump helper
-After=lxc.service
-
-[Service]
-Environment="PYTHONUNBUFFERED=1"
-Type=simple
-Restart=always
-ExecStart=/opt/testenv-coredump-helper/testenv-coredump-helper
-
-[Install]
-WantedBy=multi-user.target
diff --git a/ansible/roles/testenv-coredump-helper/handlers/main.yml b/ansible/roles/testenv-coredump-helper/handlers/main.yml
deleted file mode 100644
index 0f7ef57..0000000
--- a/ansible/roles/testenv-coredump-helper/handlers/main.yml
+++ /dev/null
@@ -1,5 +0,0 @@
----
-- name: restart testenv-coredump-helper
- service:
- name: testenv-coredump-helper
- state: restarted
diff --git a/ansible/roles/testenv-coredump-helper/tasks/main.yml b/ansible/roles/testenv-coredump-helper/tasks/main.yml
deleted file mode 100644
index 9ff2769..0000000
--- a/ansible/roles/testenv-coredump-helper/tasks/main.yml
+++ /dev/null
@@ -1,31 +0,0 @@
----
-- name: install coredumpctl
- apt:
- name:
- - systemd-coredump
- cache_valid_time: 3600
- update_cache: yes
-
-- name: mkdir /opt/testenv-coredump-helper
- ansible.builtin.file:
- path: /opt/testenv-coredump-helper
- state: directory
-
-- name: install testenv-coredump-helper
- ansible.builtin.copy:
- src: testenv-coredump-helper.py
- dest: /opt/testenv-coredump-helper/testenv-coredump-helper
- mode: '0755'
- notify: restart testenv-coredump-helper
-
-- name: install testenv-coredump-helper service
- ansible.builtin.copy:
- src: testenv-coredump-helper.service
- dest: /etc/systemd/system/testenv-coredump-helper.service
- mode: '0644'
- notify: restart testenv-coredump-helper
-
-- name: enable testenv-coredump-helper service
- ansible.builtin.systemd_service:
- name: testenv-coredump-helper
- enabled: true
diff --git a/ansible/setup-build-host.yml b/ansible/setup-build-host.yml
index d1d9874..ed8def5 100644
--- a/ansible/setup-build-host.yml
+++ b/ansible/setup-build-host.yml
@@ -18,4 +18,3 @@
update_cache: yes
roles:
- name: apt-allow-relinfo-change
- - name: testenv-coredump-helper
To view, visit change 41154. To unsubscribe, or for help writing mail filters, visit settings.