January 2025 - gerrit-log - lists.osmocom.org

[S] Change in osmo-pcap[master]: server: Avoid lseek syscall on every packet write to pcap file

by pespin

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-pcap/+/39303?usp=email ) Change subject: server: Avoid lseek syscall on every packet write to pcap file ...................................................................... server: Avoid lseek syscall on every packet write to pcap file Instead of requesting the current size to the OS, keep track of the amount of bytes stored in conn->wr_offset, and check agains that value when we want to write new data to the file. This should remove quite a lot of oeverhead when lots of traffic is being handled. Change-Id: I437eaca982d65d0f06e7d24863875f85267f0e44 --- M include/osmo-pcap/osmo_pcap_server.h M src/osmo_server_core.c 2 files changed, 22 insertions(+), 16 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/03/39303/1 diff --git a/include/osmo-pcap/osmo_pcap_server.h b/include/osmo-pcap/osmo_pcap_server.h index 9a44fce..5d21c64 100644 --- a/include/osmo-pcap/osmo_pcap_server.h +++ b/include/osmo-pcap/osmo_pcap_server.h @@ -87,6 +87,8 @@ int local_fd; /* canonicalized absolute pathname of pcap file we write to */ char *curr_filename; + /* Current write offset of the file we write to (local_fd) */ + off_t wr_offset; /* pcap stuff */ enum osmo_pcap_fmt file_fmt; diff --git a/src/osmo_server_core.c b/src/osmo_server_core.c index b35d2b7..cc834a2 100644 --- a/src/osmo_server_core.c +++ b/src/osmo_server_core.c @@ -308,12 +308,17 @@ talloc_free(curr_filename_cpy_dname); } +static void close_local_fd(struct osmo_pcap_conn *conn) +{ + close(conn->local_fd); + conn->local_fd = -1; + conn->wr_offset = 0; +} + void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn) { - if (conn->local_fd >= 0) { - close(conn->local_fd); - conn->local_fd = -1; - } + if (conn->local_fd >= 0) + close_local_fd(conn); move_completed_trace_if_needed(conn); @@ -342,7 +347,7 @@ /* Update conn->last_write if needed. This field is used to keep the last time * period where we wrote to the pcap file. Once a new write period (based on * rotation VTY config) is detected, the pcap file we write to is rotated. */ -static void update_last_write(struct osmo_pcap_conn *conn, time_t now) +static void update_last_write(struct osmo_pcap_conn *conn, time_t now, size_t len) { time_t last = mktime(&conn->last_write); @@ -352,6 +357,8 @@ * using the current one. */ if (now > last) localtime_r(&now, &conn->last_write); + + conn->wr_offset += len; } void osmo_pcap_conn_restart_trace(struct osmo_pcap_conn *conn) @@ -365,7 +372,7 @@ /* omit any storing/creation of the file */ if (!conn->store) { - update_last_write(conn, now); + update_last_write(conn, now, 0); talloc_free(conn->curr_filename); conn->curr_filename = NULL; return; @@ -395,28 +402,25 @@ conn->curr_filename, strerror(errno)); return; } + conn->wr_offset = 0; rc = write(conn->local_fd, conn->file_hdr, conn->file_hdr_len); if (rc != conn->file_hdr_len) { LOGP(DSERVER, LOGL_ERROR, "Failed to write the header: %d\n", errno); - close(conn->local_fd); - conn->local_fd = -1; + close_local_fd(conn); return; } - - update_last_write(conn, now); + update_last_write(conn, now, rc); } /* Returns true if pcap was re-opened */ static bool check_restart_pcap_max_size(struct osmo_pcap_conn *conn, size_t data_len) { - off_t cur; - if (!pcap_server->max_size_enabled) return false; - cur = lseek(conn->local_fd, 0, SEEK_CUR); - if (cur + data_len <= conn->server->max_size) + if (conn->wr_offset + data_len <= conn->server->max_size) return false; + LOGP(DSERVER, LOGL_NOTICE, "Rolling over file for %s (max-size)\n", conn->name); osmo_pcap_conn_restart_trace(conn); return true; @@ -554,7 +558,7 @@ zmq_send_client_data(conn, data, len); if (!conn->store) { - update_last_write(conn, now); + update_last_write(conn, now, 0); return 0; } @@ -567,11 +571,11 @@ if (!check_restart_pcap_max_size(conn, len)) check_restart_pcap_localtime(conn, now); - update_last_write(conn, now); rc = write(conn->local_fd, data, len); if (rc != len) { LOGP(DSERVER, LOGL_ERROR, "Failed to write for %s\n", conn->name); return -1; } + update_last_write(conn, now, rc); return 0; } -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39303?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I437eaca982d65d0f06e7d24863875f85267f0e44 Gerrit-Change-Number: 39303 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[XL] Change in osmo-pcap[master]: server: Split network code into its own file

by pespin

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-pcap/+/39302?usp=email ) Change subject: server: Split network code into its own file ...................................................................... server: Split network code into its own file Separate code related to network socket handling into its own separate file, as already done in osmo-pcap-client. This allows easily finding out which code is operating on the socket and which code is operating on the pcap file being locally stored. This will make modifying subsystems easier too. Change-Id: I41d6e0398605becbea109a7bb5aac8981d65ab1b --- M include/osmo-pcap/osmo_pcap_server.h M src/Makefile.am A src/osmo_server_core.c M src/osmo_server_network.c M tests/rotate_localtime/Makefile.am 5 files changed, 592 insertions(+), 531 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/02/39302/1 diff --git a/include/osmo-pcap/osmo_pcap_server.h b/include/osmo-pcap/osmo_pcap_server.h index d91ffb3..9a44fce 100644 --- a/include/osmo-pcap/osmo_pcap_server.h +++ b/include/osmo-pcap/osmo_pcap_server.h @@ -168,5 +168,9 @@ const char *name); void osmo_pcap_conn_free(struct osmo_pcap_conn *conn); void vty_server_init(void); -void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn); void osmo_pcap_conn_close(struct osmo_pcap_conn *conn); +int osmo_pcap_conn_process_data(struct osmo_pcap_conn *conn, const uint8_t *data, size_t len); +void osmo_pcap_conn_restart_trace(struct osmo_pcap_conn *conn); +void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn); +void osmo_pcap_conn_event(struct osmo_pcap_conn *conn, + const char *event, const char *data); diff --git a/src/Makefile.am b/src/Makefile.am index 0fb16a2..c3576d1 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -36,9 +36,10 @@ osmo_server_main.c \ osmo_common.c \ osmo_pcap_file.c \ - osmo_server_vty.c \ + osmo_server_core.c \ osmo_server_network.c \ osmo_server_stats.c \ + osmo_server_vty.c \ osmo_tls.c \ $(NULL) diff --git a/src/osmo_server_core.c b/src/osmo_server_core.c new file mode 100644 index 0000000..b35d2b7 --- /dev/null +++ b/src/osmo_server_core.c @@ -0,0 +1,577 @@ +/* + * osmo-pcap-server code + * + * (C) 2024 by sysmocom - s.f.m.c. GmbH <info(a)sysmocom.de> + * (C) 2011-2016 by Holger Hans Peter Freyther <holger(a)moiji-mobile.com> + * (C) 2011 by On-Waves + * All Rights Reserved + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * + */ + +#include <osmo-pcap/osmo_pcap_server.h> +#include <osmo-pcap/common.h> +#include <osmo-pcap/wireformat.h> + +#include <osmocom/core/socket.h> +#include <osmocom/core/talloc.h> +#include <osmocom/core/rate_ctr.h> + +#include <arpa/inet.h> +#include <sys/socket.h> +#include <sys/types.h> + +#include <zmq.h> + +#include <fcntl.h> +#include <errno.h> +#include <string.h> +#include <unistd.h> +#include <limits.h> +#include <stdlib.h> +#include <inttypes.h> +#include <libgen.h> + +static void pcap_zmq_send(void *publ, const void *data, size_t len, int flags) +{ + int rc; + zmq_msg_t msg; + + rc = zmq_msg_init_size(&msg, len); + if (rc != 0) { + /* sigh.. we said SNDMORE but can't... */ + LOGP(DSERVER, LOGL_ERROR, "Failed to init rc=%d errno=%d/%s\n", + rc, errno, strerror(errno)); + return; + } + memcpy(zmq_msg_data(&msg), data, len); + rc = zmq_msg_send(&msg, publ, flags); + if (rc == -1) { + /* is the zmq_msg now owned? leak??? */ + LOGP(DSERVER, LOGL_ERROR, "Failed to send data rc=%d errno=%d/%s\n", + rc, errno, strerror(errno)); + return; + } +} + +void osmo_pcap_conn_event(struct osmo_pcap_conn *conn, + const char *event, const char *data) +{ + char *event_name; + + if (!conn->server->zmq_publ) + return; + + /* + * This multi-part support is insane... so if we lose the first + * or the last part of the multipart message stuff is going out + * of sync. *great* As we can't do anything about it right now + * just close the eyese and send it. + */ + event_name = talloc_asprintf(conn, "event.v1.%s.%s", + event, conn->name); + pcap_zmq_send(conn->server->zmq_publ, + event_name, strlen(event_name), + data ? ZMQ_SNDMORE : 0); + talloc_free(event_name); + if (data) + pcap_zmq_send(conn->server->zmq_publ, data, strlen(data), 0); +} + +static void zmq_send_client_data(struct osmo_pcap_conn *conn, + const uint8_t *data, size_t len) +{ + char *event_name; + + if (!conn->server->zmq_publ) + return; + + /* + * This multi-part support is insane... so if we lose the first + * or the last part of the multipart message stuff is going out + * of sync. *great* As we can't do anything about it right now + * just close the eyes and send it. + */ + event_name = talloc_asprintf(conn, "data.v1.%s", conn->name); + pcap_zmq_send(conn->server->zmq_publ, event_name, strlen(event_name), ZMQ_SNDMORE); + talloc_free(event_name); + + pcap_zmq_send(conn->server->zmq_publ, + conn->file_hdr, conn->file_hdr_len, + ZMQ_SNDMORE); + pcap_zmq_send(conn->server->zmq_publ, + data, len, + 0); +} + +static inline size_t calc_data_max_len(const struct osmo_pcap_server *server) +{ + size_t data_max_len; + + /* Some safe value regarding variable size options in a given pcapng block... */ + const size_t pcapng_max_len_opt = 4096; + const size_t pcapng_max_iface_len = 256; + + /* Maximum of the 2 types of .pcap blocks: */ + data_max_len = OSMO_MAX(sizeof(struct pcap_file_header), + sizeof(struct osmo_pcap_pkthdr) + server->max_snaplen); + + /* pcapng SHB: */ + const size_t pcapng_shb_max_len = sizeof(struct pcapng_block_header) + + sizeof(struct pcapng_section_header_block) + + pcapng_max_len_opt + + sizeof(uint32_t); + /* pcapng IDB: */ + const size_t pcapng_idb_max_len = sizeof(struct pcapng_block_header) + + sizeof(struct pcapng_iface_descr_block) + + pcapng_max_len_opt + + sizeof(uint32_t); + /* hdr_link for pcapng (SHB + N*IDB) */ + const size_t pcapg_hdr_link_max_len = pcapng_shb_max_len + // SHB + (pcapng_max_iface_len * pcapng_idb_max_len); // N * IDB + data_max_len = OSMO_MAX(data_max_len, pcapg_hdr_link_max_len); + + /* pcapng EPB: */ + const size_t pcapng_epb_max_len = sizeof(struct pcapng_block_header) + + sizeof(struct pcapng_enhanced_packet_block) + + pcapng_max_len_opt + + sizeof(uint32_t); + data_max_len = OSMO_MAX(data_max_len, pcapng_epb_max_len + server->max_snaplen); + return data_max_len; +} + +static struct osmo_pcap_conn *osmo_pcap_conn_alloc(struct osmo_pcap_server *server, + const char *name) +{ + struct osmo_pcap_conn *conn; + struct rate_ctr_group_desc *desc; + + conn = talloc_zero(server, struct osmo_pcap_conn); + if (!conn) { + LOGP(DSERVER, LOGL_ERROR, + "Failed to allocate the connection peer=%s.\n", name); + return NULL; + } + + conn->data_max_len = calc_data_max_len(server); + conn->data = talloc_zero_size(conn, sizeof(struct osmo_pcap_data) + conn->data_max_len); + /* a bit nasty. we do not work with ids but names */ + desc = talloc_zero(conn, struct rate_ctr_group_desc); + if (!desc) { + LOGP(DSERVER, LOGL_ERROR, + "Failed to allocate rate ctr desc peer=%s\n", name); + talloc_free(conn); + return NULL; + } + memcpy(desc, &pcap_peer_group_desc, sizeof(pcap_peer_group_desc)); + desc->group_name_prefix = talloc_asprintf(desc, "pcap:peer:%s", name); + if (!desc->group_name_prefix) { + LOGP(DSERVER, LOGL_ERROR, + "Failed to allocate group name prefix peer=%s\n", name); + talloc_free(conn); + return NULL; + } + desc->group_description = talloc_asprintf(desc, "PCAP peer statistics %s", name); + if (!desc->group_description) { + LOGP(DSERVER, LOGL_ERROR, + "Failed to allocate group description peer=%s\n", name); + talloc_free(conn); + return NULL; + } + + conn->ctrg = rate_ctr_group_alloc(desc, desc, 0); + if (!conn->ctrg) { + LOGP(DSERVER, LOGL_ERROR, + "Failed to allocate rate ctr peer=%s\n", name); + talloc_free(conn); + return NULL; + } + + + conn->name = talloc_strdup(conn, name); + /* we never write */ + osmo_wqueue_init(&conn->rem_wq, 0); + conn->rem_wq.bfd.fd = -1; + conn->local_fd = -1; + conn->server = server; + llist_add_tail(&conn->entry, &server->conn); + return conn; +} + +struct osmo_pcap_conn *osmo_pcap_server_find_or_create( + struct osmo_pcap_server *server, + const char *name) +{ + struct osmo_pcap_conn *conn; + + llist_for_each_entry(conn, &server->conn, entry) { + if (strcmp(conn->name, name) == 0) + return conn; + } + return osmo_pcap_conn_alloc(server, name); +} + +void osmo_pcap_conn_free(struct osmo_pcap_conn *conn) +{ + osmo_pcap_conn_close(conn); + llist_del(&conn->entry); + talloc_free(conn); +} + +/* Move pcap file from base_path to completed_path, and updates + * conn->curr_filename to point to new location. */ +void move_completed_trace_if_needed(struct osmo_pcap_conn *conn) +{ + struct osmo_pcap_server *server = conn->server; + char *curr_filename_cpy_bname = NULL; + char *curr_filename_cpy_dname = NULL; + char *bname = NULL; + char *curr_dirname = NULL; + char *new_dirname = NULL; + char *new_filename = NULL; + size_t new_filename_len; + int rc; + + if (!conn->curr_filename) + return; + + if (!server->completed_path) + return; + + /* Assumption: curr_filename is canonical absolute pathname. */ + + /* basename and dirname may modify input param, and return a string + * which shall not be freed, potentially pointing to the input param. */ + curr_filename_cpy_dname = talloc_strdup(conn, conn->curr_filename); + curr_filename_cpy_bname = talloc_strdup(conn, conn->curr_filename); + if (!curr_filename_cpy_dname || !curr_filename_cpy_bname) + goto ret_free1; + + curr_dirname = dirname(curr_filename_cpy_dname); + bname = basename(curr_filename_cpy_bname); + if (!curr_dirname || !bname) { + LOGP(DSERVER, LOGL_ERROR, "Failed to resolve dirname and basename for '%s'\n", + conn->curr_filename); + goto ret_free1; + } + + new_dirname = realpath(server->completed_path, NULL); + if (!new_dirname) { + LOGP(DSERVER, LOGL_ERROR, "Failed to resolve path '%s': %s\n", + server->completed_path, strerror(errno)); + goto ret_free1; + } + + new_filename_len = strlen(new_dirname) + 1 /* '/' */ + strlen(bname) + 1 /* '\0' */; + new_filename = talloc_size(conn, new_filename_len); + if (!new_filename) + goto ret_free1; + rc = snprintf(new_filename, new_filename_len, "%s/%s", new_dirname, bname); + if (rc != new_filename_len - 1) + goto ret_free2; + + LOGP(DSERVER, LOGL_INFO, "Moving completed pcap file '%s' -> '%s'\n", conn->curr_filename, new_filename); + rc = rename(conn->curr_filename, new_filename); + if (rc == -1) { + int err = errno; + LOGP(DSERVER, LOGL_ERROR, "Failed moving completed pcap file '%s' -> '%s': %s\n", + conn->curr_filename, new_filename, strerror(err)); + if (err == EXDEV) + LOGP(DSERVER, LOGL_ERROR, "Fix your config! %s and %s shall not be in different filesystems!\n", + curr_dirname, new_dirname); + goto ret_free2; + } + + /* Now replace conn->curr_filename with new path: */ + talloc_free(conn->curr_filename); + conn->curr_filename = new_filename; + /* new_filename has been assigned, so we don't want to free it, hence move to ret_free1: */ + goto ret_free1; + +ret_free2: + talloc_free(new_filename); +ret_free1: + free(new_dirname); + talloc_free(curr_filename_cpy_bname); + talloc_free(curr_filename_cpy_dname); +} + +void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn) +{ + if (conn->local_fd >= 0) { + close(conn->local_fd); + conn->local_fd = -1; + } + + move_completed_trace_if_needed(conn); + + if (conn->curr_filename) { + osmo_pcap_conn_event(conn, "closingtracefile", conn->curr_filename); + rate_ctr_inc2(conn->ctrg, PEER_CTR_PROTATE); + rate_ctr_inc2(conn->server->ctrg, SERVER_CTR_PROTATE); + talloc_free(conn->curr_filename); + conn->curr_filename = NULL; + } +} + +void osmo_pcap_conn_close(struct osmo_pcap_conn *conn) +{ + if (conn->rem_wq.bfd.fd >= 0) { + osmo_fd_unregister(&conn->rem_wq.bfd); + close(conn->rem_wq.bfd.fd); + conn->rem_wq.bfd.fd = -1; + osmo_tls_release(&conn->tls_session); + } + + osmo_pcap_conn_close_trace(conn); + osmo_pcap_conn_event(conn, "disconnect", NULL); +} + +/* Update conn->last_write if needed. This field is used to keep the last time + * period where we wrote to the pcap file. Once a new write period (based on + * rotation VTY config) is detected, the pcap file we write to is rotated. */ +static void update_last_write(struct osmo_pcap_conn *conn, time_t now) +{ + time_t last = mktime(&conn->last_write); + + /* Skip time udpates if wall clock went backwards (ie. due to drift + * correction or DST). As a result, time rotation checks will skip + * opening a new pcap file with an older timestamp, and instead keep + * using the current one. */ + if (now > last) + localtime_r(&now, &conn->last_write); +} + +void osmo_pcap_conn_restart_trace(struct osmo_pcap_conn *conn) +{ + time_t now = time(NULL); + struct tm tm; + int rc; + char *real_base_path; + + osmo_pcap_conn_close_trace(conn); + + /* omit any storing/creation of the file */ + if (!conn->store) { + update_last_write(conn, now); + talloc_free(conn->curr_filename); + conn->curr_filename = NULL; + return; + } + + localtime_r(&now, &tm); + real_base_path = realpath(conn->server->base_path, NULL); + if (!real_base_path) { + LOGP(DSERVER, LOGL_ERROR, "Failed to resolve real path '%s': %s\n", + conn->server->base_path, strerror(errno)); + return; + } + conn->curr_filename = talloc_asprintf(conn, "%s/trace-%s-%d%.2d%.2d_%.2d%.2d%.2d.%s", + real_base_path, conn->name, + tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, + tm.tm_hour, tm.tm_min, tm.tm_sec, + conn->file_fmt == OSMO_PCAP_FMT_PCAP ? "pcap" : "pcapng"); + free(real_base_path); + if (!conn->curr_filename) { + LOGP(DSERVER, LOGL_ERROR, "Failed to assemble filename for %s.\n", conn->name); + return; + } + + conn->local_fd = creat(conn->curr_filename, conn->server->permission_mask); + if (conn->local_fd < 0) { + LOGP(DSERVER, LOGL_ERROR, "Failed to create file '%s': %s\n", + conn->curr_filename, strerror(errno)); + return; + } + + rc = write(conn->local_fd, conn->file_hdr, conn->file_hdr_len); + if (rc != conn->file_hdr_len) { + LOGP(DSERVER, LOGL_ERROR, "Failed to write the header: %d\n", errno); + close(conn->local_fd); + conn->local_fd = -1; + return; + } + + update_last_write(conn, now); +} + +/* Returns true if pcap was re-opened */ +static bool check_restart_pcap_max_size(struct osmo_pcap_conn *conn, size_t data_len) +{ + off_t cur; + + if (!pcap_server->max_size_enabled) + return false; + cur = lseek(conn->local_fd, 0, SEEK_CUR); + if (cur + data_len <= conn->server->max_size) + return false; + LOGP(DSERVER, LOGL_NOTICE, "Rolling over file for %s (max-size)\n", conn->name); + osmo_pcap_conn_restart_trace(conn); + return true; +} + +/* Checks if we are in a new record period since last time we wrote to a pcap, to know (return true) + * whether a new pcap file needs to be opened. It calculates period based on intv and mod. + * NOTE: Kept non-static to be able to validate it with unit tests. */ +bool check_localtime(const struct tm *last_write, const struct tm *tm, enum time_interval intv, unsigned int mod) +{ + unsigned long long last_start, current_start; + + switch (intv) { + case TIME_INTERVAL_SEC: + if (last_write->tm_sec == tm->tm_sec && + last_write->tm_min == tm->tm_min && + last_write->tm_hour == tm->tm_hour && + last_write->tm_mday == tm->tm_mday && + last_write->tm_mon == tm->tm_mon && + last_write->tm_year == tm->tm_year) + return false; + /* If minute/hour/day/month/year changed (passed through second 0), always rotate: */ + if (last_write->tm_min < tm->tm_min || + last_write->tm_hour < tm->tm_hour || + last_write->tm_mday < tm->tm_mday || + last_write->tm_mon < tm->tm_mon || + last_write->tm_year < tm->tm_year) + return true; + /* Same minute/hour/day/month/year, second changed. Check if we are still in same period: */ + last_start = last_write->tm_sec - (last_write->tm_sec % mod); + current_start = tm->tm_sec - (tm->tm_sec % mod); + if (current_start <= last_start) + return false; + return true; + case TIME_INTERVAL_MIN: + if (last_write->tm_min == tm->tm_min && + last_write->tm_hour == tm->tm_hour && + last_write->tm_mday == tm->tm_mday && + last_write->tm_mon == tm->tm_mon && + last_write->tm_year == tm->tm_year) + return false; + /* If hour/day/month/year changed (passed through minute 0), always rotate: */ + if (last_write->tm_hour < tm->tm_hour || + last_write->tm_mday < tm->tm_mday || + last_write->tm_mon < tm->tm_mon || + last_write->tm_year < tm->tm_year) + return true; + /* Same hour/day/month/year, minute changed. Check if we are still in same period: */ + last_start = last_write->tm_min - (last_write->tm_min % mod); + current_start = tm->tm_min - (tm->tm_min % mod); + if (current_start <= last_start) + return false; + return true; + case TIME_INTERVAL_HOUR: + if (last_write->tm_hour == tm->tm_hour && + last_write->tm_mday == tm->tm_mday && + last_write->tm_mon == tm->tm_mon && + last_write->tm_year == tm->tm_year) + return false; + /* If day/month/year changed (passed through hour 0), always rotate: */ + if (last_write->tm_mday < tm->tm_mday || + last_write->tm_mon < tm->tm_mon || + last_write->tm_year < tm->tm_year) + return true; + /* Same day/month/year, hour changed. Check if we are still in same period: */ + last_start = last_write->tm_hour - (last_write->tm_hour % mod); + current_start = tm->tm_hour - (tm->tm_hour % mod); + if (current_start <= last_start) + return false; + return true; + case TIME_INTERVAL_DAY: + if (last_write->tm_mday == tm->tm_mday && + last_write->tm_mon == tm->tm_mon && + last_write->tm_year == tm->tm_year) + return false; + /* If month/year changed (passed through day 1), always rotate: */ + if (last_write->tm_mon < tm->tm_mon || + last_write->tm_year < tm->tm_year) + return true; + /* Same month/year, day changed. Check if we are still in same period: */ + /* Note: tm_mday is [1, 31], hence the -1 below: */ + last_start = (last_write->tm_mday - 1) - ((last_write->tm_mday - 1) % mod); + current_start = (tm->tm_mday - 1) - ((tm->tm_mday - 1) % mod); + if (current_start <= last_start) + return false; + return true; + case TIME_INTERVAL_MONTH: + if (last_write->tm_mon == tm->tm_mon && + last_write->tm_year == tm->tm_year) + return false; + /* If year changed (passed through month 1), always rotate: */ + if (last_write->tm_year < tm->tm_year) + return true; + /* Same year, month changed. Check if we are still in same period: */ + last_start = last_write->tm_mon - (last_write->tm_mon % mod); + current_start = tm->tm_mon - (tm->tm_mon % mod); + if (current_start <= last_start) + return false; + return true; + case TIME_INTERVAL_YEAR: + /* Year changed. Check if we are still in same period: */ + last_start = last_write->tm_year - (last_write->tm_year % mod); + current_start = tm->tm_year - (tm->tm_year % mod); + if (current_start <= last_start) + return false; + return true; + default: + OSMO_ASSERT(false); + } +} + +static bool check_restart_pcap_localtime(struct osmo_pcap_conn *conn, time_t now) +{ + struct tm tm; + if (!pcap_server->rotate_localtime.enabled) + return false; + + localtime_r(&now, &tm); + if (!check_localtime(&conn->last_write, &tm, + pcap_server->rotate_localtime.intv, + pcap_server->rotate_localtime.modulus)) + return false; + LOGP(DSERVER, LOGL_NOTICE, "Rolling over file for %s (localtime)\n", conn->name); + osmo_pcap_conn_restart_trace(conn); + return true; +} + +/* New recorded packet is received. + * Returns 0 on success, negative on error. */ +int osmo_pcap_conn_process_data(struct osmo_pcap_conn *conn, const uint8_t *data, size_t len) +{ + time_t now = time(NULL); + int rc; + + zmq_send_client_data(conn, data, len); + + if (!conn->store) { + update_last_write(conn, now); + return 0; + } + + if (conn->local_fd < -1) { + LOGP(DSERVER, LOGL_ERROR, "No file is open. close connection.\n"); + return -1; + } + + /* Check if we are past the limit or on a day change. */ + if (!check_restart_pcap_max_size(conn, len)) + check_restart_pcap_localtime(conn, now); + + update_last_write(conn, now); + rc = write(conn->local_fd, data, len); + if (rc != len) { + LOGP(DSERVER, LOGL_ERROR, "Failed to write for %s\n", conn->name); + return -1; + } + return 0; +} diff --git a/src/osmo_server_network.c b/src/osmo_server_network.c index b6e3be4..af2154b 100644 --- a/src/osmo_server_network.c +++ b/src/osmo_server_network.c @@ -44,255 +44,6 @@ #include <inttypes.h> #include <libgen.h> -static void pcap_zmq_send(void *publ, const void *data, size_t len, int flags) -{ - int rc; - zmq_msg_t msg; - - rc = zmq_msg_init_size(&msg, len); - if (rc != 0) { - /* sigh.. we said SNDMORE but can't... */ - LOGP(DSERVER, LOGL_ERROR, "Failed to init rc=%d errno=%d/%s\n", - rc, errno, strerror(errno)); - return; - } - memcpy(zmq_msg_data(&msg), data, len); - rc = zmq_msg_send(&msg, publ, flags); - if (rc == -1) { - /* is the zmq_msg now owned? leak??? */ - LOGP(DSERVER, LOGL_ERROR, "Failed to send data rc=%d errno=%d/%s\n", - rc, errno, strerror(errno)); - return; - } -} - -static void client_event(struct osmo_pcap_conn *conn, - const char *event, const char *data) -{ - char *event_name; - - if (!conn->server->zmq_publ) - return; - - /* - * This multi-part support is insane... so if we lose the first - * or the last part of the multipart message stuff is going out - * of sync. *great* As we can't do anything about it right now - * just close the eyese and send it. - */ - event_name = talloc_asprintf(conn, "event.v1.%s.%s", - event, conn->name); - pcap_zmq_send(conn->server->zmq_publ, - event_name, strlen(event_name), - data ? ZMQ_SNDMORE : 0); - talloc_free(event_name); - if (data) - pcap_zmq_send(conn->server->zmq_publ, data, strlen(data), 0); -} - -static void zmq_send_client_data(struct osmo_pcap_conn *conn, - struct osmo_pcap_data *data) -{ - char *event_name; - - if (!conn->server->zmq_publ) - return; - - /* - * This multi-part support is insane... so if we lose the first - * or the last part of the multipart message stuff is going out - * of sync. *great* As we can't do anything about it right now - * just close the eyes and send it. - */ - event_name = talloc_asprintf(conn, "data.v1.%s", conn->name); - pcap_zmq_send(conn->server->zmq_publ, event_name, strlen(event_name), ZMQ_SNDMORE); - talloc_free(event_name); - - pcap_zmq_send(conn->server->zmq_publ, - conn->file_hdr, conn->file_hdr_len, - ZMQ_SNDMORE); - pcap_zmq_send(conn->server->zmq_publ, - &data->data[0], data->len, - 0); -} - -/* Move pcap file from base_path to completed_path, and updates - * conn->curr_filename to point to new location. */ -void move_completed_trace_if_needed(struct osmo_pcap_conn *conn) -{ - struct osmo_pcap_server *server = conn->server; - char *curr_filename_cpy_bname = NULL; - char *curr_filename_cpy_dname = NULL; - char *bname = NULL; - char *curr_dirname = NULL; - char *new_dirname = NULL; - char *new_filename = NULL; - size_t new_filename_len; - int rc; - - if (!conn->curr_filename) - return; - - if (!server->completed_path) - return; - - /* Assumption: curr_filename is canonical absolute pathname. */ - - /* basename and dirname may modify input param, and return a string - * which shall not be freed, potentially pointing to the input param. */ - curr_filename_cpy_dname = talloc_strdup(conn, conn->curr_filename); - curr_filename_cpy_bname = talloc_strdup(conn, conn->curr_filename); - if (!curr_filename_cpy_dname || !curr_filename_cpy_bname) - goto ret_free1; - - curr_dirname = dirname(curr_filename_cpy_dname); - bname = basename(curr_filename_cpy_bname); - if (!curr_dirname || !bname) { - LOGP(DSERVER, LOGL_ERROR, "Failed to resolve dirname and basename for '%s'\n", - conn->curr_filename); - goto ret_free1; - } - - new_dirname = realpath(server->completed_path, NULL); - if (!new_dirname) { - LOGP(DSERVER, LOGL_ERROR, "Failed to resolve path '%s': %s\n", - server->completed_path, strerror(errno)); - goto ret_free1; - } - - new_filename_len = strlen(new_dirname) + 1 /* '/' */ + strlen(bname) + 1 /* '\0' */; - new_filename = talloc_size(conn, new_filename_len); - if (!new_filename) - goto ret_free1; - rc = snprintf(new_filename, new_filename_len, "%s/%s", new_dirname, bname); - if (rc != new_filename_len - 1) - goto ret_free2; - - LOGP(DSERVER, LOGL_INFO, "Moving completed pcap file '%s' -> '%s'\n", conn->curr_filename, new_filename); - rc = rename(conn->curr_filename, new_filename); - if (rc == -1) { - int err = errno; - LOGP(DSERVER, LOGL_ERROR, "Failed moving completed pcap file '%s' -> '%s': %s\n", - conn->curr_filename, new_filename, strerror(err)); - if (err == EXDEV) - LOGP(DSERVER, LOGL_ERROR, "Fix your config! %s and %s shall not be in different filesystems!\n", - curr_dirname, new_dirname); - goto ret_free2; - } - - /* Now replace conn->curr_filename with new path: */ - talloc_free(conn->curr_filename); - conn->curr_filename = new_filename; - /* new_filename has been assigned, so we don't want to free it, hence move to ret_free1: */ - goto ret_free1; - -ret_free2: - talloc_free(new_filename); -ret_free1: - free(new_dirname); - talloc_free(curr_filename_cpy_bname); - talloc_free(curr_filename_cpy_dname); -} - -void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn) -{ - if (conn->local_fd >= 0) { - close(conn->local_fd); - conn->local_fd = -1; - } - - move_completed_trace_if_needed(conn); - - if (conn->curr_filename) { - client_event(conn, "closingtracefile", conn->curr_filename); - rate_ctr_inc2(conn->ctrg, PEER_CTR_PROTATE); - rate_ctr_inc2(conn->server->ctrg, SERVER_CTR_PROTATE); - talloc_free(conn->curr_filename); - conn->curr_filename = NULL; - } -} - -void osmo_pcap_conn_close(struct osmo_pcap_conn *conn) -{ - if (conn->rem_wq.bfd.fd >= 0) { - osmo_fd_unregister(&conn->rem_wq.bfd); - close(conn->rem_wq.bfd.fd); - conn->rem_wq.bfd.fd = -1; - osmo_tls_release(&conn->tls_session); - } - - osmo_pcap_conn_close_trace(conn); - client_event(conn, "disconnect", NULL); -} - -/* Update conn->last_write if needed. This field is used to keep the last time - * period where we wrote to the pcap file. Once a new write period (based on - * rotation VTY config) is detected, the pcap file we write to is rotated. */ -static void update_last_write(struct osmo_pcap_conn *conn, time_t now) -{ - time_t last = mktime(&conn->last_write); - - /* Skip time udpates if wall clock went backwards (ie. due to drift - * correction or DST). As a result, time rotation checks will skip - * opening a new pcap file with an older timestamp, and instead keep - * using the current one. */ - if (now > last) - localtime_r(&now, &conn->last_write); -} - -static void restart_pcap(struct osmo_pcap_conn *conn) -{ - time_t now = time(NULL); - struct tm tm; - int rc; - char *real_base_path; - - osmo_pcap_conn_close_trace(conn); - - /* omit any storing/creation of the file */ - if (!conn->store) { - update_last_write(conn, now); - talloc_free(conn->curr_filename); - conn->curr_filename = NULL; - return; - } - - localtime_r(&now, &tm); - real_base_path = realpath(conn->server->base_path, NULL); - if (!real_base_path) { - LOGP(DSERVER, LOGL_ERROR, "Failed to resolve real path '%s': %s\n", - conn->server->base_path, strerror(errno)); - return; - } - conn->curr_filename = talloc_asprintf(conn, "%s/trace-%s-%d%.2d%.2d_%.2d%.2d%.2d.%s", - real_base_path, conn->name, - tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, - tm.tm_hour, tm.tm_min, tm.tm_sec, - conn->file_fmt == OSMO_PCAP_FMT_PCAP ? "pcap" : "pcapng"); - free(real_base_path); - if (!conn->curr_filename) { - LOGP(DSERVER, LOGL_ERROR, "Failed to assemble filename for %s.\n", conn->name); - return; - } - - conn->local_fd = creat(conn->curr_filename, conn->server->permission_mask); - if (conn->local_fd < 0) { - LOGP(DSERVER, LOGL_ERROR, "Failed to create file '%s': %s\n", - conn->curr_filename, strerror(errno)); - return; - } - - rc = write(conn->local_fd, conn->file_hdr, conn->file_hdr_len); - if (rc != conn->file_hdr_len) { - LOGP(DSERVER, LOGL_ERROR, "Failed to write the header: %d\n", errno); - close(conn->local_fd); - conn->local_fd = -1; - return; - } - - update_last_write(conn, now); -} - static int validate_link_hdr_pcap(const struct osmo_pcap_conn *conn, const struct osmo_pcap_data *data) { struct pcap_file_header *hdr; @@ -395,7 +146,7 @@ conn->file_hdr = talloc_size(conn, data->len); memcpy(conn->file_hdr, data->data, data->len); conn->file_hdr_len = data->len; - restart_pcap(conn); + osmo_pcap_conn_restart_trace(conn); } else if (conn->file_hdr_len != data->len || memcmp(&conn->file_hdr, data->data, data->len) != 0) { /* Client changed the link hdr in conn */ @@ -403,149 +154,12 @@ conn->file_hdr = talloc_size(conn, data->len); memcpy(conn->file_hdr, data->data, data->len); conn->file_hdr_len = data->len; - restart_pcap(conn); + osmo_pcap_conn_restart_trace(conn); } return 1; } -/* Returns true if pcap was re-opened */ -static bool check_restart_pcap_max_size(struct osmo_pcap_conn *conn, const struct osmo_pcap_data *data) -{ - off_t cur; - - if (!pcap_server->max_size_enabled) - return false; - cur = lseek(conn->local_fd, 0, SEEK_CUR); - if (cur + data->len <= conn->server->max_size) - return false; - LOGP(DSERVER, LOGL_NOTICE, "Rolling over file for %s (max-size)\n", conn->name); - restart_pcap(conn); - return true; -} - -/* Checks if we are in a new record period since last time we wrote to a pcap, to know (return true) - * whether a new pcap file needs to be opened. It calculates period based on intv and mod. - * NOTE: Kept non-static to be able to validate it with unit tests. */ -bool check_localtime(const struct tm *last_write, const struct tm *tm, enum time_interval intv, unsigned int mod) -{ - unsigned long long last_start, current_start; - - switch (intv) { - case TIME_INTERVAL_SEC: - if (last_write->tm_sec == tm->tm_sec && - last_write->tm_min == tm->tm_min && - last_write->tm_hour == tm->tm_hour && - last_write->tm_mday == tm->tm_mday && - last_write->tm_mon == tm->tm_mon && - last_write->tm_year == tm->tm_year) - return false; - /* If minute/hour/day/month/year changed (passed through second 0), always rotate: */ - if (last_write->tm_min < tm->tm_min || - last_write->tm_hour < tm->tm_hour || - last_write->tm_mday < tm->tm_mday || - last_write->tm_mon < tm->tm_mon || - last_write->tm_year < tm->tm_year) - return true; - /* Same minute/hour/day/month/year, second changed. Check if we are still in same period: */ - last_start = last_write->tm_sec - (last_write->tm_sec % mod); - current_start = tm->tm_sec - (tm->tm_sec % mod); - if (current_start <= last_start) - return false; - return true; - case TIME_INTERVAL_MIN: - if (last_write->tm_min == tm->tm_min && - last_write->tm_hour == tm->tm_hour && - last_write->tm_mday == tm->tm_mday && - last_write->tm_mon == tm->tm_mon && - last_write->tm_year == tm->tm_year) - return false; - /* If hour/day/month/year changed (passed through minute 0), always rotate: */ - if (last_write->tm_hour < tm->tm_hour || - last_write->tm_mday < tm->tm_mday || - last_write->tm_mon < tm->tm_mon || - last_write->tm_year < tm->tm_year) - return true; - /* Same hour/day/month/year, minute changed. Check if we are still in same period: */ - last_start = last_write->tm_min - (last_write->tm_min % mod); - current_start = tm->tm_min - (tm->tm_min % mod); - if (current_start <= last_start) - return false; - return true; - case TIME_INTERVAL_HOUR: - if (last_write->tm_hour == tm->tm_hour && - last_write->tm_mday == tm->tm_mday && - last_write->tm_mon == tm->tm_mon && - last_write->tm_year == tm->tm_year) - return false; - /* If day/month/year changed (passed through hour 0), always rotate: */ - if (last_write->tm_mday < tm->tm_mday || - last_write->tm_mon < tm->tm_mon || - last_write->tm_year < tm->tm_year) - return true; - /* Same day/month/year, hour changed. Check if we are still in same period: */ - last_start = last_write->tm_hour - (last_write->tm_hour % mod); - current_start = tm->tm_hour - (tm->tm_hour % mod); - if (current_start <= last_start) - return false; - return true; - case TIME_INTERVAL_DAY: - if (last_write->tm_mday == tm->tm_mday && - last_write->tm_mon == tm->tm_mon && - last_write->tm_year == tm->tm_year) - return false; - /* If month/year changed (passed through day 1), always rotate: */ - if (last_write->tm_mon < tm->tm_mon || - last_write->tm_year < tm->tm_year) - return true; - /* Same month/year, day changed. Check if we are still in same period: */ - /* Note: tm_mday is [1, 31], hence the -1 below: */ - last_start = (last_write->tm_mday - 1) - ((last_write->tm_mday - 1) % mod); - current_start = (tm->tm_mday - 1) - ((tm->tm_mday - 1) % mod); - if (current_start <= last_start) - return false; - return true; - case TIME_INTERVAL_MONTH: - if (last_write->tm_mon == tm->tm_mon && - last_write->tm_year == tm->tm_year) - return false; - /* If year changed (passed through month 1), always rotate: */ - if (last_write->tm_year < tm->tm_year) - return true; - /* Same year, month changed. Check if we are still in same period: */ - last_start = last_write->tm_mon - (last_write->tm_mon % mod); - current_start = tm->tm_mon - (tm->tm_mon % mod); - if (current_start <= last_start) - return false; - return true; - case TIME_INTERVAL_YEAR: - /* Year changed. Check if we are still in same period: */ - last_start = last_write->tm_year - (last_write->tm_year % mod); - current_start = tm->tm_year - (tm->tm_year % mod); - if (current_start <= last_start) - return false; - return true; - default: - OSMO_ASSERT(false); - } -} - -static bool check_restart_pcap_localtime(struct osmo_pcap_conn *conn, time_t now) -{ - struct tm tm; - if (!pcap_server->rotate_localtime.enabled) - return false; - - localtime_r(&now, &tm); - if (!check_localtime(&conn->last_write, &tm, - pcap_server->rotate_localtime.intv, - pcap_server->rotate_localtime.modulus)) - return false; - LOGP(DSERVER, LOGL_NOTICE, "Rolling over file for %s (localtime)\n", conn->name); - restart_pcap(conn); - return true; -} - static int validate_link_data_pcap(const struct osmo_pcap_conn *conn, const struct osmo_pcap_data *data) { unsigned int min_len, max_len; @@ -629,152 +243,16 @@ /* returns >0 on success, <= 0 on failure (closes conn) */ static int rx_link_data(struct osmo_pcap_conn *conn, struct osmo_pcap_data *data) { - time_t now = time(NULL); int rc; if ((rc = validate_link_data(conn, data)) < 0) return rc; - zmq_send_client_data(conn, data); - - if (!conn->store) { - update_last_write(conn, now); - return 1; - } - - if (conn->local_fd < -1) { - LOGP(DSERVER, LOGL_ERROR, "No file is open. close connection.\n"); - return -1; - } - - /* Check if we are past the limit or on a day change. */ - if (!check_restart_pcap_max_size(conn, data)) - check_restart_pcap_localtime(conn, now); - - update_last_write(conn, now); - rc = write(conn->local_fd, &data->data[0], data->len); - if (rc != data->len) { - LOGP(DSERVER, LOGL_ERROR, "Failed to write for %s\n", conn->name); - return -1; - } + if ((rc = osmo_pcap_conn_process_data(conn, &data->data[0], data->len))) + return rc; return 1; } - -void osmo_pcap_conn_free(struct osmo_pcap_conn *conn) -{ - osmo_pcap_conn_close(conn); - llist_del(&conn->entry); - talloc_free(conn); -} - -static inline size_t calc_data_max_len(const struct osmo_pcap_server *server) -{ - size_t data_max_len; - - /* Some safe value regarding variable size options in a given pcapng block... */ - const size_t pcapng_max_len_opt = 4096; - const size_t pcapng_max_iface_len = 256; - - /* Maximum of the 2 types of .pcap blocks: */ - data_max_len = OSMO_MAX(sizeof(struct pcap_file_header), - sizeof(struct osmo_pcap_pkthdr) + server->max_snaplen); - - /* pcapng SHB: */ - const size_t pcapng_shb_max_len = sizeof(struct pcapng_block_header) + - sizeof(struct pcapng_section_header_block) + - pcapng_max_len_opt + - sizeof(uint32_t); - /* pcapng IDB: */ - const size_t pcapng_idb_max_len = sizeof(struct pcapng_block_header) + - sizeof(struct pcapng_iface_descr_block) + - pcapng_max_len_opt + - sizeof(uint32_t); - /* hdr_link for pcapng (SHB + N*IDB) */ - const size_t pcapg_hdr_link_max_len = pcapng_shb_max_len + // SHB - (pcapng_max_iface_len * pcapng_idb_max_len); // N * IDB - data_max_len = OSMO_MAX(data_max_len, pcapg_hdr_link_max_len); - - /* pcapng EPB: */ - const size_t pcapng_epb_max_len = sizeof(struct pcapng_block_header) + - sizeof(struct pcapng_enhanced_packet_block) + - pcapng_max_len_opt + - sizeof(uint32_t); - data_max_len = OSMO_MAX(data_max_len, pcapng_epb_max_len + server->max_snaplen); - return data_max_len; -} - -static struct osmo_pcap_conn *osmo_pcap_conn_alloc(struct osmo_pcap_server *server, - const char *name) -{ - struct osmo_pcap_conn *conn; - struct rate_ctr_group_desc *desc; - - conn = talloc_zero(server, struct osmo_pcap_conn); - if (!conn) { - LOGP(DSERVER, LOGL_ERROR, - "Failed to allocate the connection peer=%s.\n", name); - return NULL; - } - - conn->data_max_len = calc_data_max_len(server); - conn->data = talloc_zero_size(conn, sizeof(struct osmo_pcap_data) + conn->data_max_len); - /* a bit nasty. we do not work with ids but names */ - desc = talloc_zero(conn, struct rate_ctr_group_desc); - if (!desc) { - LOGP(DSERVER, LOGL_ERROR, - "Failed to allocate rate ctr desc peer=%s\n", name); - talloc_free(conn); - return NULL; - } - memcpy(desc, &pcap_peer_group_desc, sizeof(pcap_peer_group_desc)); - desc->group_name_prefix = talloc_asprintf(desc, "pcap:peer:%s", name); - if (!desc->group_name_prefix) { - LOGP(DSERVER, LOGL_ERROR, - "Failed to allocate group name prefix peer=%s\n", name); - talloc_free(conn); - return NULL; - } - desc->group_description = talloc_asprintf(desc, "PCAP peer statistics %s", name); - if (!desc->group_description) { - LOGP(DSERVER, LOGL_ERROR, - "Failed to allocate group description peer=%s\n", name); - talloc_free(conn); - return NULL; - } - - conn->ctrg = rate_ctr_group_alloc(desc, desc, 0); - if (!conn->ctrg) { - LOGP(DSERVER, LOGL_ERROR, - "Failed to allocate rate ctr peer=%s\n", name); - talloc_free(conn); - return NULL; - } - - - conn->name = talloc_strdup(conn, name); - /* we never write */ - osmo_wqueue_init(&conn->rem_wq, 0); - conn->rem_wq.bfd.fd = -1; - conn->local_fd = -1; - conn->server = server; - llist_add_tail(&conn->entry, &server->conn); - return conn; -} - -struct osmo_pcap_conn *osmo_pcap_server_find_or_create( - struct osmo_pcap_server *server, - const char *name) -{ - struct osmo_pcap_conn *conn; - - llist_for_each_entry(conn, &server->conn, entry) { - if (strcmp(conn->name, name) == 0) - return conn; - } - return osmo_pcap_conn_alloc(server, name); -} - static int do_read_tls(struct osmo_pcap_conn *conn, void *buf, size_t want_size) { size_t size = want_size; @@ -871,7 +349,7 @@ if (conn->reopen_delayed) { LOGP(DSERVER, LOGL_INFO, "Reopening log for %s now.\n", conn->name); - restart_pcap(conn); + osmo_pcap_conn_restart_trace(conn); conn->reopen_delayed = false; } @@ -1005,7 +483,7 @@ if (conn->remote_addr.s_addr == addr.sin_addr.s_addr) { LOGP(DSERVER, LOGL_NOTICE, "New connection from %s\n", conn->name); - client_event(conn, "connect", NULL); + osmo_pcap_conn_event(conn, "connect", NULL); new_connection(server, conn, new_fd); return 0; } @@ -1056,7 +534,7 @@ llist_for_each_entry(conn, &server->conn, entry) { /* Write the complete packet out first */ if (conn->state == STATE_INITIAL) { - restart_pcap(conn); + osmo_pcap_conn_restart_trace(conn); } else { LOGP(DSERVER, LOGL_INFO, "Delaying %s until current packet is complete.\n", conn->name); conn->reopen_delayed = true; diff --git a/tests/rotate_localtime/Makefile.am b/tests/rotate_localtime/Makefile.am index 265e4fe..f6ced6e 100644 --- a/tests/rotate_localtime/Makefile.am +++ b/tests/rotate_localtime/Makefile.am @@ -29,6 +29,7 @@ rotate_localtime_test_LDADD = \ $(top_builddir)/src/osmo_pcap_file.o \ + $(top_builddir)/src/osmo_server_core.o \ $(top_builddir)/src/osmo_server_network.o \ $(top_builddir)/src/osmo_server_stats.o \ $(top_builddir)/src/osmo_tls.o \ -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39302?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I41d6e0398605becbea109a7bb5aac8981d65ab1b Gerrit-Change-Number: 39302 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[S] Change in osmo-pcap[master]: server: Rename osmo_pcap_server_close_trace -> osmo_pcap_conn_close_t...

by pespin

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-pcap/+/39300?usp=email ) Change subject: server: Rename osmo_pcap_server_close_trace -> osmo_pcap_conn_close_trace ...................................................................... server: Rename osmo_pcap_server_close_trace -> osmo_pcap_conn_close_trace Change-Id: I261c6a54bd31b8f511cb103840323e7343bfdc61 --- M include/osmo-pcap/osmo_pcap_server.h M src/osmo_server_network.c M src/osmo_server_vty.c 3 files changed, 5 insertions(+), 5 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/00/39300/1 diff --git a/include/osmo-pcap/osmo_pcap_server.h b/include/osmo-pcap/osmo_pcap_server.h index f44781a..bdb0b81 100644 --- a/include/osmo-pcap/osmo_pcap_server.h +++ b/include/osmo-pcap/osmo_pcap_server.h @@ -168,5 +168,5 @@ const char *name); void osmo_pcap_conn_free(struct osmo_pcap_conn *conn); void vty_server_init(void); -void osmo_pcap_server_close_trace(struct osmo_pcap_conn *conn); +void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn); void osmo_pcap_server_close_conn(struct osmo_pcap_conn *conn); diff --git a/src/osmo_server_network.c b/src/osmo_server_network.c index ac8aff6..30a89d5 100644 --- a/src/osmo_server_network.c +++ b/src/osmo_server_network.c @@ -194,7 +194,7 @@ talloc_free(curr_filename_cpy_dname); } -void osmo_pcap_server_close_trace(struct osmo_pcap_conn *conn) +void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn) { if (conn->local_fd >= 0) { close(conn->local_fd); @@ -221,7 +221,7 @@ osmo_tls_release(&conn->tls_session); } - osmo_pcap_server_close_trace(conn); + osmo_pcap_conn_close_trace(conn); client_event(conn, "disconnect", NULL); } @@ -252,7 +252,7 @@ int rc; char *real_base_path; - osmo_pcap_server_close_trace(conn); + osmo_pcap_conn_close_trace(conn); /* omit any storing/creation of the file */ if (!conn->store) { diff --git a/src/osmo_server_vty.c b/src/osmo_server_vty.c index 9ba5f74..c019657 100644 --- a/src/osmo_server_vty.c +++ b/src/osmo_server_vty.c @@ -366,7 +366,7 @@ /* Checking store and maybe closing a pcap file */ if (!store) - osmo_pcap_server_close_trace(conn); + osmo_pcap_conn_close_trace(conn); conn->store = store; if (use_tls) { -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39300?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I261c6a54bd31b8f511cb103840323e7343bfdc61 Gerrit-Change-Number: 39300 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[S] Change in osmo-pcap[master]: Rename osmo_pcap_server_close_conn -> osmo_pcap_conn_close

by pespin

pespin has uploaded this change for review. ( https://gerrit.osmocom.org/c/osmo-pcap/+/39301?usp=email ) Change subject: Rename osmo_pcap_server_close_conn -> osmo_pcap_conn_close ...................................................................... Rename osmo_pcap_server_close_conn -> osmo_pcap_conn_close While at it, remove unnecessary helper function close_connection(). Change-Id: I116f0c4d61d391faa30760be3afdf189ff921d0c --- M include/osmo-pcap/osmo_pcap_server.h M src/osmo_server_network.c M src/osmo_server_vty.c 3 files changed, 9 insertions(+), 14 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/01/39301/1 diff --git a/include/osmo-pcap/osmo_pcap_server.h b/include/osmo-pcap/osmo_pcap_server.h index bdb0b81..d91ffb3 100644 --- a/include/osmo-pcap/osmo_pcap_server.h +++ b/include/osmo-pcap/osmo_pcap_server.h @@ -169,4 +169,4 @@ void osmo_pcap_conn_free(struct osmo_pcap_conn *conn); void vty_server_init(void); void osmo_pcap_conn_close_trace(struct osmo_pcap_conn *conn); -void osmo_pcap_server_close_conn(struct osmo_pcap_conn *conn); +void osmo_pcap_conn_close(struct osmo_pcap_conn *conn); diff --git a/src/osmo_server_network.c b/src/osmo_server_network.c index 30a89d5..b6e3be4 100644 --- a/src/osmo_server_network.c +++ b/src/osmo_server_network.c @@ -212,7 +212,7 @@ } } -static void close_connection(struct osmo_pcap_conn *conn) +void osmo_pcap_conn_close(struct osmo_pcap_conn *conn) { if (conn->rem_wq.bfd.fd >= 0) { osmo_fd_unregister(&conn->rem_wq.bfd); @@ -225,11 +225,6 @@ client_event(conn, "disconnect", NULL); } -void osmo_pcap_server_close_conn(struct osmo_pcap_conn *conn) -{ - return close_connection(conn); -} - /* Update conn->last_write if needed. This field is used to keep the last time * period where we wrote to the pcap file. Once a new write period (based on * rotation VTY config) is detected, the pcap file we write to is rotated. */ @@ -668,7 +663,7 @@ void osmo_pcap_conn_free(struct osmo_pcap_conn *conn) { - close_connection(conn); + osmo_pcap_conn_close(conn); llist_del(&conn->entry); talloc_free(conn); } @@ -903,7 +898,7 @@ conn = fd->data; rc = dispatch_read(conn); if (rc <= 0) - close_connection(conn); + osmo_pcap_conn_close(conn); return 0; } @@ -911,7 +906,7 @@ { struct osmo_pcap_conn *conn; conn = container_of(session, struct osmo_pcap_conn, tls_session); - close_connection(conn); + osmo_pcap_conn_close(conn); } static int tls_read_cb(struct osmo_tls_session *session) @@ -946,7 +941,7 @@ static void new_connection(struct osmo_pcap_server *server, struct osmo_pcap_conn *client, int new_fd) { - close_connection(client); + osmo_pcap_conn_close(client); TALLOC_FREE(client->file_hdr); client->file_hdr_len = 0; @@ -968,11 +963,11 @@ LOGP(DSERVER, LOGL_NOTICE, "Require TLS but not enabled on conn=%s\n", client->name); - close_connection(client); + osmo_pcap_conn_close(client); return; } else if (client->tls_use) { if (!osmo_tls_init_server_session(client, server)) { - close_connection(client); + osmo_pcap_conn_close(client); return; } client->tls_session.error = tls_error_cb; diff --git a/src/osmo_server_vty.c b/src/osmo_server_vty.c index c019657..eab5cb4 100644 --- a/src/osmo_server_vty.c +++ b/src/osmo_server_vty.c @@ -372,7 +372,7 @@ if (use_tls) { /* force moving to TLS */ if (!conn->tls_use) - osmo_pcap_server_close_conn(conn); + osmo_pcap_conn_close(conn); conn->tls_use = true; } else { conn->tls_use = false; -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39301?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I116f0c4d61d391faa30760be3afdf189ff921d0c Gerrit-Change-Number: 39301 Gerrit-PatchSet: 1 Gerrit-Owner: pespin <pespin(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[L] Change in osmo-pcap[master]: server: Support storing of pcapng format

by osmith

Attention is currently required from: daniel, laforge, pespin. osmith has posted comments on this change by pespin. ( https://gerrit.osmocom.org/c/osmo-pcap/+/39287?usp=email ) Change subject: server: Support storing of pcapng format ...................................................................... Patch Set 5: Code-Review+1 -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39287?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I275d28ab418a1514fa9c5c7c20f3d831cc6af8bb Gerrit-Change-Number: 39287 Gerrit-PatchSet: 5 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: daniel <dwillmann(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Attention: daniel <dwillmann(a)sysmocom.de> Gerrit-Comment-Date: Mon, 13 Jan 2025 16:46:10 +0000 Gerrit-HasComments: No Gerrit-Has-Labels: Yes

5 months, 1 week

1
0
0 0

[L] Change in osmo-pcap[master]: client: Support generation of pcapng file format

by osmith

Attention is currently required from: daniel, laforge, pespin. osmith has posted comments on this change by pespin. ( https://gerrit.osmocom.org/c/osmo-pcap/+/39265?usp=email ) Change subject: client: Support generation of pcapng file format ...................................................................... Patch Set 9: Code-Review+1 (1 comment) File include/osmo-pcap/osmo_pcap_file.h: https://gerrit.osmocom.org/c/osmo-pcap/+/39265/comment/f1e7c62d_b3620d38?us… : PS8, Line 87: /* uint32_t block_total_length */ > Nope, it's fine, the field is duplicated at the end of block_body so that the file can be navigated […] ah nice -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39265?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I3c80518a1e53a1f77e1aca8dfa83f683f9516ad6 Gerrit-Change-Number: 39265 Gerrit-PatchSet: 9 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: daniel <dwillmann(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Attention: daniel <dwillmann(a)sysmocom.de> Gerrit-Comment-Date: Mon, 13 Jan 2025 16:45:44 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: Yes Comment-In-Reply-To: osmith <osmith(a)sysmocom.de> Comment-In-Reply-To: pespin <pespin(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[M] Change in osmo-bsc[master]: nokia_site: introduce hopping control for Nokia *Site

by pespin

Attention is currently required from: csaba.sipos, fixeria, laforge. pespin has posted comments on this change by csaba.sipos. ( https://gerrit.osmocom.org/c/osmo-bsc/+/39290?usp=email ) Change subject: nokia_site: introduce hopping control for Nokia *Site ...................................................................... Patch Set 7: (2 comments) File src/osmo-bsc/bts_vty.c: https://gerrit.osmocom.org/c/osmo-bsc/+/39290/comment/f3af5b7f_a6e84c99?usp… : PS6, Line 468: DEFUN_ATTR(cfg_bts_nokia_site_hopping_type, > This is just a Nokia specific OML extension the BTS needs. […] Done https://gerrit.osmocom.org/c/osmo-bsc/+/39290/comment/5be16043_cfaa4e93?usp… : PS6, Line 470: nokia_hopping > Done Unless that 0|1 numbering is a well know thing from nokia, I'd also favour to have proper string literal which are somehow self-descriptive as Vadim proposed. -- To view, visit https://gerrit.osmocom.org/c/osmo-bsc/+/39290?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmo-bsc Gerrit-Branch: master Gerrit-Change-Id: Idb1d269d2ee80d72ede2394125e8acedf0ee8b06 Gerrit-Change-Number: 39290 Gerrit-PatchSet: 7 Gerrit-Owner: csaba.sipos <metro4(a)freemail.hu> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-CC: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-CC: pespin <pespin(a)sysmocom.de> Gerrit-Attention: csaba.sipos <metro4(a)freemail.hu> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: fixeria <vyanitskiy(a)sysmocom.de> Gerrit-Comment-Date: Mon, 13 Jan 2025 16:18:43 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: csaba.sipos <metro4(a)freemail.hu> Comment-In-Reply-To: pespin <pespin(a)sysmocom.de> Comment-In-Reply-To: fixeria <vyanitskiy(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[L] Change in osmo-pcap[master]: server: Support storing of pcapng format

by pespin

Attention is currently required from: daniel, laforge, osmith. pespin has posted comments on this change by pespin. ( https://gerrit.osmocom.org/c/osmo-pcap/+/39287?usp=email ) Change subject: server: Support storing of pcapng format ...................................................................... Patch Set 4: (1 comment) File src/osmo_server_network.c: https://gerrit.osmocom.org/c/osmo-pcap/+/39287/comment/9d33c7e3_85325a1f?us… : PS4, Line 314: "The recvd pcap_file_header contains too big snaplen %zu > %zu\n", : (size_t) hdr->snaplen, (size_t) conn->server->max_snaplen); > missing indentation Done -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39287?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I275d28ab418a1514fa9c5c7c20f3d831cc6af8bb Gerrit-Change-Number: 39287 Gerrit-PatchSet: 4 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: daniel <dwillmann(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: osmith <osmith(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: daniel <dwillmann(a)sysmocom.de> Gerrit-Comment-Date: Mon, 13 Jan 2025 16:15:06 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: osmith <osmith(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[L] Change in osmo-pcap[master]: client: Support generation of pcapng file format

by pespin

Attention is currently required from: daniel, laforge, osmith. Hello Jenkins Builder, daniel, laforge, osmith, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/c/osmo-pcap/+/39265?usp=email to look at the new patch set (#9). The following approvals got outdated and were removed: Code-Review+1 by osmith, Verified+1 by Jenkins Builder Change subject: client: Support generation of pcapng file format ...................................................................... client: Support generation of pcapng file format Support transmitting traffic recordings in pcapng towards osmo-pca-server based on VTY config. Old libpcap's file format (.pcap) is still left as default to to keep backward compatibility towards older osmo-pcap-server instances still running. Furthermore, .pcap support is more extended than .pcapng since it's way older. When pcapng file format is selected, osmo-pcap-client will use the same osmo-pcap protocol used in pcap mode, but the payloads encoded in messages will be in pcapng file format instead of pcap. This means, for LINK_HDR message type it will encode a file header with a payload consisting of 1 SHB + N IDB blocks (one for each network interface being monitored). Upon each packet recording received from the monitoring interface, it will encode a LINK_DATA message containing a payload with a encoded pcapng block. This works mostly transparetly on the osmo-pcap-server side, since in general it handles those payloads mostly transparently. Only some sanity checks will need to be updated and improve there to account for the new pcapng blocks. This in turn requires identifying pcap vs pcapng format being recieved from the client, and also placing the proper file suffix when creating the file. This will be done in a follow-up patch modyfing the server side. Related: SYS#5822 Change-Id: I3c80518a1e53a1f77e1aca8dfa83f683f9516ad6 --- M doc/manuals/chapters/client.adoc M include/osmo-pcap/osmo_pcap_client.h M include/osmo-pcap/osmo_pcap_file.h M src/osmo_client_core.c M src/osmo_client_network.c M src/osmo_client_vty.c M src/osmo_pcap_file.c 7 files changed, 534 insertions(+), 10 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/65/39265/9 -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39265?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newpatchset Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I3c80518a1e53a1f77e1aca8dfa83f683f9516ad6 Gerrit-Change-Number: 39265 Gerrit-PatchSet: 9 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: daniel <dwillmann(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: osmith <osmith(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: daniel <dwillmann(a)sysmocom.de>

5 months, 1 week

1
0
0 0

[L] Change in osmo-pcap[master]: server: Support storing of pcapng format

by pespin

Attention is currently required from: daniel, laforge, osmith, pespin. Hello Jenkins Builder, daniel, laforge, osmith, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/c/osmo-pcap/+/39287?usp=email to look at the new patch set (#5). The following approvals got outdated and were removed: Code-Review+1 by osmith, Verified+1 by Jenkins Builder Change subject: server: Support storing of pcapng format ...................................................................... server: Support storing of pcapng format Improve existing logic doing validation on payload data and generalize the file_hdr field to account for the possibily of receiving recordings in pcapng format. This requires, during LINK_HDR when file header is received, discering whether the recording is being done in pcap vs pcapng, (and in pcapng figuring out the endianness of the file) and then applying validation based on that knowledge. With that knowledge, osmo-pcap-server can now store the pcap(ng) file using the proper suffix for the file. Related: SYS#5822 Change-Id: I275d28ab418a1514fa9c5c7c20f3d831cc6af8bb --- M include/osmo-pcap/osmo_pcap_file.h M include/osmo-pcap/osmo_pcap_server.h M src/Makefile.am M src/osmo_pcap_file.c M src/osmo_server_network.c M tests/rotate_localtime/Makefile.am 6 files changed, 333 insertions(+), 46 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/87/39287/5 -- To view, visit https://gerrit.osmocom.org/c/osmo-pcap/+/39287?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email Gerrit-MessageType: newpatchset Gerrit-Project: osmo-pcap Gerrit-Branch: master Gerrit-Change-Id: I275d28ab418a1514fa9c5c7c20f3d831cc6af8bb Gerrit-Change-Number: 39287 Gerrit-PatchSet: 5 Gerrit-Owner: pespin <pespin(a)sysmocom.de> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: daniel <dwillmann(a)sysmocom.de> Gerrit-Reviewer: laforge <laforge(a)osmocom.org> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-Attention: osmith <osmith(a)sysmocom.de> Gerrit-Attention: laforge <laforge(a)osmocom.org> Gerrit-Attention: pespin <pespin(a)sysmocom.de> Gerrit-Attention: daniel <dwillmann(a)sysmocom.de>

5 months, 1 week

1
0
0 0

2025

2024

2023

2022

gerrit-log January 2025