Hi Neels,
On Mon, Sep 28, 2020 at 10:25:48PM +0200, Neels Hofmeyr wrote:
On Sat, Sep 26, 2020 at 09:52:51PM +0200, Harald Welte
wrote:
> The other idea would be to do our own
buffering using osmo_wqueue() on
[...]
I just implemented that. Results are promising.
I'm wondering, when logging to a wqueue, you are measuring the timing of adding
the log statement to the queue? Or also the write operation via select() later?
I'm only measuring the delay of adding the log statement to the queue.
Working off the logging wqueue could also delay
handling pending incoming
packets? So maybe it just moves latency, at least in high load situations with
a lot of incoming work pending?
For sure, that write will take some time (later). However,
* we only process one log write per select() loop, so we have some kind of fair
sharing of opportunity with incoming packets
* I would definitely expect those writes to take less time, as contrary to right
now, the file descriptors are non-blocking.
I'm happy to do some additional benchmarking if you'd like, but I a non-blocking
write - as the name implies - doesn't block and hence we shouldn't see any
significant
delays. I would expect this to be in the < 1ms order of magnitude, compared to the
multiple-ms to multiple hundreds of ms category. That is of course assuming that
the CPU has time to schedule the process again after the syscall completes. But
we have to assume that is the case; if your CPU is overloaded then of course you
will see higher latencies, no matter what.
When we do a select() for, say, an incoming message,
and that creates a lot of
interesting logging. We'll only actually see the logging on some output in
subsequent select() loop iterations, right?
yes, that's correct. but only one log statements per select() loop iteration to
ensure there is no starving by logging.
And if we get a program crash in between? Then the
stderr (for example) won't
show the final logging leading up to the crash, right?
correct.
So maybe there should be some trivial cfg or cmdline
switch that changes over
to non-wqueue logging at least for debugging?
fine with me.
--
- Harald Welte <laforge(a)osmocom.org>
http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)