libosmocore stderr log performance / blocking

[Harald Welte]

one more follow-up:

On Tue, Sep 29, 2020 at 11:04:49AM +0200, Harald Welte wrote:

> > And if we get a program crash in between? Then the stderr (for example) won't
> > show the final logging leading up to the crash, right?
> 
> correct.

The only way around that would be to pus the job of 'flushing the queue' to
a separate process.  So the log messages would be sent over a socket (or in
shared memory, if you want to optimize further), and that separate process then
would take care of flushing/draining the queue even after the process exited.

To get back down to earth, I think the stderr/file write_queue would
usually only contain one element to begin with.  Only in the exceptional
case of some stalling, we'd have some log messages queued up.

The fact that we currently only stall occasionally when using blocking writes
shows us how infrequent the existing stdlib buffer (or pipe buffer) is
insufficient.  That gives us an idea about how infrequent we would have
a significant size of a write queue (in excess of the current stdlib or
pipe buffering).

In the "ideal world" we would probably have a system that is somewhere
in between of tht two extremes of
* old behavior: always a synchronous, potentially blocking write
* new behavior: always asynchronous, queued non-blocking write

So what I think would be good is:

* If the write_queue is empty, attempt a synchronous, non-blocking write
  to stderr/file.
* Only if that write returns short (in the extreme case '0'), enqueue the
  [remainder of the] log message to the write_queue for delayed storage.

This way we should get the best of both worlds: More or less unmodified
behavior for 99% of all cases, while avoiding any delays in those few
situations where right now we are stalling/blocking due to a full stdio buffer.

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)