4 Apr
2015
4 Apr
'15
9:05 a.m.
Dear Pablo, Ciaby,
from what I see we have the following OpenBSC/OS integration.
* OpenBSC wants to send some RSL data and calls abis_rsl_sendmsg
** abis_rsl_sendmsg entails the msgb into a queue and informs
the lower layer driver (ipa in our case)
** ipa sets the when to ~= BSC_FD_WRITE
* code returns
* OpenBSC runs select for all fds/timers..
* libosmocore dispatches fd's
** libosmo-abis/ipaccess.c will try to write a single message
** libosmo-abis/ipaccess.c will set the BSC_FD_WRITE again if
the queue is not empty
* Linux/TCP will run nagle to combine these messages
* OpenBSC runs select for all fds/timers...
....
The integration does work but appears to be a bit painful for
busy and high latency links. E.g. OpenBSC starts timers when
abis_rsl_sendmsg is invoked which might just be a little bit
later.
I wonder if there is a better way? For reading we could read
until -EWOULDBLOCK but then this might not be too fair for
the other parts of the software. For writing we might end up
in the situation where a write only partially succeeds and we
need to remember how much of the msgb to write next... So we
would need to do ioctl's to check how much space is left in
the send buffer
Do you have ideas? comments? is it a non issue? is it something
we can do better? Is there a TCP mode where a "write" either
fully succeeds or fails with -ENOSPC or such?
holger
5 Apr
5 Apr
3:46 p.m.
Hi Holger, Pablo, Ciaby,
On Sat, Apr 04, 2015 at 09:05:20AM +0200, Holger Hans Peter Freyther wrote:
* Linux/TCP will run nagle to combine these messages
we could easily disable nagle if that's what we want to avoid
latencies [see below]
I wonder if there is a better way? For reading we
could read
until -EWOULDBLOCK but then this might not be too fair for
the other parts of the software.
In terms of msgb reading, I think a fixed upper limit might make sense,
i.e. read up to [configurable] N messages in one BSC_FD_READ callback.
For writing we might end up in the situation where a
write only
partially succeeds and we need to remember how much of the msgb to
write next... So we would need to do ioctl's to check how much space
is left in the send buffer
Even if you do the ioctl(SICOUTQ) / getsockopt(SO_SNDBUF), what would
you do next? i.e. if you have determined not to write the msgb due to
insufficient socket buffer space. If you then leave BSC_FD_WRITE set,
then your next select will immediately return, and you have a potential
to busy-loop until finally the buffer space is available. In case the
remote end / network link is stalled this would be quite dangerous.
If you don't set BSC_FD_WRITE, then you wouldn't notice once the buffer
has sufficient space again. If you did periodic checks, then the
latency introduced by those checks are probably counter-productive.
So I think we will have to deal with partial writes from the IPA msgb
sending code, just like the IPA msgb receiving code has to deal with
partial reads.
Do you have ideas? comments? is it a non issue? is it
something
we can do better?
It depends probably on the use case. On expensive (e.g. sat) links you
probably want nagle to combine multiple packets. In all other cases
disabling nagle is probably a good idea to get started.
Is there a TCP mode where a "write" either
fully succeeds or fails
with -ENOSPC or such?
Not to my knowledge. The proper answer would probably to use DCCP or
SCTP or any other protocol that can provide reliable delivery of packets
(possibly with ordering guarantees) and not abuse a stream protocol
like TCP. This would of course break IPA compatibility... but it might
be an interesting experiment to compare different L4 protocol
performance against TCP for the given use case.
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
11 Apr
11 Apr
2:43 p.m.
Hi!
On Sun, Apr 05, 2015 at 03:46:16PM +0200, Harald Welte wrote:
Looking at the current state of art of the Linux stack, TCP is very
well tuned and there are more people looking into it than those other
protocols.
Hi Holger, Pablo, Ciaby,
On Sat, Apr 04, 2015 at 09:05:20AM +0200, Holger Hans Peter Freyther wrote:
I think the same, batching seems like the way to go to me.
[...]
* Linux/TCP will run nagle to combine these
messages
we could easily disable nagle if that's what we want to avoid
latencies [see below]
I wonder if there is a better way? For reading we
could read
until -EWOULDBLOCK but then this might not be too fair for
the other parts of the software.
In terms of msgb reading, I think a fixed upper limit might make sense,
i.e. read up to [configurable] N messages in one BSC_FD_READ callback. Is there a TCP
mode where a "write" either fully succeeds or fails
with -ENOSPC or such?
Not to my knowledge. The proper answer would probably to use DCCP or
SCTP or any other protocol that can provide reliable delivery of packets
(possibly with ordering guarantees) and not abuse a stream protocol
like TCP. This would of course break IPA compatibility... but it might
be an interesting experiment to compare different L4 protocol
performance against TCP for the given use case.
3325
days inactive
3332
days old
2 comments
3 participants
participants (3)
-
Harald Welte -
Holger Hans Peter Freyther -
Pablo Neira Ayuso