Attention is currently required from: laforge, pespin.
jolly has posted comments on this change by jolly. ( https://gerrit.osmocom.org/c/libosmocore/+/40584?usp=email )
Change subject: Avoid reusing pending buffer; append incoming data instead
......................................................................
Patch Set 4:
(5 comments)
Commit Message:
https://gerrit.osmocom.org/c/libosmocore/+/40584/comment/ff2050e4_17cbe464?… :
PS3, Line 17: This change ensures that newly received data is appended to the existing
> First you say we don't want to reuse the pending buffer, but here you say "This change ensures that […]
Maybe I should state that I don't want to reuse the pending buffer for the read operation. Instead I copy from the buffer that was used for the read operation to the pending buffer.
File src/core/osmo_io.c:
https://gerrit.osmocom.org/c/libosmocore/+/40584/comment/2b7b3610_96fc97ba?… :
PS3, Line 177: /*! convenience wrapper to call msgb_alloc with parameters from osmo_io_fd (with extra size) */
> "with extra size" here doesn't apply anymore?
Done
https://gerrit.osmocom.org/c/libosmocore/+/40584/comment/d54ad17c_59b4ea84?… :
PS3, Line 178: struct msgb *iofd_msgb_alloc2(struct osmo_io_fd *iofd, size_t size)
> This is probably breaking the promise that msgb passed to the app are at least of a certain size con […]
In case the pending buffer is used, the size may be larger, but never smaller. I don't see any problem with that. It happens when reading/receiing. The application should only care about the length of data in the buffer not the size.
https://gerrit.osmocom.org/c/libosmocore/+/40584/comment/3ca7bf60_cd0f8648?… :
PS3, Line 335: * If the pending message is not large enough, create a larger message. */
> I may be wrong, but I'd expect user to configure iofd with a max_size big enough to allow whatever s […]
Imagine the pending buffer is almost full. Now we read extra data that is larger than the remaining space of the pending buffer. In order to merge them, a larger buffer is needed.
https://gerrit.osmocom.org/c/libosmocore/+/40584/comment/f6d10647_ee937061?… :
PS3, Line 344: memcpy(msgb_put(iofd->pending, msgb_length(msg)), msgb_data(msg), msgb_length(msg));
> Can we avoid 2 memcpys (here and above) in the specific code path?
How? The fist memcopy is used, if the pending buffer must be enlarged. The second memcopy will append the newly received data to the pending buffer.
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40584?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: I08df9736ccc5e9a7df61ca6dcf94629ee010752f
Gerrit-Change-Number: 40584
Gerrit-PatchSet: 4
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Gerrit-Attention: pespin <pespin(a)sysmocom.de>
Gerrit-Comment-Date: Wed, 23 Jul 2025 10:46:18 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: pespin <pespin(a)sysmocom.de>
Attention is currently required from: laforge, pespin.
jolly has posted comments on this change by jolly. ( https://gerrit.osmocom.org/c/libosmocore/+/40489?usp=email )
Change subject: Allow io_uring_submit batching just ahead of poll/select
......................................................................
Patch Set 4:
(4 comments)
Patchset:
PS3:
> I'm still not convinced we want to add more envvars. […]
Right now I use the envvars for running TTCN3 tests with different io_uring features. It makes it easier to compare.
File src/core/osmo_io_uring.c:
https://gerrit.osmocom.org/c/libosmocore/+/40489/comment/889cdea7_e7f90d3f?… :
PS2, Line 186: if (!g_io_uring_batch)
> maybe use the OSMO_LIKELY macro for branch prediction optimizaton, as we assume the default is the c […]
Done
https://gerrit.osmocom.org/c/libosmocore/+/40489/comment/afbdc520_829fc90e?… :
PS2, Line 329: if (!g_io_uring_batch)
> same here and further below
Done
File src/core/select.c:
https://gerrit.osmocom.org/c/libosmocore/+/40489/comment/a9058573_f6aff482?… :
PS3, Line 440: osmo_io_uring_submit();
> I'd say it makes more sense to have the OSMO_UNLIKELY if here directly, to avoid an extra call+ret o […]
Done
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40489?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: comment
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: Id34fe2ced32c63d15b14810e145744f7509064cc
Gerrit-Change-Number: 40489
Gerrit-PatchSet: 4
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: laforge <laforge(a)osmocom.org>
Gerrit-CC: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Gerrit-Attention: pespin <pespin(a)sysmocom.de>
Gerrit-Comment-Date: Wed, 23 Jul 2025 10:46:11 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: laforge <laforge(a)osmocom.org>
Comment-In-Reply-To: pespin <pespin(a)sysmocom.de>
Attention is currently required from: jolly, laforge.
Hello Jenkins Builder, laforge, pespin,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/libosmocore/+/40584?usp=email
to look at the new patch set (#4).
The following approvals got outdated and were removed:
Code-Review+1 by laforge, Verified+1 by Jenkins Builder
Change subject: Avoid reusing pending buffer; append incoming data instead
......................................................................
Avoid reusing pending buffer; append incoming data instead
When reading from a stream, a single read may return only part of a
message segment. In such cases, the partial data was stored in
'iofd->pending' and reused for subsequent reads to complete the message.
With upcoming changes that submit multiple read SQEs to io_uring,
each read uses its own pre-submitted buffer. Reusing 'iofd->pending' is
not possible, as the next read buffer is already submitted.
This change ensures that newly received data is appended to the existing
pending buffer, allowing message segments to accumulate until complete.
Change-Id: I08df9736ccc5e9a7df61ca6dcf94629ee010752f
---
M src/core/osmo_io.c
M src/core/osmo_io_internal.h
M src/core/osmo_io_poll.c
M src/core/osmo_io_uring.c
4 files changed, 26 insertions(+), 28 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/84/40584/4
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40584?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: I08df9736ccc5e9a7df61ca6dcf94629ee010752f
Gerrit-Change-Number: 40584
Gerrit-PatchSet: 4
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: jolly <andreas(a)eversberg.eu>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Attention is currently required from: jolly, laforge.
Hello Jenkins Builder,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/libosmocore/+/40489?usp=email
to look at the new patch set (#4).
The following approvals got outdated and were removed:
Verified+1 by Jenkins Builder
Change subject: Allow io_uring_submit batching just ahead of poll/select
......................................................................
Allow io_uring_submit batching just ahead of poll/select
Let's add a mode (enabled via the LIBOSMO_IO_URING_BATCH environment
variable), where we don't call io_uring_submit() after every operation
we add to the submission queue. Rather, do that once before we go into
poll.
This should massively reduce the amount of io_uring_enter() syscalls
we're seeing.
Related: OS#6705
Change-Id: Id34fe2ced32c63d15b14810e145744f7509064cc
---
M src/core/osmo_io_uring.c
M src/core/select.c
2 files changed, 36 insertions(+), 4 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/89/40489/4
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40489?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: Id34fe2ced32c63d15b14810e145744f7509064cc
Gerrit-Change-Number: 40489
Gerrit-PatchSet: 4
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: laforge <laforge(a)osmocom.org>
Gerrit-CC: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: jolly <andreas(a)eversberg.eu>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Attention is currently required from: jolly, laforge.
Hello Jenkins Builder, pespin,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/libosmocore/+/40491?usp=email
to look at the new patch set (#6).
The following approvals got outdated and were removed:
Verified+1 by Jenkins Builder
Change subject: Add multiple messages buffers to struct iofd_msghdr
......................................................................
Add multiple messages buffers to struct iofd_msghdr
This is a prerequisite for transfering multiple buffers with a single
SQE or CQE when using osmo_io with io_uring.
The message buffers at struct iofd_msghr, as well as the io vectors are
arrays. A maximum number is stored, as well as the current number.
Having more than one buffer is optional and the number can be controlled
via environment variable.
Related: OS#6705
Change-Id: I4fb1067de4615cc22cc6caf99b481491e7f2ef92
---
M include/osmocom/core/osmo_io.h
M src/core/osmo_io.c
M src/core/osmo_io_internal.h
M src/core/osmo_io_poll.c
M src/core/osmo_io_uring.c
5 files changed, 48 insertions(+), 16 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/91/40491/6
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40491?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: I4fb1067de4615cc22cc6caf99b481491e7f2ef92
Gerrit-Change-Number: 40491
Gerrit-PatchSet: 6
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: pespin <pespin(a)sysmocom.de>
Gerrit-CC: laforge <laforge(a)osmocom.org>
Gerrit-Attention: jolly <andreas(a)eversberg.eu>
Gerrit-Attention: laforge <laforge(a)osmocom.org>
Attention is currently required from: jolly.
Hello Jenkins Builder,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/libosmocore/+/40493?usp=email
to look at the new patch set (#7).
The following approvals got outdated and were removed:
Verified+1 by Jenkins Builder
Change subject: Add multiple messages buffers to io_uring write operations
......................................................................
Add multiple messages buffers to io_uring write operations
Multiple message buffers can be writen by sending a single SQE when
using io_uring. If there is less data written, the completely written
buffers are removed and the partly written buffers are truncated.
Afterwards they are re-queued for next write operation.
Having more than one buffer is optional and the number can be controlled
via environment variable.
Related: OS#6705
Change-Id: I8c4e0a785cf66becd7fb5b2caf718c9724b56686
---
M src/core/osmo_io.c
M src/core/osmo_io_uring.c
2 files changed, 104 insertions(+), 51 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/93/40493/7
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40493?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: I8c4e0a785cf66becd7fb5b2caf718c9724b56686
Gerrit-Change-Number: 40493
Gerrit-PatchSet: 7
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: jolly <andreas(a)eversberg.eu>
Attention is currently required from: jolly, pespin.
Hello Jenkins Builder, laforge,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/libosmocore/+/40725?usp=email
to look at the new patch set (#3).
The following approvals got outdated and were removed:
Verified+1 by Jenkins Builder
Change subject: Automatically increase io_uring, if too small.
......................................................................
Automatically increase io_uring, if too small.
The ring may be too small to store all SQEs before the kernel can
handle them. If this happens, a new ring is allocated with twice of the
size of the old ring. The old ring will not be destroyed, as it still
contains uncompleted elements. Some of them may never be completed.
A pointer to the current ring will be stored within the msghdr
structure. It is used when cancelling an SQE. The cancellation must be
performed in the same ring where it was created.
It is quite unlikely that the old ring cannot store the cancellation
SQE. If this happens, the kernel is given some time to handle the SQEs.
The initial size of the ring can be changed via environment variable.
Related: OS#6705
Change-Id: Id9230146acc8d54bfd44834e783c31b37bd64bca
---
M src/core/osmo_io_internal.h
M src/core/osmo_io_uring.c
2 files changed, 65 insertions(+), 25 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/25/40725/3
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40725?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: Id9230146acc8d54bfd44834e783c31b37bd64bca
Gerrit-Change-Number: 40725
Gerrit-PatchSet: 3
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: laforge <laforge(a)osmocom.org>
Gerrit-CC: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: jolly <andreas(a)eversberg.eu>
Gerrit-Attention: pespin <pespin(a)sysmocom.de>
Attention is currently required from: jolly.
Hello Jenkins Builder,
I'd like you to reexamine a change. Please visit
https://gerrit.osmocom.org/c/libosmocore/+/40494?usp=email
to look at the new patch set (#7).
The following approvals got outdated and were removed:
Verified+1 by Jenkins Builder
Change subject: Send multiple read/recvfrom SQEs in advance
......................................................................
Send multiple read/recvfrom SQEs in advance
Multiple read or recvfrom operations can be submitted via SQEs when
using io_uring. This allows reading multiple packet / more data between
calls of osmo_select_main() the main loop.
Having more than one SQE submitted is optional and the number can be
controlled via environment variable.
Related: OS#6705
Change-Id: Id50a8900fa2fe6de553e5025feae7e1e8d501e30
---
M src/core/osmo_io_internal.h
M src/core/osmo_io_uring.c
2 files changed, 46 insertions(+), 13 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/libosmocore refs/changes/94/40494/7
--
To view, visit https://gerrit.osmocom.org/c/libosmocore/+/40494?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: libosmocore
Gerrit-Branch: master
Gerrit-Change-Id: Id50a8900fa2fe6de553e5025feae7e1e8d501e30
Gerrit-Change-Number: 40494
Gerrit-PatchSet: 7
Gerrit-Owner: jolly <andreas(a)eversberg.eu>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: pespin <pespin(a)sysmocom.de>
Gerrit-Attention: jolly <andreas(a)eversberg.eu>