7 Feb
2018
7 Feb
'18
10:31 a.m.
On 4. Feb 2018, at 22:35, Holger Freyther
<holger(a)freyther.de> wrote:
Hi,
* SIGCHLD arrives
* Something will be written into one end of a socketpair[1]
* In the python code on wait(2) will be called on every registered process
(https://github.com/python/cpython/blob/3.6/Lib/asyncio/unix_events.py#L819)
there is a way to switch to a child reaper that is using wait(-1.. but even
then we have two issues.
1.) If I use await asyncio.create_subprocess_exec... then we can't tick
with 50ms. On a 60s schedule to start 10k processes there is a delay of +60s.
2.) If I delay the process creation then I can almost keep the schedule most
of the time but we run into buffering exceptions and/or not all tasks are
started within a reasonable amount of time.
Knowing what I know now I would have gone for Go. It is made for the kind of
concurrency where python seems unusable. I hesitated to build my own event
loop but will build something that looks like our C apps in python (single
select, no async/await).
holger