On 4. Feb 2018, at 22:35, Holger Freyther holger@freyther.de wrote:
Hi,
- SIGCHLD arrives
- Something will be written into one end of a socketpair[1]
- In the python code on wait(2) will be called on every registered process
(https://github.com/python/cpython/blob/3.6/Lib/asyncio/unix_events.py#L819)
there is a way to switch to a child reaper that is using wait(-1.. but even then we have two issues.
1.) If I use await asyncio.create_subprocess_exec... then we can't tick with 50ms. On a 60s schedule to start 10k processes there is a delay of +60s.
2.) If I delay the process creation then I can almost keep the schedule most of the time but we run into buffering exceptions and/or not all tasks are started within a reasonable amount of time.
Knowing what I know now I would have gone for Go. It is made for the kind of concurrency where python seems unusable. I hesitated to build my own event loop but will build something that looks like our C apps in python (single select, no async/await).
holger