hnbgw counter thread: another complication [solved]

So, we've been discussing IRL about the osmo-hnbgw implementation of interacting with nftables from a separate thread. The idea now is that a separate thread gets the results from nftables, caches them and enqueues them, so that the main thread can update the hnbgw counters in-sync. A second separate thread does nft add/del rule maintenance. There is another complication, and I want to make sure we agree on the solution: Problem: to delete a rule, I need to retrieve its 'handle' first. With the amount of threading we want, this becomes a bit complex. So. We 'add rule' and 'delete rule', as hNodeB attach and detach. But, to be able to 'delete rule', we need to retrieve an id that nftables assigned to this rule when it was added, the 'handle': add rule inet osmo-hnbgw gtpu-ul ip saddr 1.2.3.4 counter comment "ul:1-2-3-4"; ... list table inet osmo-hnbgw --> table inet osmo-hnbgw { chain gtpu-ul { ip saddr 1.2.3.4 counter packets 5 bytes 100 comment "ul:1-2-3-4" # handle 23; } ^^^^^^^^^ } ... delete rule inet osmo-hnbgw gtpu-ul handle 23; ^^^^^^^^^ AFAICT I cannot set a handle right from the start. I can also not delete a rule by just stating it again as it is, nor by using the 'comment' as a handle. nft wants a 'handle', period. Does anyone know better? Every time we read counters, osmo-hnbgw also retrieves the handles for each rule, and correlates via the 'comment'. Similarly to counters, we can only get all handles at once, via a complete table dump (possibly inefficient). It works out fine in most practical cases. But the complex corner case is when an hNodeB detaches quickly, before the counters had a chance to run and update the hNodeB's state, including the handle. In the current agreement on our implementation with two threads for nft, with queues back to the main thread, it would go something like this, to avoid races: main() nft_maintenance_thread() nft_read_counters_thread() | | | |---add hNodeB-------->| | |<--ok-----------------| | X | | |---del hNodeB-------->| | | [rule handle unknown!] | ..|<--EAGAIN-------------| | . | | | . |---do counters--------------------------------->| . | | [work] . | | [work] . | | [work] . |<--update-counters-and-handles------------------| . | | | ..|---del hNodeB-------->| | | [delete rule] | |<--done---------------| | Simplifying a bit... main() nft_maintenance_thread() nft_read_counters_thread() | | | |---add hNodeB-------->| | |<--ok-----------------| | X | | [want: del hNodeB] | | [rule handle unknown!] | | [set some "del" flag] | | . | | | . |---do counters--------------------------------->| . | | [work] . | | [work] . | | [work] . |<--update-counters-and-handles------------------| . | | | [del flag!] | | |---del hNodeB-------->| | | [delete rule] | |<--done---------------| | I'm not liking very much the amount of complexity spawning here, but this last approach is ok I guess, do you agree? i.e. when the handle is not set yet, set a flag on the hNodeB, and trigger the 'delete' the next time the counter thread reports back a handle for the hNodeB. ~N -- - Neels Hofmeyr &lt;nhofmeyr(a)sysmocom.de&gt; https://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Siemensstr. 26a * 10551 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte