Making NITB Database code async

[Holger Hans Peter Freyther]

Hi all,

I wonder what is the easiest way to make our database code async. Looking at
our tables and code we do not seem to have a very complicated use.

Problem:
	libdbi makes queries to SQLite that kills performance (e.g. every
	SELECT sends out pragma queries for every column to be queried). We
	have a hacked up libdbi from the last congress.

	We have some issues inserting new data into the database (e.g. the db
	being locked). In one way or another we have some issues there
	(either nitb will block, or the external process will...)

	When moving to postgres and a database on a different system we will
	have to deal with latencies and round trip time, but our code is sync.


Goal:
	Most of our setting is 'set and forget'. We don't have complicate
	transactions (e.g. we don't have two nitb's setting the LAC of a
	subscriber, or wouldn't really care).

	When updating the in-memory representation, we don't mind how long it
	will take until it hits the disk on the remote system.


Proposal:
	Also setting/storing will can remain like they are, the return value
	does only mean that a request has been sent.

	Only loading the subscriber, SMS, need to be asynchronous.


M1:
	Change the gsm_subscriber code to load a subscriber asynchronously,
	this will mostly touch the gsm48 code and the VTY area. On load one
	can specify a callback.

M2:
	Change the SMS code to load a SMS asynchronously, this will mostly
	change the SMS queue, some parts of the SMS submit (e.g. to check if
	there are still SMS to be sent) and the submit in the VTY

M3:
	Probably create our own DB abstraction and provide a SQLite3 and
	Postgres backend using the native API of both of them (and getting rid
	of the libdbi and gdb issue)


biggest problem:
	What do we do with the DB queries done from VTY that are 	
	asynchronous, e.g to confirm a SMS has been stored? Do we care about
	it? Do we turn this to a 'notification'?


comments
	holger