22 Dec
2009
22 Dec
'09
9:10 p.m.
Hi all!
I've just added some code to OpenBSC which keeps counters for various events,
such as the number of channel requests, how often we were out of channels,
the number of delivered/submitted sms, paging statistics, etc.
Those counters are available through the VTY for evaluation at runtime.
There is now a "Counters" table in our database schema. It stores those
counters in a generic way. The counters are stored to the database every
minute. Each counter is identified by a string like "net.sms.delivered"
The counters increment all the time, until OpenBSC terminates, when they will
re-start from zero. I did not want to make the counters persistent (i.e. load
them from the database) as it would cofuse users of the VTY.
What's still missing now is some kind of script that uses those counters to
generate something like nice charts. Given the way how the counters are stored
in the database, it should be relatively easy to write a fairly generic tool
that does not need any modification even if we add new counters.
What I'm thinking is something like specifying the specific counters one is
interested in at the command line, maybe also providing a start and end time.
The graphs should then generally show the increments of the counters between
two counter stores in the database. So if e.g. one minute ago the number of
delivered SMS was '10', and now it is '15, then the actual value plotted
in the graph should only be '5'.
The only "difficulty" is to handle restarts of OpenBSC... but that's quite
easy. All counters are monotonically incrementing, i.e. whenever you see a
smaller counter, you can safely assume that openbsc was restarted and that we
simply continue with the next diff, ignoring the negative one.
If anyone is interested in working in such a tool (preferrably in python or
perl, using their respective sqlite3 bindings), it would be greatly
appreciated.
However, please do coordinate on this mailing list... your time is too
precious to be wasted by code duplication.
Thanks in advance,
Harald
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
24 Dec
24 Dec
1:11 a.m.
Is someone already working on this ?
It should be pretty straightforward to pipe this into a RRD database
(rrdtool) and from there, do pretty much any graph you want ...
It will handle wrap around / reset to 0 / differential all by itself.
I'll have a look at this tomorrow if nobody manifests itself as already
working on it.
Sylvain
9:27 a.m.
On Thu, Dec 24, 2009 at 01:11:46AM +0100, Sylvain Munaut wrote:
Is someone already working on this ?
not that I'm aware of.
It should be pretty straightforward to pipe this into
a RRD database
(rrdtool) and from there, do pretty much any graph you want ...
It will handle wrap around / reset to 0 / differential all by itself.
great idea.
I'll have a look at this tomorrow if nobody
manifests itself as already
working on it.
thanks. But please, SMS stuff has higher priority. You can still work on
the rrdtool integration after you've arrived at the event..
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
25 Dec
25 Dec
1:41 a.m.
Hi,
I also thought about RRD, I'm writing on a small python script that gets the
new values from the db (since the last update of the target rrd file) and
updates the rrd file.
At the moment it only supports one data source for a single rrd file, but I'll
add commandline options for the file and counter name.
I think this is the easiest solution, adding a new counter is just creating a
new rrd file and adding a cronjob.
It would be great if someone could post some sample data to do some tests.
cu,
Clemens
Am Donnerstag 24 Dezember 2009 09:27:49 schrieb Harald Welte:
On Thu, Dec 24, 2009 at 01:11:46AM +0100, Sylvain
Munaut wrote:
Is someone already working on this ?
not that I'm aware of.
It should be pretty straightforward to pipe this
into a RRD database
(rrdtool) and from there, do pretty much any graph you want ...
It will handle wrap around / reset to 0 / differential all by itself.
great idea.
I'll have a look at this tomorrow if nobody
manifests itself as already
working on it.
thanks. But please, SMS stuff has higher priority. You can still work on
the rrdtool integration after you've arrived at the event..
3:07 a.m.
Hi Clemens,
On Fri, Dec 25, 2009 at 01:41:06AM +0100, Clemens Hopfer wrote:
At the moment it only supports one data source for a
single rrd file, but I'll
add commandline options for the file and counter name.
I think this is the easiest solution, adding a new counter is just creating a
new rrd file and adding a cronjob.
well, it's also easy to generate a list of the counters from the sql database
by using something like "select unique(name) from counters" and then iterate
over that list...
It would be great if someone could post some sample
data to do some tests.
I cannot send you the full hlr.sqlite3 of my test machine, sorry. but I've
attached the output of 'sqlite3 -csv hlr.sqlite3 "select * from
counters;"'.
The counter values are very low as I've been playing around with few phones and
few events, but it should give you a feeling for it.
btw: Will you be at 26C3?
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
3:56 a.m.
Am Freitag 25 Dezember 2009 03:07:34 schrieb Harald Welte:
thanks,
that's everything I wanted :-)
Hi Clemens,
On Fri, Dec 25, 2009 at 01:41:06AM +0100, Clemens Hopfer wrote:
hmm, I'm not very familiar with rrdtool, but
afaics there is no direct option
in rrdtool to add a data source after creating a rrd file.
However the Perl RRD bindings actually got a function to do that and there are
several suggestions to dump the rrd file to xml, change that file and recreate
the rrd file but I'm not sure if this is a really good idea...
That's one reason (apart from simplicity) why i only used one data source for
now.
It would be also possible to use a dedicated directory and create RRDs of new
counters on-the-fly.
I'll have a look at the Perl api, although I don't like Perl that much... ;-)
At the moment it only supports one data source
for a single rrd file, but
I'll add commandline options for the file and counter name.
I think this is the easiest solution, adding a new counter is just
creating a new rrd file and adding a cronjob.
well, it's also easy to generate a list of the counters from the sql
database by using something like "select unique(name) from counters" and
then iterate over that list... It would be
great if someone could post some sample data to do some
tests.
I cannot send you the full hlr.sqlite3 of my test machine, sorry. but I've
attached the output of 'sqlite3 -csv hlr.sqlite3 "select * from
counters;"'. The counter values are very low as I've been playing around
with few phones and few events, but it should give you a feeling for it. btw: Will you be at 26C3?
yep, I'll arrive at Saturday afternoon.
4:45 a.m.
Am Freitag 25 Dezember 2009 03:56:37 schrieb Clemens Hopfer:
[..]
hmm, I'm not very familiar with rrdtool, but afaics there is no direct
option in rrdtool to add a data source after creating a rrd file.
However the Perl RRD bindings actually got a function to do that and there
are several suggestions to dump the rrd file to xml, change that file and
recreate the rrd file but I'm not sure if this is a really good idea...
ok,
the perl api is also just dumping the whole rrd file to xml, changing it
and recreating the rrd file.
The xml file seems to be about the factor 10 bigger than the rrd file, i
expected 100 or more.
I'll have a deeper look into the xml structure tomorrow.
Clemens
26 Dec
26 Dec
2:07 a.m.
Hi,
there is one problem with the sqlite db: Afaik sqlite3 doesn't really know
about datatypes apart from the standard types (like integer, text etc..) and
uses a dynamic type system.
So the query for inserting a new counter value to the database does not insert
the timestamp as unix time as expected but as string ( db.c:1037
datetime('now') ).
Something like strftime('%s','now') would make much more sense, rrd also
uses
unix time for updating rrd files.
Am Freitag 25 Dezember 2009 03:07:34 schrieb Harald Welte:
Hi Clemens,
On Fri, Dec 25, 2009 at 01:41:06AM +0100, Clemens Hopfer wrote:
At the moment it only supports one data source
for a single rrd file, but
I'll add commandline options for the file and counter name.
I think this is the easiest solution, adding a new counter is just
creating a new rrd file and adding a cronjob.
well, it's also easy to generate a list of the counters from the sql
database by using something like "select unique(name) from counters" and
then iterate over that list...
It would be great if someone could post some
sample data to do some
tests.
I cannot send you the full hlr.sqlite3 of my test machine, sorry. but I've
attached the output of 'sqlite3 -csv hlr.sqlite3 "select * from
counters;"'. The counter values are very low as I've been playing around
with few phones and few events, but it should give you a feeling for it.
btw: Will you be at 26C3?
5237
days inactive
5241
days old
7 comments
3 participants
participants (3)
-
Clemens Hopfer -
Harald Welte -
Sylvain Munaut