6 Mar
2018
6 Mar
'18
12:15 a.m.
Hey neels, pespin,
while looking at configuration handling for the MS driver I started to have sone questions
about scenarios. I see that we have multiple yaml files and through scenarios can control
ciphering, the timeslot configuration, which BTS model to test, and more.
What would it take to get this abstraction one step further? When I diff the aoio_sms and
sms suite the difference is in setup (and then naming of msc vs. nitb)? Wouldn't this
be something we can get into the scenario as well? The requirements for this test are two
connected and routable subscribers?
low priority thing but what do you think?
holger
6 Mar
6 Mar
10:33 a.m.
On Mon, Mar 05, 2018 at 11:15:46PM +0000, Holger Freyther wrote:
Hey neels, pespin,
while looking at configuration handling for the MS driver I started to have sone
questions about scenarios. I see that we have multiple yaml files and through scenarios
can control ciphering, the timeslot configuration, which BTS model to test, and more.
What would it take to get this abstraction one step further? When I diff the aoio_sms and
sms suite the difference is in setup (and then naming of msc vs. nitb)? Wouldn't this
be something we can get into the scenario as well? The requirements for this test are two
connected and routable subscribers?
My plan was to instead provide convenience "set up the network" functions
to call from the test.
The point is we don't always know what kind of net programs we want to
start, in which order, how. In most cases we do start it up so that it
works, but some tests may want to deliberately omit one component, start
and stop it again, and so forth. So it's for me not a part in the
scenario; the scenario is all about: which hardware and which resources do
we get. The "what do we run" should be completely in the test script.
And about those "shortcut" functions to setup the standard network, we
just never got around to it and went with copy paste for the time being.
~N
11:59 a.m.
On Tue, Mar 06, 2018 at 10:33:10AM +0100, Neels Hofmeyr wrote:
My plan was to instead provide convenience "set
up the network" functions
to call from the test.
Just remembered: also, I initially had the idea of being able to have net
components running across tests. The idea was to have one, like, "0_setup"
test that would start test components, and pass ownership of the running
processes a level up to the scenario, so that the processes remain running
and are available for all of the remaining tests of that scenario (as it
is now, all processes are shut down when the test case ends).
The only benefit here is that a number of test cases don't each setup and
tear down the core net, so they would be a bit faster. And there's also
the drawback of complexity about handling situations where one test
manages to crash a program which then affects subsequent tests -- but I
guess I would just let all remaining tests fail then.
So I guess all I would implement would be some convenience function that
allows reducing code dup across the test functions, for tests that just
want a working network with the modems subscribed, ready for testing.
~N
6:25 p.m.
On 6. Mar 2018, at 10:59, Neels Hofmeyr
<nhofmeyr(a)sysmocom.de> wrote:
The only benefit here is that a number of test cases
don't each setup and
tear down the core net, so they would be a bit faster. And there's also
the drawback of complexity about handling situations where one test
manages to crash a program which then affects subsequent tests -- but I
guess I would just let all remaining tests fail then.
For optimization the state of CRIU might be good enough. Once a network
is up you might be able to take a snapshot (and resume it). But we probably
don't have this problem yet.
So I guess all I would implement would be some
convenience function that
allows reducing code dup across the test functions, for tests that just
want a working network with the modems subscribed, ready for testing.
Makes sense to solve it like this. I haven't looked at TTCN-3 test
composition but I do like the Phexample[1] approach. In one test one
can use the result of other tests. E.g. something like this:
smpp = suite.given_a_configured_network().smpp_interface()
smpp.submit_sm...
Anyway... Talk is cheap. Given that a scenario is the merged config, would
you take changes to accept the rename (scenario->config)?
holger
[1] http://smalltalkthoughts.blogspot.hk/2009/11/phexample-because-examples-exp…
7 Mar
7 Mar
1:46 p.m.
On Tue, Mar 06, 2018 at 05:25:13PM +0000, Holger Freyther wrote:
Anyway... Talk is cheap. Given that a scenario is the
merged config, would
you take changes to accept the rename (scenario->config)?
A scenario is mainly a selection of which hardware (resources) to use.
The finalization of the config templates is just one result of the chosen scenario.
I still prefer "scenario" :)
~N
2258
days inactive
2260
days old
4 comments
2 participants
participants (2)
-
Holger Freyther -
Neels Hofmeyr