Hey neels, pespin,
while looking at configuration handling for the MS driver I started to have sone questions about scenarios. I see that we have multiple yaml files and through scenarios can control ciphering, the timeslot configuration, which BTS model to test, and more.
What would it take to get this abstraction one step further? When I diff the aoio_sms and sms suite the difference is in setup (and then naming of msc vs. nitb)? Wouldn't this be something we can get into the scenario as well? The requirements for this test are two connected and routable subscribers?
low priority thing but what do you think?
holger
On Mon, Mar 05, 2018 at 11:15:46PM +0000, Holger Freyther wrote:
Hey neels, pespin,
while looking at configuration handling for the MS driver I started to have sone questions about scenarios. I see that we have multiple yaml files and through scenarios can control ciphering, the timeslot configuration, which BTS model to test, and more.
What would it take to get this abstraction one step further? When I diff the aoio_sms and sms suite the difference is in setup (and then naming of msc vs. nitb)? Wouldn't this be something we can get into the scenario as well? The requirements for this test are two connected and routable subscribers?
My plan was to instead provide convenience "set up the network" functions to call from the test.
The point is we don't always know what kind of net programs we want to start, in which order, how. In most cases we do start it up so that it works, but some tests may want to deliberately omit one component, start and stop it again, and so forth. So it's for me not a part in the scenario; the scenario is all about: which hardware and which resources do we get. The "what do we run" should be completely in the test script.
And about those "shortcut" functions to setup the standard network, we just never got around to it and went with copy paste for the time being.
~N
On Tue, Mar 06, 2018 at 10:33:10AM +0100, Neels Hofmeyr wrote:
My plan was to instead provide convenience "set up the network" functions to call from the test.
Just remembered: also, I initially had the idea of being able to have net components running across tests. The idea was to have one, like, "0_setup" test that would start test components, and pass ownership of the running processes a level up to the scenario, so that the processes remain running and are available for all of the remaining tests of that scenario (as it is now, all processes are shut down when the test case ends).
The only benefit here is that a number of test cases don't each setup and tear down the core net, so they would be a bit faster. And there's also the drawback of complexity about handling situations where one test manages to crash a program which then affects subsequent tests -- but I guess I would just let all remaining tests fail then.
So I guess all I would implement would be some convenience function that allows reducing code dup across the test functions, for tests that just want a working network with the modems subscribed, ready for testing.
~N
On 6. Mar 2018, at 10:59, Neels Hofmeyr nhofmeyr@sysmocom.de wrote:
The only benefit here is that a number of test cases don't each setup and tear down the core net, so they would be a bit faster. And there's also the drawback of complexity about handling situations where one test manages to crash a program which then affects subsequent tests -- but I guess I would just let all remaining tests fail then.
For optimization the state of CRIU might be good enough. Once a network is up you might be able to take a snapshot (and resume it). But we probably don't have this problem yet.
So I guess all I would implement would be some convenience function that allows reducing code dup across the test functions, for tests that just want a working network with the modems subscribed, ready for testing.
Makes sense to solve it like this. I haven't looked at TTCN-3 test composition but I do like the Phexample[1] approach. In one test one can use the result of other tests. E.g. something like this:
smpp = suite.given_a_configured_network().smpp_interface() smpp.submit_sm...
Anyway... Talk is cheap. Given that a scenario is the merged config, would you take changes to accept the rename (scenario->config)?
holger
[1] http://smalltalkthoughts.blogspot.hk/2009/11/phexample-because-examples-expa...
On Tue, Mar 06, 2018 at 05:25:13PM +0000, Holger Freyther wrote:
Anyway... Talk is cheap. Given that a scenario is the merged config, would you take changes to accept the rename (scenario->config)?
A scenario is mainly a selection of which hardware (resources) to use. The finalization of the config templates is just one result of the chosen scenario. I still prefer "scenario" :)
~N
-
Holger Freyther -
Neels Hofmeyr