Scaling up virtual bts tests for the BTS test - how I hold it wrong?
After a lot of work[1] the gsm tester can finally: * Start a virtual bts * Mobile/virtphy processes * Complete LUs for 10 MS. The next step (besides having a proper test verdict) is scaling this beyond what a simple physical set-up can provide. Let's go to 100+ BTS and 10k subscribers but somehow I am still holding it wrongly and would like to have feedback to see how to evolve the gsm tester. What do I need to change to scale this up and how to externally parameterize it? * suite.conf: Add one line per virtual bts to reserve (I would prefer to specify type+num) * resources.prod.conf: Add more Virtual BTS. I started to pick IPs from 127.0.1.0/24 as it avoids having to configure special IPs. * register_default_mass.py: I need to somehow know how many BTS (and NITBs) were reserved. Or in the long run the topology of how to connect M BTS to N BSCs? Currently I can run until I get an exception but that is not desirable. What's missing: * High-level API of the SuiteRun to get me the toplogy of the network. It seems undesirable in the specific suite to discover how many BTSs were reserved in suite.conf or what the topology is. * ARFCN usage. Besides the redundancy all my BTS currently have the same ARFCN. There should be an easy way to configure an band+arfcn pool. * IMSI/MSISDN pooling. I would like to specify pools of IMSI/MSISDN pairs (and size) and then draw from it. I needs these to program into the mobile, NITB/HLR/AuC and for client to client SMS transfers. * Configure these capacities from the outside. Changing from 1 to 256 BTS should be a single line (or even a parameter change). Any idea or comment on how to achieve this? cheers holger [1] Hindsight is 20/20 and the difficulty was not adding scripting to mobile but getting the GSM tester to spawn the network and we are unfortunately not done yet.
Hi Holger, On 12/30/18 4:26 PM, Holger Freyther wrote:
What do I need to change to scale this up and how to externally parameterize it?
* suite.conf: Add one line per virtual bts to reserve (I would prefer to specify type+num)
You can already do that, by using the "times" attribute for each object in suite.conf. See for instance what's done in osmo-gsm-tester/suites/gprs/suite.conf: modem: - times: 2 features: - gprs - voice One can similarly do the same to request allocation of 100 sysmoBTS: bts: - type: osmo-bts-sysmo times: 100
* resources.prod.conf: Add more Virtual BTS. I started to pick IPs from 127.0.1.0/24 as it avoids having to configure special IPs.
I'd like to be able to specifcy pools of objects in resources.conf so we don't need to add 1 line per object there. In the case of IP addresses, specifying a subnet range and let osmo-gsm-tester to expand that into a set of objects at runtime.
* register_default_mass.py: I need to somehow know how many BTS (and NITBs) were reserved. Or in the long run the topology of how to connect M BTS to N BSCs? Currently I can run until I get an exception but that is not desirable.
Yeah for now the way to do it is to ask for one until you get the specific exception thrown when there's no more objects of that type or matching the specifications. Fine for me to add more APIs to fetch this kind of information more easily.
What's missing:
* High-level API of the SuiteRun to get me the toplogy of the network. It seems undesirable in the specific suite to discover how many BTSs were reserved in suite.conf or what the topology is.
See my last text above.
* ARFCN usage. Besides the redundancy all my BTS currently have the same ARFCN. There should be an easy way to configure an band+arfcn pool.
There's a task for that, see https://osmocom.org/issues/2230. Unfortunately the algorithm to allocate ARFCN from available ones is a bit complex (lots of restrictions) and wasn't really required so far, so it's not yet implemented. As a workaround, you may want to add a modifier for ARFCN, similar to what is already done for other variables in osmo-gsm-tester/example/scenrios/mod-bts0-*.conf So you could specify manually a different ARFCN for each bts (number of elements in object modifiers must match number of elements in the resources in suite.conf). Let's say you want 3 BTS with 3 differnet arfcn, 2 of them in 1800 band and 1 in 900 band: [suite.conf] resources: ip_address: - times: 4 #or as much as needed bts: - band: GSM-1800 times: 2 - band: GSM-900 modifiers: bts: - arfcn: 868 - arfcn: 870 - arfcn: 124 # or whatever is valid for GSM900 * Band attribute should actually be reworked to be a list of supported bands for that BTS, then each BTS in resources.conf have a list with all bands supported by that BTS. * I don't think arfcn modifier is there yet, but not sure now. Grep for "modifier" in suite.py and resource.py to see how it is implemented.
* IMSI/MSISDN pooling. I would like to specify pools of IMSI/MSISDN pairs (and size) and then draw from it. I needs these to program into the mobile, NITB/HLR/AuC and for client to client SMS transfers.
See my comments regarding IP addr pools. Fine for me, but I'm not sure if its needed in this case, since there's already methods to allocate sequentially incrementing MSISDN and IMSI.
* Configure these capacities from the outside. Changing from 1 to 256 BTS should be a single line (or even a parameter change).
Unfortunately these kind of capacities are so far fixed by the suite.conf. If you want to do tests with different numbers, you can manually change the "times" attribute in there, or put a big number and only use a subset of them when running the test in the suite. Regards, Pau -- - Pau Espin Pedrol <pespin@sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte
On 30. Dec 2018, at 23:21, Pau Espin Pedrol <pespin@sysmocom.de> wrote:
Hi Holger,
Hey Pau, thank you for the instant response!
One can similarly do the same to request allocation of 100 sysmoBTS:
bts: - type: osmo-bts-sysmo times: 100
ah great! That's a good interim solution.
I'd like to be able to specifcy pools of objects in resources.conf so we don't need to add 1 line per object there. In the case of IP addresses, specifying a subnet range and let osmo-gsm-tester to expand that into a set of objects at runtime.
+1. Do you have a timeline or open issue for it? Besides the IP pool I have needs for a combined IMSI/MSISDN pool
* register_default_mass.py: I need to somehow know how many BTS (and NITBs) were reserved. Or in the long run the topology of how to connect M BTS to N BSCs? Currently I can run until I get an exception but that is not desirable.
Yeah for now the way to do it is to ask for one until you get the specific exception thrown when there's no more objects of that type or matching the specifications. Fine for me to add more APIs to fetch this kind of information more easily.
There are two parts to this. The first is convenience API but to me the more important is topology. I would like to build the network based on the suite.conf and in the python code. As a first proposal (but we require a lot more thinking) what about something like: hlr: <- define a HLR. Multiple might be possible -subs: times: 100000 pool: <- define IMSI/MSISDN pool -name: Foo imsi_start: XXXX msisdn_start: XXXX kind: draw_at_random times: 10k hlr: ... msc: <- MSCs connecting to it? High level or scope to HLR? type: nitb bsc: <- BSCs connecting to (void if NITB is used) times: 5 -bts: <- BTSs type: <- ... virtual_mobiles: <- borrowing from the HLRs? ... What we can't do with the above is simulate movements of subscribers (but we can't do that easily right now and can review it if that becomes a genuine requirement). We currently need to hardcode number of hlrs to one but that seems reasonable. One benefit is that the same test would work for both NITB and BSC/MSC.
* ARFCN usage. Besides the redundancy all my BTS currently have the same ARFCN. There should be an easy way to configure an band+arfcn pool.
So you could specify manually a different ARFCN for each bts (number of elements in object modifiers must match number of elements in the resources in suite.conf). Let's say you want 3 BTS with 3 differnet arfcn, 2 of them in 1800 band and 1 in 900 band:
I will give it a try. Please be aware that ARFCNs do not uniquely identify a band.
* Configure these capacities from the outside. Changing from 1 to 256 BTS should be a single line (or even a parameter change).
Unfortunately these kind of capacities are so far fixed by the suite.conf. If you want to do tests with different numbers, you can manually change the "times" attribute in there, or put a big number and only use a subset of them when running the test in the suite.
I am thinking of a xpath like command line parameter --suite_override=hlr[0].bts[all].type=osmo-bts-virtual --suite_override=hlr[0].bts.times=100 what do you think? holger
Hi Holger,
I'd like to be able to specifcy pools of objects in resources.conf so we don't need to add 1 line per object there. In the case of IP addresses, specifying a subnet range and let osmo-gsm-tester to expand that into a set of objects at runtime.
+1. Do you have a timeline or open issue for it? Besides the IP pool I have needs for a combined IMSI/MSISDN pool
No timeline since I could workaround that myself so far by adding new IP addresses manually. I tend to implement this kind of new features only once I find I am really blocked or constrained by previous system when trying to add new kind of tests (or more complex ones).
There are two parts to this. The first is convenience API but to me the more important is topology. I would like to build the network based on the suite.conf and in the python code. As a first proposal (but we require a lot more thinking) what about something like:
hlr: <- define a HLR. Multiple might be possible -subs: times: 100000 pool: <- define IMSI/MSISDN pool -name: Foo imsi_start: XXXX msisdn_start: XXXX kind: draw_at_random times: 10k hlr: ...
msc: <- MSCs connecting to it? High level or scope to HLR? type: nitb bsc: <- BSCs connecting to (void if NITB is used) times: 5 -bts: <- BTSs type: <- ...
virtual_mobiles: <- borrowing from the HLRs? ...
What we can't do with the above is simulate movements of subscribers (but we can't do that easily right now and can review it if that becomes a genuine requirement). We currently need to hardcode number of hlrs to one but that seems reasonable.
One benefit is that the same test would work for both NITB and BSC/MSC.
TBH I don't like the idea of making the suite/scenario yml file structure a lot more complex, I think current status is quite complex and makes it already difficult to gasp how to put everything together. The kind of stuff that you intend to do here can already be done far more easily by using (or extending) the python test API. That's mostly what the test does anyway: Setting up a specific topology with a pre-allocated sub-set of objects, and then do some actions on that. If you require several similar tests but with different number of objects, you can abstract that by using the "lib" feature of a suite. See for instance osmo-gsm-tester.git/suites/gprs/lib/testlib.py and its users in osmo-gsm-tester.git/suites/gprs/lib/iperf3*.py
I will give it a try. Please be aware that ARFCNs do not uniquely identify a band.
Yes, that's why in osmo-gsm-tester initial work around this topic expects an arfcn to be actually a dictionary with ARFCN+BAND iirc.
* Configure these capacities from the outside. Changing from 1 to 256 BTS should be a single line (or even a parameter change).
Unfortunately these kind of capacities are so far fixed by the suite.conf. If you want to do tests with different numbers, you can manually change the "times" attribute in there, or put a big number and only use a subset of them when running the test in the suite.
I am thinking of a xpath like command line parameter
--suite_override=hlr[0].bts[all].type=osmo-bts-virtual --suite_override=hlr[0].bts.times=100
This kind of feature is similar to what scenarios provide already. The issue is still though that in order to sanely combine and override yml cfg files (lists, tuples, dictionaries, etc.) we require some restrictions in order to make it "sane" and usable for most cases. For instance right now we handle lists of "non-complex" attributes (integers, strings, etc.) as sets when combining them, but we expect lists of dictionaries to be the same size when merging them to have meaningful result (so order of items in list is important when combining or overwriting them). Lists of objects are expanded before merging suites and scenarios, so the "times" attribute is removed and item put in place in the list before combine() and override() operations are done. You may want to look at git history to understand reasons behind current system. Maybe one could use scenario modifiers to change the amount of objects from the suite.conf, that's something to explore, but I'd need to invest some time testing and reading the code too. IIRC summary is that regular suite.conf + scenarios are merged using combine(), while scenario modifiers are merged using override(). Something like [suite.conf] bts: - times: 1 [mod-num-bts.cfg] modifiers: bts: - times: 2 I think to that in order to override() you also need same list size, due to sanity checks. We could perhaps add a new "section" in scenarios which overpasses these checks, let's call it "appends" or whatever, which let's you add new requires resources to the suite. These are my 5cents, not an ultimate answer but just some advises and ideas. Regards, Pau -- - Pau Espin Pedrol <pespin@sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte
On 3. Jan 2019, at 15:13, Pau Espin Pedrol <pespin@sysmocom.de> wrote:
Hi! let me spin this into a new thread.
What we can't do with the above is simulate movements of subscribers (but we can't do that easily right now and can review it if that becomes a genuine requirement). We currently need to hardcode number of hlrs to one but that seems reasonable. One benefit is that the same test would work for both NITB and BSC/MSC.
TBH I don't like the idea of making the suite/scenario yml file structure a lot more complex, I think current status is quite complex and makes it already difficult to gasp how to put everything together.
The kind of stuff that you intend to do here can already be done far more easily by using (or extending) the python test API. That's mostly what the test does anyway: Setting up a specific topology with a pre-allocated sub-set of objects, and then do some actions on that.
SCNR. This sounds too much like "just write more code". ;)
If you require several similar tests but with different number of objects, you can abstract that by using the "lib" feature of a suite. See for instance osmo-gsm-tester.git/suites/gprs/lib/testlib.py and its users in osmo-gsm-tester.git/suites/gprs/lib/iperf3*.py
The number of "objects" is secondary. Let's say 40k subscribers, 256 BSCs, 512 BTS. The numbers are constant but there are many ways a network can be organized. And many present a nice problem/failure potential for our software stack. But what I want to stretch here. The topology does not matter to the success criteria (99% of subs manage to do X within Y units of time) or execution of the test (start mobiles and ask them to do stuff). If the topology does not matter, why would I want to change the test code? I want the suite to give me a configured network and then use the functional identities (e.g. connect to the SMPP interfaces, etc). If it is in a suite.conf, topology.conf or a RPC call is secondary to me. Does this change anything for you? holger
On 4. Jan 2019, at 02:36, Holger Freyther <holger@freyther.de> wrote:
Hey Pau! Please take your time to respond to this. I understand that this will stall for some time now.
Does this change anything for you?
When looking at the p4[1][2] examples I noticed they are in a similar situation. E.g. when building a network of 100 switches, it doesn't say how they are connected (I would make it one big loop). I think we are in a similar situation and have a need to define topology (without having to rewrite the test). In the interim I will make sure we can execute our single test properly. holger [1] A language to program network flow/switches. [2] https://github.com/p4lang/tutorials/blob/dc08948a344c6ff26af47d2a2447800cab9...
participants (2)
-
Holger Freyther -
Pau Espin Pedrol