From laforge at gnumonks.org  Sat Aug 11 14:06:26 2018
From: laforge at gnumonks.org (Harald Welte)
Date: Sat, 11 Aug 2018 16:06:26 +0200
Subject: Proper measurement reporting
In-Reply-To: <CAPEnp=YfwUQ54ru-JqRSGtLd7cuNFo+cO6f1evt6dHn7jM71Wg@mail.gmail.com>
References: <CAPEnp=YfwUQ54ru-JqRSGtLd7cuNFo+cO6f1evt6dHn7jM71Wg@mail.gmail.com>
Message-ID: <20180811140626.GY17129@nataraja>

Hi Vadim,

following up on this as I was browsing through the list of open issues
and ended up on OS#2988 and https://gerrit.osmocom.org/#/c/osmocom-bb/+/7470/

On Fri, Mar 23, 2018 at 08:20:22PM +0700, Vadim Yanitskiy wrote:
> In short, according to the GSM TS 04.08, section 3.4.1.1 "SACCH
> procedures", Measurement Report messages are sent at each possible
> occasion when nothing else has to be sent. In other words, a dummy
> LAPDm fill frame (0x01, 0x03, 0x01, 0x2b, ...) is not applicable here.

this part is clear so far.

> The Calypso PHY (i.e. the firmware) is sending self-composed
> Measurement Reports if there is nothing in transmit queue:
>
> [...]
>
> I am not sure if this is the correct way. Why?
> 
>   - The Measurement Reports coming from the higher
>     layers may contain additional information, such
>     as the neighbour measurements.
> 
>   - The higher layers may spoof indicated TA value,
>     while this code uses the actual one.
> 
>   - Measurement Reporting is already implemented
>     in the higher layers, so this duplicates...

I agree.

> My current ideas are:
> 
>   1) Create a separate L1CTL message, that would be
>      used to carry Measurement Reports. This way
>      we wouldn't need to extract them from the
>      L1CTL_DATA_REQ messages manually.

As sylvain already stated, there doesn't seem to be a need for yet another
extra primitive.  For the MS side, it is a DATA_REQ on the uplink SACCH SAPI.

>   2) Keep a last Measurement Report somewhere, and
>      transmit it until a new one is arrived from
>      the higher layers.

That makes sense.  However, that cached last measurement report should have
some kind of age attached.  There's no point in tranmitting neighbor measurements
that are 5 minutes old, particularly not for a mobile user.

> Probably, some parts of the Measurement Reporting implementation
> should be moved to L1, i.e. to the firmware, trxcon and VIRT-PHY.
> This way each Measurement Report would always contain the actual
> measurement results at the moment of transmission...

Measurement reports are sent every SACCH multiframe (102 or 104 frames), i.e. about
every 500ms.  Also, the end of the measurement period is defined independently from
the time at which a measurement report is being sent.  There are, AFAICT, about 12 frames
between the end of the measurement period and the start of the transmission of the
measurement report.  That's about 55ms of time.

I think 55ms might be indeed a bit tight to account for
Caylpso->UART->osmocon->mobile->osmocon->UART->Calypso.  At 115200 bps, there's onyly 633
characters in that period.

Hoewver, I think we shouldn't focus too much on the constraints of the slow serial link
of calypso phones here.  In the worst case, our measurement reports will be 480ms old, so be it.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From axilirator at gmail.com  Sat Aug 11 20:54:31 2018
From: axilirator at gmail.com (Vadim Yanitskiy)
Date: Sun, 12 Aug 2018 03:54:31 +0700
Subject: Proper measurement reporting
In-Reply-To: <20180811140626.GY17129@nataraja>
References: <CAPEnp=YfwUQ54ru-JqRSGtLd7cuNFo+cO6f1evt6dHn7jM71Wg@mail.gmail.com>
 <20180811140626.GY17129@nataraja>
Message-ID: <CAPEnp=Ycs9gwa3yCQOUGu4z0WOkyy6Nw9pbo4k7wvur_SHCt0g@mail.gmail.com>

Hi Harald, Sylvain,

>> My current ideas are:
>>
>>   1) Create a separate L1CTL message, that would be
>>      used to carry Measurement Reports. This way
>>      we wouldn't need to extract them from the
>>      L1CTL_DATA_REQ messages manually.
>>
>> I don't see much advantage in creating a new L1CTL message.
>> I mean detecting it's a measurement report is super easy so
>> I'd just stick with L1CTL_DATA_REQ.
>
> As sylvain already stated, there doesn't seem to be a need
> for yet another extra primitive.  For the MS side, it is a
> DATA_REQ on the uplink SACCH SAPI.

Introducing a new L1CTL message for Measurement Reports would
allow us to avoid mixing up the L1CTL code with GSM 04.08 code
at least in 'firmware/layer1/l23_api.c':

> #include <osmocom/gsm/protocol/gsm_04_08.h>
> ...
> struct gsm48_hdr *gh = (struct gsm48_hdr *)(data_ind->data + 5);
> if (gh->proto_discr == GSM48_PDISC_RR
>     && gh->msg_type == GSM48_MT_RR_MEAS_REP) {
> printd("updating measurement report\n");
> l1a_meas_msgb_set(msg);
> ...

I believe the L1CTL code shall be abstracted from the upper layer's stuff.

We could instead introduce a new field in 'l1ctl_info_ul' structure
(replacing one padding byte), and reuse it somehow, for example,
as a field for some flags, one of which might indicate that this
particular message is a Measurement Report. So, this way we would
not even break backward compatibility with the firmware:

  - old FW will continue to check the gsm48_hdr,
  - new FW will get rid of GSM 04.08, and check flags.

Another interesting moment is that the mobile application actually
sends the first Measurement Report *before establishing a channel*,
so in case of trxcon, the first one is just being dropped, because
I don't mix up the L1CTL code with GSM 04.08 stuff there. Having
some kind of indication (flags or separate message) would solve
this problem.

>>   2) Keep the last Measurement Report somewhere, and
>>      transmit it until a new one is arrived from
>>      the higher layers.
>
> That makes sense.  However, that cached last measurement
> report should have some kind of age attached.  There's no
> point in tranmitting neighbor measurements that are 5 minutes
> old, particularly not for a mobile user.

The cached Measurement Report would be updated as soon as we
receive a new one from the higher layers. And AFAIK, the mobile
application does send the reports much more frequently than
once in 5 minutes ;)

> I think 55ms might be indeed a bit tight to account for
> Caylpso->UART->osmocon->mobile->osmocon->UART->Calypso.
> At 115200 bps, there's onyly 633 characters in that period.
>
> Hoewver, I think we shouldn't focus too much on the
> constraints of the slow serial link of calypso phones here.
> In the worst case, our measurement reports will
> be 480ms old, so be it.

Yep, I will ask Piotr to check what does the MS Tester from
Sylvain think about such potentially delayed measurements
from Calypso PHY...

Extremely useful device (thanks again!), but I wish it could
provide the protocol traces, or at least a bit more informative
debug messages than just 'Test failed' :)

With best regards,
Vadim Yanitskiy.


From laforge at gnumonks.org  Thu Aug 16 18:48:41 2018
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 16 Aug 2018 20:48:41 +0200
Subject: OsmoCon 2018 draft schedule announced
Message-ID: <20180816184841.GE25650@nataraja>

Dear Osmocom community,

the first schedule of the 2018 incarnation of OsmoCon 2018 has been announced,
see http://osmocom.org/news/99 for the announcment and
https://pretalx.sysmocom.de/osmocon2018/schedule/ for the actual schedule.

At OsmoCon, we are not targetting developers, but more the wider community
and Osmocom users.  It would be great to meet many of you and hear more
about your relation to Osmocom.

Tickets are available from https://pretix.sysmocom.de/sysmocom/osmocon2018/,
and until August 31st the early bird discount still applies.

For those with a community / "just for fun" background and no employer
that would cover the ticket, we have a number of subsidized community discount
vouchers available.  See the OsmoCon 2018 wiki page at
https://osmocom.org/projects/osmo-dev-con/wiki/OsmoCon2018
for more information.

Looking forward to meeting as many of you as possible in roughly two
months from now,
	Harald Welte

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From holger at freyther.de  Mon Aug 20 08:47:58 2018
From: holger at freyther.de (Holger Freyther)
Date: Mon, 20 Aug 2018 09:47:58 +0100
Subject: Approach to system testing for Osmocom stack
In-Reply-To: <4a24d738-a44c-e278-d581-6d2c5ef19bcd@sysmocom.de>
References: <52E0568B-BD58-4FC2-AC06-CE948E5833D1@freyther.de>
 <4a24d738-a44c-e278-d581-6d2c5ef19bcd@sysmocom.de>
Message-ID: <0C54C29C-38C6-47A1-A5DB-DCB6A2947873@freyther.de>



> On 10. Jan 2018, at 09:49, Pau Espin Pedrol <pespin at sysmocom.de> wrote:


Hey Pau,

> I never used virtbts but I think it should be quite easy adding support for it in osmo-gsm-tester. The only big issue I can think of is that we need to find a way to differentiate between virtbs and other bts since they use a different communication medium and then that means that the requested modems to run for the test with virtbts need to be selected accordingly (ie. don't allocate an ofono modem from the object pool if we are using a virbts). That can probably accomplished easily too, by adding new attributes to objects to describe that characteristic.
> 
> It can be a good idea to support this at some point if we want to test for compatibility between "mobile" app + virtbs and other BTS and modems, for instance see if we can place a call between a regular modem with a sysmobts and a "mobile" app + virtbts. Then we can make sure we don't add regressions in the future.

I think I have finally a routine that gives me some time to move this forward. We should have the ability for bi-directional communication between mobile and the test driver and can start implementing the mobile as a non-ofono "modem".

From a mobile lua/point of view: The mobile will start, register over unix dgram to a server. At this point the server can send commands (as JSON) and the lua script can dispatch them (e.g. on/off, cell selection, send sms, place call)


Can you help me to see what needs to be done from a osmo-gsm-tester point of view? What amount of work is necessary? The benefit might be to have a quick(er) test on Jenkins.


regards
  holger

From pespin at sysmocom.de  Mon Aug 20 10:44:36 2018
From: pespin at sysmocom.de (Pau Espin Pedrol)
Date: Mon, 20 Aug 2018 12:44:36 +0200
Subject: Approach to system testing for Osmocom stack
In-Reply-To: <0C54C29C-38C6-47A1-A5DB-DCB6A2947873@freyther.de>
References: <52E0568B-BD58-4FC2-AC06-CE948E5833D1@freyther.de>
 <4a24d738-a44c-e278-d581-6d2c5ef19bcd@sysmocom.de>
 <0C54C29C-38C6-47A1-A5DB-DCB6A2947873@freyther.de>
Message-ID: <c7539fb6-ac96-3afe-28c5-849245b9926e@sysmocom.de>

Hi Holger

Since I wrote my last e-mail in this thread, we developed some more 
stuff for osmo-gsm-tester which may be interesting for this topic.

I recently introduced some code to be able to run TTCN3 tests using real 
HW and a motorola c213 phone running osmocom-bb, since we originally 
only run TTCN3 BTS tests using osmo-bts-virtual with virtphy, see [1] 
for more information. In order to drive all HW and software processes, I 
am using osmo-gsm-tester with specific config and testsuite which you 
can find in osmo-gsm-teser.git/ttcn/. You can find the jenkins job 
running tests with that setup in [2]. The setup is running mostly fine 
but I didn't find time yet to look at the list of tests still failing 
(which doesn't fail with the regular virt TTCN3 setup).

So, as a result, it means we now support using a osmocom-bb capable 
phone using real RF (wired) communicating against already available 
osmo-bts such as osmo-bts-sysmo, osmo-bts-trx, nanobts, etc., so no need 
to add support for osmo-bts-virt and also a way to differentiate 
networks that are not compatible (for instance, you don't want to pick a 
mobile running through virtphy and connecting it to osmo-bts-trx right?).

Current working setup for TTCN3 implies having a motorola phone 
automatically powered on when the corresponding serial device is opened 
(using [3]). So osmo-gsm-tester starts and controls the "osmocon" 
application which opens and uses the serial device. (You can find it in 
./src/osmo_gsm_tester/osmocon.py). In The TTCN3 testcase, we then 
connect TTCN3 code to the L1CTL iface provided by "osmocon", but in your 
case what you want to do I guess is connecting the "mobile" application 
to it.

So what I think it's required in your case:
* Move src/osmo_gsm_tester/modem.py to 
src/osmo_gsm_tester/modem_ofono.py (and class Modem to class ModemOfono
* To make code more easy to follow and extend, create an abstract class 
"Modem" (see class Bts in src/osmo_gsm_tester/bts.py), and make 
ModemOfono inheit from it.
* Create a new class ModemOsmocomLuaMobile (or similar) in a new file 
src/osmo_gsm_tester/modem_osmocom_mobile.py and inherit from 
modem.py:Modem. In there implement all required methods used by Modem 
interface users. To integrate it with the rest of the system you need to:
** Create a Modem "type" system, like we do for BTS (see 
resources.conf.prod how it's used), and add a method similar to 
suite.py:bts_obj() which creates a subclass based on that type, then use 
it in suite.py:modem() (similar to suite.py:bts()).
** Add any new required extra config parameters for this type of modems 
in resource.py:RESOURCES_SCHEMA
** I think to make it compatible with ofono modems, you should start 
both "mobile" and "osmocon" processes in the implementation of your 
Modem class, since the user of the modem class in the tests doesn't need 
to set up a osmocon process based on the type of modem (we do similar 
stuff with osmo-trx in bts_osmotrx.py).

I think that's mostly it. Don't hesitate to ping me here or in IRC if 
you have any question or require some help.

[1] https://osmocom.org/issues/3155
[2] 
https://jenkins.osmocom.org/jenkins/view/osmo-gsm-tester/job/osmo-gsm-tester_ttcn3/
[3] https://wiki.cuvoodoo.info/doku.php?id=osmotoserial

-- 
- Pau Espin Pedrol <pespin at sysmocom.de>         http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte