From nhofmeyr at sysmocom.de  Thu Mar  2 14:34:44 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 2 Mar 2017 15:34:44 +0100
Subject: my patches prioritized
Message-ID: <20170302143444.GB1649@ass40.sysmocom.de>

I currently have a whole series of patches on gerrit, many of which fail
to build. Here are the pivotal ones I am waiting for, the other patches
can be mostly ignored until these are through:

https://gerrit.osmocom.org/1682 (struct bsc_subscr)
Waiting for a comment. VLR and 3G branches build on it.  I'm happy to
change the patch, but not sure whether I really have to. IMHO there are
benefits to this unusual way, but one word ("no") is enough.

https://gerrit.osmocom.org/1923 (cosmetic prep for 1924)
https://gerrit.osmocom.org/1924 (python tests: VTY connection speedup)
These two are needed for the openbsc python tests speedup patches to work.

https://gerrit.osmocom.org/1962 (explicitly terminate ctrl_type_vals[])
https://gerrit.osmocom.org/1940 (find unterminated value strings)
The first is a fixup for the second to work, the second one is needed for
all of the other value_string topic patches to work. I'd like to add this
check to jenkins now, so that all incoming value_string[]s are validated.

https://gerrit.osmocom.org/1906 (python tests from separate build dir)
I usually build 2G and 3G separately, and I'd like to run the py tests in
each dev cycle, now that they are faster. So I'd love to merge this.


Hope this helps to save some time...

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170302/ac931921/attachment.bin>

From nhofmeyr at sysmocom.de  Thu Mar  2 19:38:39 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 2 Mar 2017 20:38:39 +0100
Subject: how to stop paging
Message-ID: <20170302193839.GF1649@ass40.sysmocom.de>

I have a general question about the Paging procedure, specifically about
how to stop paging.

In libbsc, we have Paging management that receives a request to Page from
the MSC and then repeatedly Pages the MS on all BTSes N times or until
told to stop.

In the NITB, we have code closely tied to the BSC and all BTS management,
so that when we receive a Paging Response, we can right away stop sending
Paging Commands on all BTSes.

So can the standalone-BSC: it stops paging when it received a Paging
Response.

In OsmoMSC with Iu- and A-interfaces, that's not possible. Looking in
08.08, there seems to be no A-interface message that tells the BSCs "thank
you, you can stop paging now". If one BSC or RNC has relayed back a Paging
Response from one of the cells, the MSC has no way to tell the other base
stations that it's fine to stop sending dozens of Pagings out for
"minutes", which would be useless now anyway because the MS has called
back via another base station already.

Is that a fact of life? We continue Paging in all cells where the MS is
not visible? Should the BSC/RNC part of the network page only once, so
that "stop paging" is a no-op?

Of course we first check which LAC the MS was first seen on and page only
there. Is the idea to send paging to one cell, and that cell stops Paging
by itself when it received a Paging Response that it relays to the MSC?

But IIRC Paging is supposed to spread out across other base stations if at
first it didn't succeed -- I recall to have read that in HNB-GW specs or
somewhere close to that. Our OsmoHNBGW so far simply Pages everywhere.

The practical reason I'm asking is that in the msc_vlr tests, I so far
explicitly check whether the "stop paging" function was called. Now on the
Iu branch, without libbsc, there is no such function. Should I drop those
checks because we will never have that? Or should I add some shim function
we call just for the tests so far, which will at some point tell the base
stations that Paging has concluded?

Thanks for any hints,

~N


-- 
- Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Gesch?ftsf?hrer / Managing Directors: Harald Welte
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170302/a5a5348f/attachment.bin>

From holger at freyther.de  Fri Mar  3 01:30:33 2017
From: holger at freyther.de (Holger Freyther)
Date: Fri, 3 Mar 2017 09:30:33 +0800
Subject: osmo-sip-connector in bad state
In-Reply-To: <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
Message-ID: <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>


> On 12 Feb 2017, at 11:23, Holger Freyther <holger at freyther.de> wrote:
> 
> 

Dear Omar,


> I am traveling in south east asia right now and my test setup is quite limited but I read a bit of sofia-sip code, the glib integration and our event loop integration and while I couldn't reproduce it, there are things to improve in our eventloop code. If you are building from source you could pull this[1] to give it a try. I intend to submit the changes on Tuesday morning (giving europe/us time to review on a business day). Nightly packages should be available soon.

did you have time to see if the updated eventloop code is improving the situation?

thank you
	holger



From msuraev at sysmocom.de  Fri Mar  3 11:16:17 2017
From: msuraev at sysmocom.de (Max)
Date: Fri, 3 Mar 2017 12:16:17 +0100
Subject: osmo-(bts-)trx multi-trx is broken
Message-ID: <6a9021c6-86fa-b6f1-284f-4c2c14e8900b@sysmocom.de>

Hi.

There's been few updates to osmo-bts-trx recently so I've decided to get 
back to https://projects.osmocom.org/issues/1648 and it seems like it's 
completely broken:
- the "multiplexing" option (osmo-trx -m -c 2) which used to work now 
leads to startup error of osmo-trx
- the "separate channel" option (osmo-trx -c 2) which wasn't working 
properly but at least phone could see the network, now leads to 
situation where no network is visible

Am I missing some magic combination of .cfg and command-line parameters 
which could make it work? Am I using too old version of uhd or smth else?

The hw has not changed (usrp b210 + external 10MHz reference clock) but 
maybe I've got to do some calibrations on it?

Any ideas as to what could be causing those regressions?

Is there anyone out there for whom some multi-trx setup is working? If 
so, I'd like to compare notes to understand why it isn't working form me.

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From alexander.chemeris at gmail.com  Fri Mar  3 11:59:31 2017
From: alexander.chemeris at gmail.com (Alexander Chemeris)
Date: Fri, 3 Mar 2017 12:59:31 +0100
Subject: osmo-(bts-)trx multi-trx is broken
In-Reply-To: <6a9021c6-86fa-b6f1-284f-4c2c14e8900b@sysmocom.de>
References: <6a9021c6-86fa-b6f1-284f-4c2c14e8900b@sysmocom.de>
Message-ID: <CABmJbFXXigPOkiSGS49ms-bUO4h=EiZW6vuUsVyd-Pp6DXP4Dw@mail.gmail.com>

Hi Max,

Multi-trx works properly with UmTRX which is our reference platform. We've
also done partial testing with OpenCellular - spectrum analyser showed that
TX is fine, but we haven't tested with a phone. With UmTRX we tested
everything.

Since OpenCellular is a B210 derivative, it should work fine as well.

Could you try adding -x option to osmo-trx to enable external clock? If you
have a spectrum analyser, I would recommend you to use it to test the
signal and frequency offset of the signal.

On general, if everything starts and you don't see a network, it's most
likely a clock or RF issue.

For -m option fail it would be great if you create a ticket and attach logs.

Please excuse typos. Written with a touchscreen keyboard.

--
Regards,
Alexander Chemeris
CTO/Founder Fairwaves, Inc.
https://fairwaves.co

On Mar 3, 2017 12:17 PM, "Max" <msuraev at sysmocom.de> wrote:

> Hi.
>
> There's been few updates to osmo-bts-trx recently so I've decided to get
> back to https://projects.osmocom.org/issues/1648 and it seems like it's
> completely broken:
> - the "multiplexing" option (osmo-trx -m -c 2) which used to work now
> leads to startup error of osmo-trx
> - the "separate channel" option (osmo-trx -c 2) which wasn't working
> properly but at least phone could see the network, now leads to situation
> where no network is visible
>
> Am I missing some magic combination of .cfg and command-line parameters
> which could make it work? Am I using too old version of uhd or smth else?
>
> The hw has not changed (usrp b210 + external 10MHz reference clock) but
> maybe I've got to do some calibrations on it?
>
> Any ideas as to what could be causing those regressions?
>
> Is there anyone out there for whom some multi-trx setup is working? If so,
> I'd like to compare notes to understand why it isn't working form me.
>
> --
> Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
> =======================================================================
> * sysmocom - systems for mobile communications GmbH
> * Alt-Moabit 93
> * 10559 Berlin, Germany
> * Sitz / Registered office: Berlin, HRB 134158 B
> * Geschaeftsfuehrer / Managing Director: Harald Welte
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170303/7aea7f46/attachment.htm>

From msuraev at sysmocom.de  Fri Mar  3 12:17:23 2017
From: msuraev at sysmocom.de (Max)
Date: Fri, 3 Mar 2017 13:17:23 +0100
Subject: osmo-(bts-)trx multi-trx is broken
In-Reply-To: <CABmJbFXXigPOkiSGS49ms-bUO4h=EiZW6vuUsVyd-Pp6DXP4Dw@mail.gmail.com>
References: <6a9021c6-86fa-b6f1-284f-4c2c14e8900b@sysmocom.de>
 <CABmJbFXXigPOkiSGS49ms-bUO4h=EiZW6vuUsVyd-Pp6DXP4Dw@mail.gmail.com>
Message-ID: <6d81251f-f1e7-bc08-ec90-a0ec8967b215@sysmocom.de>

I've created https://projects.osmocom.org/issues/1963 and linked it to 
1648 for -m issue.

I've tested with -x and external clock already. Unfortunately I can't 
promise to dig deeper with spectrum analyzer any time soon due to other 
tasks at hands. Could you please attach your known-to-work config files 
to #1648 so we have some reference point for future tests?

On 03.03.2017 12:59, Alexander Chemeris wrote:
> Hi Max,
>
> Multi-trx works properly with UmTRX which is our reference platform. 
> We've also done partial testing with OpenCellular - spectrum analyser 
> showed that TX is fine, but we haven't tested with a phone. With UmTRX 
> we tested everything.
>
> Since OpenCellular is a B210 derivative, it should work fine as well.
>
> Could you try adding -x option to osmo-trx to enable external clock? 
> If you have a spectrum analyser, I would recommend you to use it to 
> test the signal and frequency offset of the signal.
>
> On general, if everything starts and you don't see a network, it's 
> most likely a clock or RF issue.
>
> For -m option fail it would be great if you create a ticket and attach 
> logs.

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From keith at rhizomatica.org  Fri Mar  3 16:38:19 2017
From: keith at rhizomatica.org (Keith)
Date: Fri, 3 Mar 2017 17:38:19 +0100
Subject: osmo-(bts-)trx multi-trx is broken
In-Reply-To: <6d81251f-f1e7-bc08-ec90-a0ec8967b215@sysmocom.de>
References: <6a9021c6-86fa-b6f1-284f-4c2c14e8900b@sysmocom.de>
 <CABmJbFXXigPOkiSGS49ms-bUO4h=EiZW6vuUsVyd-Pp6DXP4Dw@mail.gmail.com>
 <6d81251f-f1e7-bc08-ec90-a0ec8967b215@sysmocom.de>
Message-ID: <4d6163bd-f9d8-3642-140f-7f704374b7ce@rhizomatica.org>

On 03/03/2017 13:17, Max wrote:

>
> I've tested with -x and external clock already. Unfortunately I can't
> promise to dig deeper with spectrum analyzer any time soon due to
> other tasks at hands. Could you please attach your known-to-work
> config files to #1648 so we have some reference point for future tests?
>
> On 03.03.2017 12:59, Alexander Chemeris wrote:
>
>> On general, if everything starts and you don't see a network, it's
>> most likely a clock or RF issue.
>>
Hi all, I may be way off here, but maybe worth saying, three days ago
trying to track down probs with AMR, I compiled and tested everything
from master, all deps + osmo-bts-trx and osmo-trx. I ran against my
standard nitb on another box though.
osmo-bts at 75852294
osmo-trx at 2dee3e99
Everything stated up fine but phones saw no network.

So I checked the RF with an RTL-SDR and I had a very narrow band
"silent" signal in the centre of the channel until I manually set
osmotrx tx-attenuation in the vty:

phy 0
 instance 0
  osmotrx tx-attenuation 35

I mention it because your latest cfg attached to the ticket has
osmotrx tx-attenuation oml
which is also the default I believe, in the absence of the command in
the cfg. and it didn't work for me.
At time I just presumed it should with a newer nitb.
I only have an N210 so I can't test multi-trx :-(

k/




From sebastian.stumpf87 at googlemail.com  Fri Mar  3 16:56:09 2017
From: sebastian.stumpf87 at googlemail.com (Sebastian Stumpf)
Date: Fri, 3 Mar 2017 17:56:09 +0100
Subject: Virtual layer 1 - Questions
In-Reply-To: <20170228213156.q72chdhhjxtyazy4@nataraja>
References: <CALFVOefg9TXn1bJ=WgNjdE+aNjotJFfzg3WkNg2yavyb94gTyA@mail.gmail.com>
 <20170212230920.epvdsj7jwxzta23w@nataraja>
 <CALFVOedfBVboU4Lk9D9QQNhr-dqybnH=0aNj=Zgucd3FjOJU4g@mail.gmail.com>
 <20170223144124.mdba7w7avdujy23g@nataraja>
 <CALFVOeeB=REeGK6=NaZr+adr7vNUJMzE6wHYMK=XdckFnzazbA@mail.gmail.com>
 <20170228213156.q72chdhhjxtyazy4@nataraja>
Message-ID: <CALFVOecU3fK6uPAB4=LDNE9OS9CY9vgUV+pwaAJCvC+qtPC+PQ@mail.gmail.com>

Hi Harald,

> This is likely a result of the MS somehow not receiving/processing the
> downlink signalling messages. [on TCH]

I found the reason for that. I thought I need to forward incoming messages
on TCH/H TCH/F as L1CTL_TRAFFIC_IND if the ACCH flag is not set in the
GSMTAP header channel type. But in case of a msg on FACCH this is wrong
and needs to be L1CTL_DATA_IND. As the CM Service Accept is sent on
FACCH it was not forwarded correctly to l23 and thus not processed.

Unfortunately I do not know, how to distinguish FACCH from TCH based on
the information available in the GSMTAP messages.
The stealing flag indicating a FACCH is set before and after the training
sequence in the burst I think, but this info is no longer available in the msg
as it is sent over the virtual um.

Or do I miss something here?

Do you have an idea how to check if the msg is TCH or FACCH?

> Also, what's noteworthy (but likely unrelated) is that the BTS is
> sending empty idle/padding frames on the TS7 all the time (those
> containing 0x2b2b2b2b) but the MS is not doing that in uplink.

That's because I just implemented some kind of one time scheduler.
If I do not get frames/data to schedule from the upper layer for a tdma slot,
I currently do not have any logic that fills these up with empty/idle frames.
Seems as if I need this, though.

Thanks a lot for your support, I am glad you are taking the time to help me
out :).

With kind regards,
Sebastian Stumpf


From laforge at gnumonks.org  Fri Mar  3 18:10:24 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Fri, 3 Mar 2017 19:10:24 +0100
Subject: Virtual layer 1 - Questions
In-Reply-To: <CALFVOecU3fK6uPAB4=LDNE9OS9CY9vgUV+pwaAJCvC+qtPC+PQ@mail.gmail.com>
References: <CALFVOefg9TXn1bJ=WgNjdE+aNjotJFfzg3WkNg2yavyb94gTyA@mail.gmail.com>
 <20170212230920.epvdsj7jwxzta23w@nataraja>
 <CALFVOedfBVboU4Lk9D9QQNhr-dqybnH=0aNj=Zgucd3FjOJU4g@mail.gmail.com>
 <20170223144124.mdba7w7avdujy23g@nataraja>
 <CALFVOeeB=REeGK6=NaZr+adr7vNUJMzE6wHYMK=XdckFnzazbA@mail.gmail.com>
 <20170228213156.q72chdhhjxtyazy4@nataraja>
 <CALFVOecU3fK6uPAB4=LDNE9OS9CY9vgUV+pwaAJCvC+qtPC+PQ@mail.gmail.com>
Message-ID: <20170303181024.kxpv4pvk5yfmtv7m@nataraja>

Hi Sebastian,

On Fri, Mar 03, 2017 at 05:56:09PM +0100, Sebastian Stumpf wrote:

> > This is likely a result of the MS somehow not receiving/processing the
> > downlink signalling messages. [on TCH]
> 
> I found the reason for that. I thought I need to forward incoming messages
> on TCH/H TCH/F as L1CTL_TRAFFIC_IND if the ACCH flag is not set in the
> GSMTAP header channel type. But in case of a msg on FACCH this is wrong
> and needs to be L1CTL_DATA_IND. As the CM Service Accept is sent on
> FACCH it was not forwarded correctly to l23 and thus not processed.
> 
> Unfortunately I do not know, how to distinguish FACCH from TCH based on
> the information available in the GSMTAP messages.

the reaseon for that is simple: We never forwarded actual traffic/user
data in GSMTAP so far, only signalling messages.  So for now, it is safe
to assume that all TCH data is FACCH.  GSMTAP will have to be
extended with a new channel type if we want to carry voice frames later
on.

> That's because I just implemented some kind of one time scheduler.
> If I do not get frames/data to schedule from the upper layer for a tdma slot,
> I currently do not have any logic that fills these up with empty/idle frames.
> Seems as if I need this, though.

I think it is needed sooner or later, as the BTS will consider the radio
channel dead if it doesn't receive anything.  Also, it will probably
think the radio quality is very poor?  At the very least the SACCH must
be present in uplink.  But maybe not the most important bit right now.

If you have to go for a full scheduler sooner or later, I think it's
worth to "abstract out + recycle" either the scheduler tables from
OsmocomBB layer1 (but I think they're very calypso specific?), or those
from OsmoBTS, so one set of scheduling tables can be used in multiple
places.

> Thanks a lot for your support, I am glad you are taking the time to help me
> out :).

I'm happy if I can help your progress.

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From msuraev at sysmocom.de  Fri Mar  3 18:27:16 2017
From: msuraev at sysmocom.de (Max)
Date: Fri, 3 Mar 2017 19:27:16 +0100
Subject: RFC: jenkins pipeline
Message-ID: <01b7cab2-05d1-0436-6898-f569aa7f92ab@sysmocom.de>

Hi.

Right now we have rather sophisticated CI/CD setup which is spread over 
several services (OBS for nightly packages, jenkins for CI) and 
repositories.

It works well, but if I've got to somehow extend/alter it than I have to 
look in at least 3 different places:
- ocmo-ci repo for common scripts
- jenkins job in jenkins web ui
- project's jenkins*.sh scripts

I think it's not meant to be that way, it's just what has grown 
organically over the years.

Right now we use recent enough jenkins which support Pipelines [1] 
plugin. The basic idea behind it is that each project has Jenkinsfile 
[2] in the repository which is self-contained configuration for CI.

In theory enabling this plugin should not affect existing jobs as it's 
entirely separate job type.

I suggest following:
- enable Pipelines plugin in jenkins
- configure new pipelines-based job pointing to non-master branch which 
we can use as a playground
- once we comfortable with it - migrate existing CI (and possibly CD) to 
pipelines (as time permit)
- switch gerrit from old jobs to new pipelines

What do you think?

Details are available via links below but here are some quick takeaways 
which motivated me to write this email:

- we'll have actual CI job description under version control (right now 
it just sits in web ui)
- we'll have single place to look for CI-related things (ok, maybe 2 
places if we still need common library)
- it's more eye-candy (it's clear which build step we're in and how long 
each step took)
- it's easier to parallelize (e. g. run sanitizer and regular builds in 
parallel)

On a related note: in general, who's point of contact for things like 
"please update jenkins plugin X to version Y"?

[1] https://jenkins.io/doc/book/pipeline/
[2] https://jenkins.io/doc/book/pipeline/jenkinsfile/

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From tom at tsou.cc  Fri Mar  3 19:07:54 2017
From: tom at tsou.cc (Tom Tsou)
Date: Fri, 3 Mar 2017 11:07:54 -0800
Subject: osmo-(bts-)trx multi-trx is broken
In-Reply-To: <6a9021c6-86fa-b6f1-284f-4c2c14e8900b@sysmocom.de>
References: <6a9021c6-86fa-b6f1-284f-4c2c14e8900b@sysmocom.de>
Message-ID: <CAATyM+YvWvXhZUsJgCRvE=+DSrLFsQgVpYsW1kouYmKDXjj+3Q@mail.gmail.com>

On Fri, Mar 3, 2017 at 3:16 AM, Max <msuraev at sysmocom.de> wrote:
> There's been few updates to osmo-bts-trx recently so I've decided to get
> back to https://projects.osmocom.org/issues/1648 and it seems like it's
> completely broken:
> - the "multiplexing" option (osmo-trx -m -c 2) which used to work now leads
> to startup error of osmo-trx
> - the "separate channel" option (osmo-trx -c 2) which wasn't working
> properly but at least phone could see the network, now leads to situation
> where no network is visible

I'm looking into these issues. The startup error in the first case is
my first concern. The second issue should be straightforward to test.
I will be using the config files in the ticket unless somebody
suggests otherwise.

  -TT


From mardnh at gmx.de  Fri Mar  3 20:13:58 2017
From: mardnh at gmx.de (Martin Hauke)
Date: Fri, 3 Mar 2017 21:13:58 +0100
Subject: Osmocom nightly packages
Message-ID: <8de3c31b-2c3e-4282-6723-81bd79f9c228@gmx.de>

Hi,

i'm building osmocom-related packages for openSUSE since a few years on OBS.
https://build.opensuse.org/project/show/home:mnhauke:osmocom
https://build.opensuse.org/project/show/home:mnhauke:osmocom-iuh

Now i'd like to add support for RPM-based distributions to the
osmocom-nightly packages.
My current plan is to first add support for (open)SUSE based
distributions and if there's interest then find out all the subtle
changes that are needed for the spec-file to also add support for
Redhat/Fedora/CentOS.

# Phase1
- openSUSE_Leap_42.2 (x86_64)
- openSUSE_Tumbleweed (i586, x86_64)
- openSUSE_Factory_ARM (armv7l, aarch64)
- SLE_12_SP2 (x86_64)

# Phase2
- CentOS_7 (x86_64)
- Fedora_25 (i586, x86_64)
- RHEL_7 (x86_64)

How does the current workflow for the daily check-in of the nightly
debian/ubuntu packages look like?

There's probably a bunch of scripts that do
- git checkout from all relevant repositories
- update changelog based on git commit messages
- change the package version
- create the new tarball
- maybe osc local build
- upload the tarball and the dsc to OBS
- monitor if all the OBS-packages are successfully built and in state
published

Thanks in advance,
Martin



From nhofmeyr at sysmocom.de  Sat Mar  4 05:28:24 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Sat, 4 Mar 2017 06:28:24 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <01b7cab2-05d1-0436-6898-f569aa7f92ab@sysmocom.de>
References: <01b7cab2-05d1-0436-6898-f569aa7f92ab@sysmocom.de>
Message-ID: <20170304052824.GC4369@my.box>

On Fri, Mar 03, 2017 at 07:27:16PM +0100, Max wrote:
> Hi.
> 
> Right now we have rather sophisticated CI/CD setup which is spread over
> several services (OBS for nightly packages, jenkins for CI) and
> repositories.
> 
> It works well, but if I've got to somehow extend/alter it than I have to
> look in at least 3 different places:
> - ocmo-ci repo for common scripts
> - jenkins job in jenkins web ui
> - project's jenkins*.sh scripts
> 
> I think it's not meant to be that way, it's just what has grown organically
> over the years.
> 
> Right now we use recent enough jenkins which support Pipelines [1] plugin.

Well, whaddaya know, one of the accelerate3g5 contestants actually wanted to
help introduce Jenkins Pipelines. Sounds like it's going to happen :)
dr.blobb, could you join the discussion please?

About the distribution of scripts: the osmo-ci has common scripts, and each
project's jenkins.sh has project-specific steps in it. Either we keep all
project-specific details in one central place instead of with the project
(where one might say it belongs), or we copy the osmo-ci scripts to every
project duplicating the code -- I don't really see a way to get out of that
part... But I've been annoyed by having to edit a dozen different repositories
to tweak the same detail in each jenkins.sh (like adding that value_string
check), so keeping all jenkins.sh in one central place has occurred to me
several times before as being a good idea. It has the disadvantage that you
can't change jenkins.sh at the same time as a patch changes the behavior (like
your --enable-sanitizer change in libosmo-abis, which was all nicely applied in
one patch), but so far the disadvantage of having to edit N separate
repositories has far outweighed that for me.


> I suggest following:
> - enable Pipelines plugin in jenkins
> - configure new pipelines-based job pointing to non-master branch which we
> can use as a playground
> - once we comfortable with it - migrate existing CI (and possibly CD) to
> pipelines (as time permit)
> - switch gerrit from old jobs to new pipelines
> 
> What do you think?

sounds excellent.

> - we'll have actual CI job description under version control (right now it
> just sits in web ui)

except for the parts we relayed into the jenkins.sh scripts for the same
reason -- but this includes the job config as well, right? which sounds
excellent. I'd trade any web interface for text files any time.

> - we'll have single place to look for CI-related things (ok, maybe 2 places
> if we still need common library)

ah ok

> - it's more eye-candy (it's clear which build step we're in and how long
> each step took)
> - it's easier to parallelize (e. g. run sanitizer and regular builds in
> parallel)

how does this come about? Separate workspace per pipeline run?  The point
being: does jenkins keep separate network interfaces for each? Our 'make check'
often starts up things on ports that must not be reused.  That's why docker was
such an improvement for the openbsc job.  Ah, I see that pipelines can actually
integrate with docker as "agent"!

> On a related note: in general, who's point of contact for things like
> "please update jenkins plugin X to version Y"?

I guess Holger or me, but the decision whether to do that is probably more with
Holger and Harald, besides the community at large.

> [1] https://jenkins.io/doc/book/pipeline/
> [2] https://jenkins.io/doc/book/pipeline/jenkinsfile/

One thing that catches my attention: do we have to trade shell script for
groovy scripting? That would be a potential drawback, because we're familiar
with shell, not with groovy. Groovy is very Java, we're more C and sh.

It seems that shell commands become

  sh 'ls -alh'

so I guess we would put everything except one-liners into an actual .sh file
and call that from the jenkinsfile. I wouldn't want to shell-script with
prepending 'sh' everywhere and losing all of the env on every line.


Overall I would +1 pipeline trials.


The jenkins.osmocom.org plugin page though says:

"	
Pipeline
A suite of plugins that lets you orchestrate automation, simple or complex. See the Jenkins website for more details and documentation.
<red>Warning: This plugin requires dependent plugins be upgraded and at least one of these dependent plugins claims to use a different settings format than the installed version. Jobs using that plugin may need to be reconfigured, and/or you may not be able to cleanly revert to the prior version without manually restoring old settings. Consult the plugin release notes for details.</red>
"

whatever that means in practice.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170304/23f721ea/attachment.bin>

From holger at freyther.de  Sat Mar  4 07:57:08 2017
From: holger at freyther.de (Holger Freyther)
Date: Sat, 4 Mar 2017 08:57:08 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <20170304052824.GC4369@my.box>
References: <01b7cab2-05d1-0436-6898-f569aa7f92ab@sysmocom.de>
 <20170304052824.GC4369@my.box>
Message-ID: <CC18C4EB-AE7E-4CFA-A89C-635E64AD5F09@freyther.de>


> On 4 Mar 2017, at 06:28, Neels Hofmeyr <nhofmeyr at sysmocom.de> wrote:
> 
> 


> About the distribution of scripts: the osmo-ci has common scripts, and each
> project's jenkins.sh has project-specific steps in it. Either we keep all
> project-specific details in one central place instead of with the project
> (where one might say it belongs), or we copy the osmo-ci scripts to every
> project duplicating the code -- I don't really see a way to get out of that
> part... But I've been annoyed by having to edit a dozen different repositories
> to tweak the same detail in each jenkins.sh (like adding that value_string
> check), so keeping all jenkins.sh in one central place has occurred to me
> several times before as being a good idea. It has the disadvantage that you
> can't change jenkins.sh at the same time as a patch changes the behavior (like
> your --enable-sanitizer change in libosmo-abis, which was all nicely applied in
> one patch), but so far the disadvantage of having to edit N separate
> repositories has far outweighed that for me.

The reason I moved the build instruction out of the Job into a file is to
have people easily rebuild/reproduce it. E.g. it helps with people that
either don't know make distcheck or can't copy the invocation from the log.

I think if someone wants to reproduce the failure, it will be difficult for
that person to checkout the right repository with the build script. :)

What I think should be avoided is to use Jenkins specific files. You might
need to install Java and tons of jars to locally drive your build. ;)

holger



From laforge at gnumonks.org  Sat Mar  4 10:01:03 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Sat, 4 Mar 2017 11:01:03 +0100
Subject: Osmocom nightly packages
In-Reply-To: <8de3c31b-2c3e-4282-6723-81bd79f9c228@gmx.de>
References: <8de3c31b-2c3e-4282-6723-81bd79f9c228@gmx.de>
Message-ID: <20170304100103.jvrflj3y4f5rbut7@nataraja>

Hi Martin,

On Fri, Mar 03, 2017 at 09:13:58PM +0100, Martin Hauke wrote:
> i'm building osmocom-related packages for openSUSE since a few years on OBS.
> https://build.opensuse.org/project/show/home:mnhauke:osmocom
> https://build.opensuse.org/project/show/home:mnhauke:osmocom-iuh

Thanks a lot for that!

I think it would be best to merge your efforts with the "official"
osmocom nightly packages at
https://build.opensuse.org/project/show/network:osmocom:nightly

The users likely want one well-maintained set of nightly packages, not
multiple sets from multiple sources, which is likely to cause confusion.

I've added you to the osmocom:nightly project so you have full access to
all related OBS configuration.

> How does the current workflow for the daily check-in of the nightly
> debian/ubuntu packages look like?
> 
> There's probably a bunch of scripts that do

There's one script on jenkins.osmocom.org, in the 
http://jenkins.osmocom.org/jenkins/job/Osmocom_nightly_packages
build job:

====
rm -rf *
git clone git://git.osmocom.org/osmo-sip-connector
git clone git://git.osmocom.org/libosmocore
git clone git://git.osmocom.org/libosmo-sccp
git clone git://git.osmocom.org/libosmo-abis
git clone git://git.osmocom.org/libosmo-netif
git clone git://git.osmocom.org/libsmpp34
git clone git://git.osmocom.org/openggsn
git clone git://git.osmocom.org/openbsc
git clone git://git.osmocom.org/osmo-pcap
git clone git://git.osmocom.org/cellmgr-ng osmo-stp
git clone git://git.osmocom.org/osmo-trx
git clone git://git.osmocom.org/osmo-bts
git clone git://git.osmocom.org/osmo-pcu


PROJ=network:osmocom:nightly
osc co $PROJ

DT=`date +%Y%m%d`


build() {
  rm -rf data
  cd $1
  VER=`head -1 debian/changelog | cut -d ' ' -f 2 | sed s,"(","",  | sed s,")","",`
  dch -v $VER.$DT -m "Snapshot build"
  git commit -m "$DT snapshot" debian/
  gbp buildpackage -S -uc -us --git-export-dir=$PWD/../data
  cd ../$PROJ/$1
  osc rm * || true
  mv ../../data/*.dsc .
  mv ../../data/*.tar* .
  osc add *
  cd ../../
}

build libosmocore
build libosmo-sccp
build libosmo-abis
build libosmo-netif
build libsmpp34
build openggsn
build openbsc
build osmo-pcap
build osmo-stp
build osmo-trx
build osmo-sip-connector

cp openbsc/openbsc/include/openbsc/gsm_data_shared.h osmo-bts/include/openbsc/
cp openbsc/openbsc/src/libcommon/gsm_data_shared.c osmo-bts/src/common/gsm_data_shared.c
cd osmo-bts
git add include/openbsc/gsm_data_shared.h
git add src/common/gsm_data_shared.c
git commit -m "Copy OpenBSC files needed for the build"
cd ../
build osmo-bts
build osmo-pcu

cd $PROJ
osc ci -m "Snapshot $DT"
====

If you sign up to jenkins.osmocom.org, I can give you permissions to
access the configuration.

I think the general strategy should be to
* merge whatever useful bits you may have in your OBS jobs into
  osmocom-nightly.  They would be treated like any other patch via
  gerrit
* introduce support for more distributions in a next step, maintaining
  the related spec files also in the regular osmocom git repositories
  alongside with the code

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From laforge at gnumonks.org  Sat Mar  4 09:38:05 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Sat, 4 Mar 2017 10:38:05 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <CC18C4EB-AE7E-4CFA-A89C-635E64AD5F09@freyther.de>
References: <01b7cab2-05d1-0436-6898-f569aa7f92ab@sysmocom.de>
 <20170304052824.GC4369@my.box>
 <CC18C4EB-AE7E-4CFA-A89C-635E64AD5F09@freyther.de>
Message-ID: <20170304093805.vwrnx2qmk2lnaxo6@nataraja>

Hi Holger,

On Sat, Mar 04, 2017 at 08:57:08AM +0100, Holger Freyther wrote:
> The reason I moved the build instruction out of the Job into a file is to
> have people easily rebuild/reproduce it. E.g. it helps with people that
> either don't know make distcheck or can't copy the invocation from the log.

I think tat makes a lot of sense, and I would generally like to see more
(if possible) of the jenkins jobs move into the repositories, maybe even
those of the sysmocom jenkins.

It should be possible (and documented!) how somebody can locally
reproduce a build (particularly a build error) that is shown in a
jenkins job.

> I think if someone wants to reproduce the failure, it will be difficult for
> that person to checkout the right repository with the build script. :)

agreed.

> What I think should be avoided is to use Jenkins specific files. You might
> need to install Java and tons of jars to locally drive your build. ;)

also fully agreed here.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From laforge at gnumonks.org  Sat Mar  4 09:43:27 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Sat, 4 Mar 2017 10:43:27 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <20170304052824.GC4369@my.box>
References: <01b7cab2-05d1-0436-6898-f569aa7f92ab@sysmocom.de>
 <20170304052824.GC4369@my.box>
Message-ID: <20170304094327.dga27uvxqi6mqpmh@nataraja>

Hi Neels,

On Sat, Mar 04, 2017 at 06:28:24AM +0100, Neels Hofmeyr wrote:
> About the distribution of scripts: the osmo-ci has common scripts, and each
> project's jenkins.sh has project-specific steps in it. 

> Either we keep all project-specific details in one central place
> instead of with the project (where one might say it belongs), or we
> copy the osmo-ci scripts to every project duplicating the code -- I
> don't really see a way to get out of that part... 

what about git-subtree or git-submodule for osmo-ci?

> except for the parts we relayed into the jenkins.sh scripts for the same
> reason -- but this includes the job config as well, right? which sounds
> excellent. I'd trade any web interface for text files any time.

I also think that having the job configuration in the repository would
be a plus.

> > [1] https://jenkins.io/doc/book/pipeline/
> > [2] https://jenkins.io/doc/book/pipeline/jenkinsfile/
> 
> One thing that catches my attention: do we have to trade shell script for
> groovy scripting? That would be a potential drawback, because we're familiar
> with shell, not with groovy. Groovy is very Java, we're more C and sh.

I have a strong opinion against our developers having to learn a new
programming language just for continuous integration.  That would be a
*very* high price to pay.

Also, in general I appreciate steps that improve productivity.  But
please keep in mind that using new tools/toys just because they exist
and may be hyped is not a good reason.  I'm not saying this is the case
here, but I'm saying we have to be careful.

Regards,
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Sat Mar  4 15:24:36 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Sat, 4 Mar 2017 16:24:36 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <CC18C4EB-AE7E-4CFA-A89C-635E64AD5F09@freyther.de>
References: <01b7cab2-05d1-0436-6898-f569aa7f92ab@sysmocom.de>
 <20170304052824.GC4369@my.box>
 <CC18C4EB-AE7E-4CFA-A89C-635E64AD5F09@freyther.de>
Message-ID: <20170304152436.GB2089@my.box>

On Sat, Mar 04, 2017 at 08:57:08AM +0100, Holger Freyther wrote:
> > About the distribution of scripts: the osmo-ci has common scripts, and each
> > project's jenkins.sh has project-specific steps in it. Either we keep all
> 
> The reason I moved the build instruction out of the Job into a file is to
> have people easily rebuild/reproduce it. E.g. it helps with people that
> either don't know make distcheck or can't copy the invocation from the log.
> 
> I think if someone wants to reproduce the failure, it will be difficult for
> that person to checkout the right repository with the build script. :)

At the moment the jenkins.sh actually depends on osmo-ci scripts, so said
someone would also need the right repository as well -- short of understanding
what 'osmo-build-dep.sh libosmocore' means :P

On Sat, Mar 04, 2017 at 10:43:27AM +0100, Harald Welte wrote:
> what about git-subtree or git-submodule for osmo-ci?

Probably a good idea. Though, submodules require manual steps to check out,
right? So a README is needed anyway.

With osmo-ci submoduled it's also a small step to actually move the various
jenkins.sh into osmo-ci and be able to edit them centrally? :)  anyway, not
high up on the agenda.


On Sat, Mar 04, 2017 at 08:57:08AM +0100, Holger Freyther wrote:
> What I think should be avoided is to use Jenkins specific files. You might
> need to install Java and tons of jars to locally drive your build. ;)
[and]
On Sat, Mar 04, 2017 at 10:43:27AM +0100, Harald Welte wrote:
> I have a strong opinion against our developers having to learn a new
> programming language just for continuous integration.  That would be a
> *very* high price to pay.

Have all build steps in shell scripts that can run on their own, and accompany
that with a jenkins specific file to drive the pipeline, which basically calls
the shell scripts?



On Sat, Mar 04, 2017 at 10:43:27AM +0100, Harald Welte wrote:
> Also, in general I appreciate steps that improve productivity.  But
> please keep in mind that using new tools/toys just because they exist
> and may be hyped is not a good reason.  I'm not saying this is the case
> here, but I'm saying we have to be careful.

So, what would be the actual benefits?

* complete job config in git? (to be confirmed)
* faster because less things have to be rebuilt? (to be confirmed)
* faster because easily parallelizable/easier integration with docker? (to be
  confirmed)

If dr.blobb and/or Max could clarify these points that would be great
(while I guess they will clarify in depth only after actual trials).
First tests could run on a privately setup Jenkins ... dr.blobb?

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170304/c3600cdf/attachment.bin>

From dr.blobb at gmail.com  Sun Mar  5 21:03:20 2017
From: dr.blobb at gmail.com (=?UTF-8?Q?Klaus_M=C3=BCller?=)
Date: Sun, 5 Mar 2017 22:03:20 +0100
Subject: RFC: jenkins pipeline
Message-ID: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>

Hi,

>> So, what would be the actual benefits?

>>* complete job config in git? (to be confirmed)

Just to clarify, the Pipeline plugin on its own doesn't provide a
complete config handling in git. This means only the code shown in
image [3] lives in git, configurations within the "General" or "Build
Triggers" tab will still be handled in the web-ui.***
For the sake of completeness the JobDSL plugin [2][ has to be mention.
This plugin allows to put the entire job configuration to git and
deploys jobs without previous manual setup in the web-ui.

>>* faster because less things have to be rebuilt? (to be confirmed)

Somehow I doubt that this improvement dependents on the Pipeline
plugin, but I know to few details to have an opinion based on facts.
:)

>>* faster because easily parallelizable/easier integration with docker? (to be
>>  confirmed)

first point: e.g. Pipeline can trigger "down-stream-projects/stages"
after one specific part of a
parallel-block/matrix-configurations-project is finished.

second point: Pipeline brings a docker wrapper, so one don't have to
pass one build script holding the entire build sequence when invoking
the container. You can invoke as much scripts as you want within the
docker wrapper. (no black-box god-script :)

>> If dr.blobb and/or Max could clarify these points that would be great
>> (while I guess they will clarify in depth only after actual trials).
>> First tests could run on a privately setup Jenkins ... dr.blobb?

For sure, let's use this Jenkins instance [1] for trials. Everyone is
welcome to sign up and send me a mail to get the necessary
permissions.

I start migrating the OpenBSC build pipeline to Pipeline and keep you
updated. Is this mailing list the appropriate communication channel?

Is there already a osmocom ci wiki page, which I overlooked? I'd be
happy to contribute!


~Andr?


*** Some "web-ui configurations" can be injected from the jenkinsfile, though.

[1] https://jenkins.blobb.me/view/osmocom/
[2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
[2.5] https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.DslFactory.pipelineJob
[3] http://tinyurl.com/zbavrtp


From nhofmeyr at sysmocom.de  Mon Mar  6 13:38:23 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 6 Mar 2017 14:38:23 +0100
Subject: packages broken by osmo-pcu commit "BTS: Convert relative frame
 numbers to absolute frame numbers"
Message-ID: <20170306133823.GA12874@my.box>

Hi,

the line 

  if (abs(rfn - m_cur_rfn) > RFN_THRESHOLD) {

in your recent osmo-pcu patch breaks the package builds (on ubuntu 16.10 x86_64 as well as i586).
1275a3f91a744e011b0dba82b09124d249c7abb5 / I74f00c11e5739d49f370ce6c357149e81d9aa759

I guess you should cast the uint32_t to e.g. long int explicitly to avoid the overloading ambiguity.

  abs((long int)(...))

It puzzles me why our gerrit build job did not run into this ambiguity though.
My own build on my laptop doesn't even print a warning for this line.


https://build.opensuse.org/package/live_build_log/network:osmocom:nightly/osmo-pcu/xUbuntu_16.10/x86_64

[  110s] bts.cpp: In member function 'uint32_t BTS::rfn_to_fn(uint32_t)':
[  110s] bts.cpp:553:25: error: call of overloaded 'abs(uint32_t)' is ambiguous
[  110s]   if (abs(rfn - m_cur_rfn) > RFN_THRESHOLD) {
[  110s]                          ^
[  110s] In file included from /usr/include/c++/6/cstdlib:75:0,
[  110s]                  from /usr/include/c++/6/stdlib.h:36,
[  110s]                  from /usr/include/osmocom/core/linuxrbtree.h:97,
[  110s]                  from /usr/include/osmocom/core/timer.h:35,
[  110s]                  from ./bts.h:29,
[  110s]                  from bts.cpp:21:
[  110s] /usr/include/stdlib.h:735:12: note: candidate: int abs(int)
[  110s]  extern int abs (int __x) __THROW __attribute__ ((__const__)) __wur;
[  110s]             ^~~
[  110s] In file included from /usr/include/c++/6/stdlib.h:36:0,
[  110s]                  from /usr/include/osmocom/core/linuxrbtree.h:97,
[  110s]                  from /usr/include/osmocom/core/timer.h:35,
[  110s]                  from ./bts.h:29,
[  110s]                  from bts.cpp:21:
[  110s] /usr/include/c++/6/cstdlib:185:3: note: candidate: __int128 std::abs(__int128)
[  110s]    abs(__GLIBCXX_TYPE_INT_N_0 __x) { return __x >= 0 ? __x : -__x; }
[  110s]    ^~~
[  110s] /usr/include/c++/6/cstdlib:180:3: note: candidate: long long int std::abs(long long int)
[  110s]    abs(long long __x) { return __builtin_llabs (__x); }
[  110s]    ^~~
[  110s] /usr/include/c++/6/cstdlib:172:3: note: candidate: long int std::abs(long int)
[  110s]    abs(long __i) { return __builtin_labs(__i); }
[  110s]    ^~~

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170306/636ca4ba/attachment.bin>

From nhofmeyr at sysmocom.de  Mon Mar  6 13:53:39 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 6 Mar 2017 14:53:39 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
Message-ID: <20170306135339.GB12874@my.box>

On Sun, Mar 05, 2017 at 10:03:20PM +0100, Klaus M?ller wrote:
> Hi,
> 
> >> So, what would be the actual benefits?
> 
> >>* complete job config in git? (to be confirmed)
> 
> Just to clarify, the Pipeline plugin on its own doesn't provide a
> complete config handling in git. This means only the code shown in
> image [3] lives in git, configurations within the "General" or "Build
> Triggers" tab will still be handled in the web-ui.***
> For the sake of completeness the JobDSL plugin [2][ has to be mention.
> This plugin allows to put the entire job configuration to git and
> deploys jobs without previous manual setup in the web-ui.
> 
> >>* faster because less things have to be rebuilt? (to be confirmed)
> 
> Somehow I doubt that this improvement dependents on the Pipeline
> plugin, but I know to few details to have an opinion based on facts.
> :)
> 
> >>* faster because easily parallelizable/easier integration with docker? (to be
> >>  confirmed)
> 
> first point: e.g. Pipeline can trigger "down-stream-projects/stages"
> after one specific part of a
> parallel-block/matrix-configurations-project is finished.
> 
> second point: Pipeline brings a docker wrapper, so one don't have to
> pass one build script holding the entire build sequence when invoking
> the container. You can invoke as much scripts as you want within the
> docker wrapper. (no black-box god-script :)

This sounds to me like Pipelines don't actually provide any of the features I
was dreaming about, except maybe easier docker integration?

> Is this mailing list the appropriate communication channel?

yes.

> Is there already a osmocom ci wiki page, which I overlooked? I'd be
> happy to contribute!

well, searching osmocom.org for jenkins gave me

https://osmocom.org/projects/cellular-infrastructure/wiki/Jenkins
https://osmocom.org/projects/osmocom-servers/wiki/Jenkins_Node_Setup
https://osmocom.org/projects/cellular-infrastructure/wiki/Nightly_Builds
[...]

and "Wiki" is an ancient African word for "hopelessly outdated"...
We appreciate every checked and/or corrected facts on those pages,
and the Jenkins_Node_Setup could probably be merged into the Jenkins one.


> [1] https://jenkins.blobb.me/view/osmocom/

nice, a sleepy pipeline ;)

> [2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin

This sounds like we want to use it!

> [2.5] https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.DslFactory.pipelineJob
> [3] http://tinyurl.com/zbavrtp

direct link without javascript cruft: https://media.comsysto.com/images/2016-11-23-jenkins-pipelines/new-jenkins-pipeline-config--c.png

so far that doesn't look like an improvement of

  #!/bin/sh
  ./contrib/jenkins.sh

~N

-- 
- Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Gesch?ftsf?hrer / Managing Directors: Harald Welte
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170306/afeaa3f0/attachment.bin>

From holger at freyther.de  Mon Mar  6 14:28:08 2017
From: holger at freyther.de (Holger Freyther)
Date: Mon, 6 Mar 2017 15:28:08 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
Message-ID: <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>


> On 3 Mar 2017, at 02:30, Holger Freyther <holger at freyther.de> wrote:
> 
> 
>> On 12 Feb 2017, at 11:23, Holger Freyther <holger at freyther.de> wrote:
>> 
>> 
> 
> Dear Omar,
> 
> 
>> I am traveling in south east asia right now and my test setup is quite limited but I read a bit of sofia-sip code, the glib integration and our event loop integration and while I couldn't reproduce it, there are things to improve in our eventloop code. If you are building from source you could pull this[1] to give it a try. I intend to submit the changes on Tuesday morning (giving europe/us time to review on a business day). Nightly packages should be available soon.
> 
> did you have time to see if the updated eventloop code is improving the situation?

I was working on a manual testcase for DTMF handling.. and I am able to reproduce this now. Will analyze it now and then return to DTMF.

holger

From dr.blobb at gmail.com  Mon Mar  6 15:15:20 2017
From: dr.blobb at gmail.com (=?UTF-8?Q?Andr=C3=A9_Boddenberg?=)
Date: Mon, 6 Mar 2017 16:15:20 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <20170306135339.GB12874@my.box>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
Message-ID: <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>

>> This sounds to me like Pipelines don't actually provide any of the features I
>> was dreaming about, except maybe easier docker integration?

first yes! second as you said -> maybe, still doubting it :)

When we speak about the "easier docker integration", we mean that
everyone would be happy if not every matrix-configuration axis builds
all deps like libosmocore, libosmo-netif etc pp, right?

To address this issue I'd like to create a local temporary docker
image which extends osmocom:amd64  and holds all deps. This image can
then be used for all openBSC builds and the temporary local docker
image can be removed as a "post-build"-step (which should get
triggered regardless the build result).


A slightly different topic, did you think about pushing your docker
images to hub.docker.com [2]?

In my opinion this will be a step forward to a transparent and local
reproducible build environment. Afair Harald was quite clear about
this requirement?



>> and "Wiki" is an ancient African word for "hopelessly outdated"...
>> We appreciate every checked and/or corrected facts on those pages,
>> and the Jenkins_Node_Setup could probably be merged into the Jenkins one.

Thanks for pointing this out, usual I am hesitating to correct
something to avoid being called nit-picky.



>> >> [2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
>> This sounds like we want to use it!

Alright, so I will work on the following migrations:
     - "as is" -> "JobDSL"
     - "as is" -> "Pipeline" (probably just for Max)


>> so far that doesn't look like an improvement of
>> #!/bin/sh
>> ./contrib/jenkins.sh

Agreed, the biggest advantages of Pipeline are:

-  the "eye-candy" (but who cares? :)

- easy artifacts sharing across different steps, which run on
different nodes (but afaics you don't even use the "Copy Artifacts
Plugin", which makes this argument pointless).

- the duration/sub-steps can be easily seen (as mentioned by Max), but
this can be achieved which some simple python scripts/plugins +
influxDB + grafana as well.



Neels, can you please help me fixing the following build error [3]:

01:59:20 checking dbi/dbd.h usability... no
01:59:20 checking dbi/dbd.h presence... no
01:59:20 checking for dbi/dbd.h... no
01:59:20 configure: error: DBI library is not installed
01:59:22 Build step 'Execute shell' marked build as failure
01:59:22 [WARNINGS] Skipping publisher since build result is FAILURE
01:59:22 Finished: FAILURE


I already added the following package to the image, by:

docker run ........ osmocom:amd64 /bin/bash -c sudo apt-get install -y
r-cran-dbi; /build/contrib/jenkins.sh

but the r-cran-dbi package doesn't solve the issue, which package is needed?

Why is this build dependency not part of the Docker image?


Andr?

[1] https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
[2] https://hub.docker.com/
[3] https://jenkins.blobb.me/job/openBSC_multi-configuration/label=master/36/console

2017-03-06 14:53 GMT+01:00 Neels Hofmeyr <nhofmeyr at sysmocom.de>:
> On Sun, Mar 05, 2017 at 10:03:20PM +0100, Klaus M?ller wrote:
>> Hi,
>>
>> >> So, what would be the actual benefits?
>>
>> >>* complete job config in git? (to be confirmed)
>>
>> Just to clarify, the Pipeline plugin on its own doesn't provide a
>> complete config handling in git. This means only the code shown in
>> image [3] lives in git, configurations within the "General" or "Build
>> Triggers" tab will still be handled in the web-ui.***
>> For the sake of completeness the JobDSL plugin [2][ has to be mention.
>> This plugin allows to put the entire job configuration to git and
>> deploys jobs without previous manual setup in the web-ui.
>>
>> >>* faster because less things have to be rebuilt? (to be confirmed)
>>
>> Somehow I doubt that this improvement dependents on the Pipeline
>> plugin, but I know to few details to have an opinion based on facts.
>> :)
>>
>> >>* faster because easily parallelizable/easier integration with docker? (to be
>> >>  confirmed)
>>
>> first point: e.g. Pipeline can trigger "down-stream-projects/stages"
>> after one specific part of a
>> parallel-block/matrix-configurations-project is finished.
>>
>> second point: Pipeline brings a docker wrapper, so one don't have to
>> pass one build script holding the entire build sequence when invoking
>> the container. You can invoke as much scripts as you want within the
>> docker wrapper. (no black-box god-script :)
>
> This sounds to me like Pipelines don't actually provide any of the features I
> was dreaming about, except maybe easier docker integration?
>
>> Is this mailing list the appropriate communication channel?
>
> yes.
>
>> Is there already a osmocom ci wiki page, which I overlooked? I'd be
>> happy to contribute!
>
> well, searching osmocom.org for jenkins gave me
>
> https://osmocom.org/projects/cellular-infrastructure/wiki/Jenkins
> https://osmocom.org/projects/osmocom-servers/wiki/Jenkins_Node_Setup
> https://osmocom.org/projects/cellular-infrastructure/wiki/Nightly_Builds
> [...]
>
> and "Wiki" is an ancient African word for "hopelessly outdated"...
> We appreciate every checked and/or corrected facts on those pages,
> and the Jenkins_Node_Setup could probably be merged into the Jenkins one.
>
>
>> [1] https://jenkins.blobb.me/view/osmocom/
>
> nice, a sleepy pipeline ;)
>
>> [2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
>
> This sounds like we want to use it!
>
>> [2.5] https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.DslFactory.pipelineJob
>> [3] http://tinyurl.com/zbavrtp
>
> direct link without javascript cruft: https://media.comsysto.com/images/2016-11-23-jenkins-pipelines/new-jenkins-pipeline-config--c.png
>
> so far that doesn't look like an improvement of
>
>   #!/bin/sh
>   ./contrib/jenkins.sh
>
> ~N
>
> --
> - Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
> =======================================================================
> * sysmocom - systems for mobile communications GmbH
> * Alt-Moabit 93
> * 10559 Berlin, Germany
> * Sitz / Registered office: Berlin, HRB 134158 B
> * Gesch?ftsf?hrer / Managing Directors: Harald Welte



-- 
Gru?
Blobb


From holger at freyther.de  Mon Mar  6 20:51:52 2017
From: holger at freyther.de (Holger Freyther)
Date: Mon, 6 Mar 2017 21:51:52 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
 <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
Message-ID: <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>


> On 6 Mar 2017, at 15:28, Holger Freyther <holger at freyther.de> wrote:
> 

Hi!


> I was working on a manual testcase for DTMF handling.. and I am able to reproduce this now. Will analyze it now and then return to DTMF.

I don't have a fix yet but a workaround. One can patch sofia-sip to not use
the IP_RECVERR sockopt. Either patch it out or configure with an ac_... var
to not enable the feature.


Sofia SIP is using the IP_RECVERR socket option to understand if an error
occurred on send (or after, e.g. ICMP unreachable). In my case I tried to
the INVITE to 127.0.0._2_:5060 and the kernel knows that no one is there
and enqueues an error. The error can only be read with recvmsg+MSG_ERRQUEUE.
This happens in sofia-sip in tport_udp_error and is called by the
tport_error_event function.

Now to the issue. There are three ways to use sofia-sip:

	* Just call it to poll every X units of time (like LCR)
	* Implement the complex vtable for event loop integration
	* Integrate with glib to have the vtable thing work


I had ruled out polling early (it wastes energy and has higher latency),
and picked glib as it seemed easier to integrate than the abstraction of
sofia-sip.

After sofia-sip enables the IP_RECVERR option it is doing:

	events |= SU_WAIT_ERR;

This is then registered with g_source_add_poll, e.g. like:

	0xb7fd7573 in su_source_register (self=0x807d564, root=0x807d858, wait=0xbfffef2c, 
    	callback=0xb7f45fdd <tport_wakeup_pri>, arg=0x8081548, priority=0) at su_source.c:650
	650	  g_source_add_poll(self->sup_source, (GPollFD*)&self->sup_waits[n]);
	(gdb) p *(su_wait_t *) 0x8081860
	$2 = {fd = 5, events = 9, revents = 0}

so SU_WAIT_ERR is set.. but when glib is calling us:


	(gdb) p fds[2]
	$23 = {fd = 5, events = 1, revents = 0}


I am currently figuring out where it is mapped and lost and will then
try to find a conclusion for this.


holger

From holger at freyther.de  Mon Mar  6 21:08:57 2017
From: holger at freyther.de (Holger Freyther)
Date: Mon, 6 Mar 2017 22:08:57 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
 <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
 <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>
Message-ID: <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>


> On 6 Mar 2017, at 21:51, Holger Freyther <holger at freyther.de> wrote:
> 
> 
>> On 6 Mar 2017, at 15:28, Holger Freyther <holger at freyther.de> wrote:
>> 
> 
> Hi!
> 


> I am currently figuring out where it is mapped and lost and will then
> try to find a conclusion for this.


glib/gmain.c:g_main_context_query

      /* In direct contradiction to the Unix98 spec, IRIX runs into
       * difficulty if you pass in POLLERR, POLLHUP or POLLNVAL
       * flags in the events field of the pollfd while it should
       * just ignoring them. So we mask them out here.
       */
      events = pollrec->fd->events & ~(G_IO_ERR|G_IO_HUP|G_IO_NVAL);


in sofia-sip:

#define SU_WAIT_ERR     POLLERR

we should probably map this POLLPRI.. not sure how to fix this up. Maybe
copy the glib code and tweak it, start implementing a direct osmocom
version... Not sure how to progress and if people are willing to patch
sofia-sip or not. :(


holger


From holger at freyther.de  Tue Mar  7 14:16:01 2017
From: holger at freyther.de (Holger Freyther)
Date: Tue, 7 Mar 2017 15:16:01 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
 <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
 <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>
 <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>
Message-ID: <AB3E187F-4390-46D5-ACA3-6FE78F89EEA1@freyther.de>


> On 6 Mar 2017, at 22:08, Holger Freyther <holger at freyther.de> wrote:
> 


Hi!


> we should probably map this POLLPRI.. not sure how to fix this up. Maybe
> copy the glib code and tweak it, start implementing a direct osmocom
> version... Not sure how to progress and if people are willing to patch
> sofia-sip or not. :(

so there is a simple workaround to always signal POLLERR and two changes
to map select to poll instead of poll to select. They seem to work but are
bigger than the simple workaround.

All can be seen in the osmo-sip-connector project on gerrit.osmocom.org.
If you could try any of the two that would be nice.

holger


From laforge at gnumonks.org  Tue Mar  7 15:06:23 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Tue, 7 Mar 2017 16:06:23 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
 <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
 <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>
 <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>
Message-ID: <20170307150623.dhxbey6uvbnkhrt4@nataraja>

Hi Holger,

On Mon, Mar 06, 2017 at 10:08:57PM +0100, Holger Freyther wrote:

> glib/gmain.c:g_main_context_query
> 
>       /* In direct contradiction to the Unix98 spec, IRIX runs into
>        * difficulty if you pass in POLLERR, POLLHUP or POLLNVAL
>        * flags in the events field of the pollfd while it should
>        * just ignoring them. So we mask them out here.
>        */
>       events = pollrec->fd->events & ~(G_IO_ERR|G_IO_HUP|G_IO_NVAL);

I think we should try to get this fixed in upstream.  If there's an IRIX
work-around, then it shuold be a compile-time decision and only enabled
on IRIX, right?

It won't help the problem in the short term, as fixed/updated glib would
first have to dissipate through their next release, get picked up by
distributions, etc. - but sooner or later somebody else will run into
the same trap, with glib disabling POLERR on Linux and thus destroying
quite a bit of capability the operating system offers.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From laforge at gnumonks.org  Tue Mar  7 15:03:56 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Tue, 7 Mar 2017 16:03:56 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <AB3E187F-4390-46D5-ACA3-6FE78F89EEA1@freyther.de>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
 <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
 <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>
 <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>
 <AB3E187F-4390-46D5-ACA3-6FE78F89EEA1@freyther.de>
Message-ID: <20170307150356.b2vns3enmdivdfi5@nataraja>

Hi Holger,

On Tue, Mar 07, 2017 at 03:16:01PM +0100, Holger Freyther wrote:
> so there is a simple workaround to always signal POLLERR and two changes
> to map select to poll instead of poll to select. They seem to work but are
> bigger than the simple workaround.

what's the problem / disadvantage with the simple POLERR
hack/workaround? Yes, it's not elegant, but it should come without any
performance overhead, as poll will only return when actual errors are
pending in the queue.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From holger at freyther.de  Tue Mar  7 15:40:55 2017
From: holger at freyther.de (Holger Freyther)
Date: Tue, 7 Mar 2017 16:40:55 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <20170307150356.b2vns3enmdivdfi5@nataraja>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
 <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
 <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>
 <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>
 <AB3E187F-4390-46D5-ACA3-6FE78F89EEA1@freyther.de>
 <20170307150356.b2vns3enmdivdfi5@nataraja>
Message-ID: <450B4678-5833-4B08-9CB2-24426D2CE203@freyther.de>


> On 7 Mar 2017, at 16:03, Harald Welte <laforge at gnumonks.org> wrote:
> 
> Hi Holger,
> 

Hey LaForge,

> what's the problem / disadvantage with the simple POLERR
> hack/workaround? Yes, it's not elegant, but it should come without any
> performance overhead, as poll will only return when actual errors are
> pending in the queue.

The logic is FD_ISSET(fd, &readset) => POLLIN | POLLERR. We speculate
that there is an error for every possible and force another recvmsg call.

Not an act of beauty but only done when there is work anyway and not
performance critical. At the same time the emulate select will poll seems
to be working(tm) so we might use that.


holger




From holger at freyther.de  Tue Mar  7 15:47:49 2017
From: holger at freyther.de (Holger Freyther)
Date: Tue, 7 Mar 2017 16:47:49 +0100
Subject: osmo-sip-connector in bad state
In-Reply-To: <20170307150623.dhxbey6uvbnkhrt4@nataraja>
References: <CAJcGFd6ZbyZ2a9tcFF5Pzj_rAc2m=RKC_S1qNZcQJfB2se1uBg@mail.gmail.com>
 <4514C195-4509-416D-A3A2-97C17C682CED@freyther.de>
 <CAJcGFd7RGSifiOoZuiTukp-BC34A4rEQO1K2XDwx-7xeb6X5Hw@mail.gmail.com>
 <26031043-CBF8-4602-BCDA-AB8C28C4F0E4@freyther.de>
 <AC00EB5F-0206-47BA-BADC-49681AB41DC0@freyther.de>
 <85A4CD8D-4AF2-49C7-9901-A596A9FC56F3@freyther.de>
 <458C48A9-5CEE-45EF-9D87-79F83D3C9E98@freyther.de>
 <5154742F-3A31-435B-BAC8-15FF23BDB65E@freyther.de>
 <C4447E96-C579-4311-8156-846463FA52FB@freyther.de>
 <20170307150623.dhxbey6uvbnkhrt4@nataraja>
Message-ID: <6D7E8F71-32E5-40B6-93E6-561F8CF04843@freyther.de>


> On 7 Mar 2017, at 16:06, Harald Welte <laforge at gnumonks.org> wrote:
> 
> Hi Holger,
> 

Hey!



> I think we should try to get this fixed in upstream.  If there's an IRIX
> work-around, then it shuold be a compile-time decision and only enabled
> on IRIX, right?
> 
> It won't help the problem in the short term, as fixed/updated glib would
> first have to dissipate through their next release, get picked up by
> distributions, etc. - but sooner or later somebody else will run into
> the same trap, with glib disabling POLERR on Linux and thus destroying
> quite a bit of capability the operating system offers.

I didn't know how poll works (hehe, used select all my life). Putting these
into pollfd.events have no effect only the kernel will set the on the
revents. For us they would indicate the intention of sofia-sip but then the
RECVERR will not be signaled in the exception set unless we set another
socket option (for a socket we can't access directly).

So the joke is on me and I didn't know what I was doing when implementing
poll with select. So...

a.) We do POLLIN | POLLERR when a socket becomes readable
b.) We merge the code to use ppoll (added in 2.6.14)

cheers
	holger


From gerrit-no-reply at lists.osmocom.org  Tue Mar  7 22:18:46 2017
From: gerrit-no-reply at lists.osmocom.org (Holger Freyther)
Date: Tue, 7 Mar 2017 22:18:46 +0000
Subject: [PATCH] osmo-pcap[master]: debian: Make a new release with the new
 feature
Message-ID: <gerrit.1488925126540.Ibe86b761b494e0fb78bbbc78e3c1982e44185750@gerrit.osmocom.org>


Review at  https://gerrit.osmocom.org/1999

debian: Make a new release with the new feature

Change-Id: Ibe86b761b494e0fb78bbbc78e3c1982e44185750
---
M debian/changelog
1 file changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/99/1999/1

diff --git a/debian/changelog b/debian/changelog
index fd4259c..6e4df55 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+osmo-pcap (0.0.11) UNRELEASED; urgency=medium
+
+  * Add "source ip A.B.C.D" option to use specific address.
+
+ -- Holger Hans Peter Freyther <holger at moiji-mobile.com>  Tue, 17 Jan 2017 09:12:52 +0100
+
 osmo-pcap (0.0.10) unstable; urgency=medium
 
   * New release with new features

-- 
To view, visit https://gerrit.osmocom.org/1999
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibe86b761b494e0fb78bbbc78e3c1982e44185750
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>


From gerrit-no-reply at lists.osmocom.org  Tue Mar  7 22:18:46 2017
From: gerrit-no-reply at lists.osmocom.org (Holger Freyther)
Date: Tue, 7 Mar 2017 22:18:46 +0000
Subject: [PATCH] osmo-pcap[master]: debian: Add -dbg packages for the
 osmo-pcap-client and osmo-...
Message-ID: <gerrit.1488925126540.I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4@gerrit.osmocom.org>


Review at  https://gerrit.osmocom.org/2000

debian: Add -dbg packages for the osmo-pcap-client and osmo-pcap-server

Currently looking at a weird issue. Make it possible to install the
-dbg packages.

Change-Id: I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4
---
M debian/changelog
M debian/control
M debian/rules
3 files changed, 15 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.osmocom.org:29418/osmo-pcap refs/changes/00/2000/1

diff --git a/debian/changelog b/debian/changelog
index 6e4df55..13bce30 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,6 +1,7 @@
 osmo-pcap (0.0.11) UNRELEASED; urgency=medium
 
   * Add "source ip A.B.C.D" option to use specific address.
+  * Add osmo-pcap-client-dbg/osmo-pcap-server-dbg package
 
  -- Holger Hans Peter Freyther <holger at moiji-mobile.com>  Tue, 17 Jan 2017 09:12:52 +0100
 
diff --git a/debian/control b/debian/control
index 3364e34..2677bc0 100644
--- a/debian/control
+++ b/debian/control
@@ -17,3 +17,13 @@
 Architecture: any
 Depends: ${shlibs:Depends}, ${misc:Depends}
 Description: Collect traces from other systems.
+
+Package: osmo-pcap-client-dbg
+Architecture: any
+Depends: osmo-pcap-client (= ${binary:Version})
+Description: Debug symbols of osmo-pcap-client-dbg
+
+Package: osmo-pcap-server-dbg
+Architecture: any
+Depends: osmo-pcap-server (= ${binary:Version})
+Description: Debug symbols of osmo-pcap-server-dbg
diff --git a/debian/rules b/debian/rules
index 78ddc43..872097c 100755
--- a/debian/rules
+++ b/debian/rules
@@ -39,3 +39,7 @@
 
 	install -d -m 0755 $(CURDIR)/debian/osmo-pcap-server/etc/cron.daily/
 	install -m 0755 $(CURDIR)/contrib/osmo_pcap_clean_old $(CURDIR)/debian/osmo-pcap-server/etc/cron.daily/
+
+override_dh_strip:
+	dh_strip -posmo-pcap-client --dbg-package=osmo-pcap-client-dbg
+	dh_strip -posmo-pcap-server --dbg-package=osmo-pcap-server-dbg

-- 
To view, visit https://gerrit.osmocom.org/2000
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>


From nhofmeyr at sysmocom.de  Tue Mar  7 23:39:23 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Wed, 8 Mar 2017 00:39:23 +0100
Subject: libosmocore build failure cascade
Message-ID: <20170307233923.GA31335@my.box>

As you may have noticed, Vadim's new coding generation code caused numerous
builds to fall on their faces. The reasons why what failed are not so trivial,
so let me explain.

In short,

- we place generated artifacts in the $builddir, which ensures clean builds
  after removing the $builddir. Because we do that, we sometimes need to also
  -I$(builddir)/include for builds where $(builddir) != $(srcdir).

- for generated files, it is important to have proper make dependencies to
  rebuild them when the sources for generation change.

- in build jobs, it is important to wipe the workspace clean, particularly in
  the presence of generated source files that don't have proper make
  dependencies.


What I saw today:

(1) libosmocore master build failed
https://jenkins.osmocom.org/jenkins/job/libosmocore/866/

(2) gerrit libosmocore patch checking build *succeeded* for the failing patch

(3) libosmocore build actually first *succeeded* even in clean src dir

(4) libosmocore build from different dir on clean *system* failed

(5) new generated files ended up in $srcdir

(6) debian build complained about files present but not installed


My first guess was: the gerrit job fails to clean the workspace and hence
works because old headers remain; the master build fails because it properly
cleans up the workspace and checks building from another dir. Not at all :)

(1)
In fact the libosmocore *master* build failed because it did not properly clean
up the workspace. The old headers remained, but lacked some symbols that were
now undefined during compilation.
--> added rm -rf *; git checkout . to the libosmocore master build.
--> https://gerrit.osmocom.org/2002 ensures regeneration by changed gen code
(If there are more deps that could be added there, please go ahead!)

(2)
The gerrit libosmocore patch succeeded because it properly cleans up the
workspace, and does not check building from a separate dir. So it regenerated
all headers in $srcdir, which is fine when also building in $srcdir.
--> https://gerrit.osmocom.org/1998 tests separate dir in jenkins.sh

(3)
If I still have libosmocore headers in e.g. /usr/local/include, the build will
take those when missing in the src tree and may actually succeed. So I had to
go and rm -rf /usr/local/include/osmocom to reproduce the failure. :/

(4)
Build from different dir fails because coding/gsm0503_interleaving.c #includes
a header that is generated in $builddir (already before this patch), but
coding/Makefile.am did not -I$(builddir)/include.
--> https://gerrit.osmocom.org/2003 adds -I builddir in coding/

(5)
While generating files in $srcdir does not necessarily break builds, it's
better to write to $builddir instead -- removing $builddir then clears the
build state completely. Fixed up to generate to $builddir, also in:
--> https://gerrit.osmocom.org/1996 generates gsm0503 to builddir

(6)
*All* files part of 'make install' need to be mentioned in debian/*, either as
files to be ignored or files to be installed. Max fixed it in:
--> https://gerrit.osmocom.org/1989
--> do we want to add deb build checking to the gerrit job?


I spent some time on cleaning failures from that patch, but then spent some
extra time to help us not run into these problems again... Thanks for being
aware of these issues.

~N


-- 
- Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Gesch?ftsf?hrer / Managing Directors: Harald Welte
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170308/3c670a38/attachment.bin>

From gerrit-no-reply at lists.osmocom.org  Wed Mar  8 08:23:32 2017
From: gerrit-no-reply at lists.osmocom.org (Max)
Date: Wed, 8 Mar 2017 08:23:32 +0000
Subject: osmo-pcap[master]: debian: Add -dbg packages for the osmo-pcap-client
 and osmo-...
In-Reply-To: <gerrit.1488925126540.I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4@gerrit.osmocom.org>
References: <gerrit.1488925126540.I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4@gerrit.osmocom.org>
Message-ID: <mailman.519.1641744211.42898.openbsc@lists.osmocom.org>


Patch Set 1: Code-Review+1

-- 
To view, visit https://gerrit.osmocom.org/2000
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Max <msuraev at sysmocom.de>
Gerrit-HasComments: No


From gerrit-no-reply at lists.osmocom.org  Wed Mar  8 08:24:11 2017
From: gerrit-no-reply at lists.osmocom.org (Max)
Date: Wed, 8 Mar 2017 08:24:11 +0000
Subject: osmo-pcap[master]: debian: Make a new release with the new feature
In-Reply-To: <gerrit.1488925126540.Ibe86b761b494e0fb78bbbc78e3c1982e44185750@gerrit.osmocom.org>
References: <gerrit.1488925126540.Ibe86b761b494e0fb78bbbc78e3c1982e44185750@gerrit.osmocom.org>
Message-ID: <mailman.520.1641744211.42898.openbsc@lists.osmocom.org>


Patch Set 1: Code-Review+1

-- 
To view, visit https://gerrit.osmocom.org/1999
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibe86b761b494e0fb78bbbc78e3c1982e44185750
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Max <msuraev at sysmocom.de>
Gerrit-HasComments: No


From msuraev at sysmocom.de  Wed Mar  8 12:13:40 2017
From: msuraev at sysmocom.de (Max)
Date: Wed, 8 Mar 2017 13:13:40 +0100
Subject: libosmocore build failure cascade
In-Reply-To: <20170307233923.GA31335@my.box>
References: <20170307233923.GA31335@my.box>
Message-ID: <26b00913-1e03-c23a-ef5a-d4482c87b4a3@sysmocom.de>

Definitely +1 on this. It'll take a bit longer but it's way more 
convenient to avoid breaking builds than to fix them afterwards. See 
also gerrit 1981 <https://gerrit.osmocom.org/1981> for example.


On 08.03.2017 00:39, Neels Hofmeyr wrote:
> --> do we want to add deb build checking to the gerrit job?
>

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From nhofmeyr at sysmocom.de  Wed Mar  8 16:48:26 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Wed, 8 Mar 2017 17:48:26 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
 <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
Message-ID: <20170308164826.GD31335@my.box>

FYI, Blobb visited us in the sysmocom office yesterday, and we had a personal
conversation on the jenkins build setup. It was good meeting you in person,
Andr? :)  For the sake of this ML, let me answer briefly here as well.

On Mon, Mar 06, 2017 at 04:15:20PM +0100, Andr? Boddenberg wrote:
> When we speak about the "easier docker integration", we mean that
> everyone would be happy if not every matrix-configuration axis builds
> all deps like libosmocore, libosmo-netif etc pp, right?

We're rebuilding everything simply because no-one has found time to make this
more efficient yet. There is no need to rebuild all of libosmocore through to
to libsmpp34 for every matrix cell.

It would also be useful to build all libosmo* only once for each update of the
master branch, and in the dependent builds (openbsc, openbsc-gerrit, ...)
always re-use the last successfully built binaries of the dependencies.

> To address this issue I'd like to create a local temporary docker
> image which extends osmocom:amd64  and holds all deps. This image can
> then be used for all openBSC builds and the temporary local docker
> image can be removed as a "post-build"-step (which should get
> triggered regardless the build result).

How about each built library extends a docker image produced by a previous
dependency's last successful build? e.g. if libosmocore rebuilds, that updates
the libosmocore docker image, then libosmo-abis is triggered and adds itself,
producing a last-successfully-built libosmo-abis docker image, and so on, down
to openbsc re-using a docker image that already holds all its dependencies?

Even though our libraries don't necessarily have a linear dependency, it could
make sense to artificially define such a linear line of dependencies (e.g. even
though libsmpp doesn't need libosmo-abis, we make libsmpp build on top of the
libosmo-abis docker to collect it into the dependecies docker image).  But,
then again, if libosmocore succeeded and libosmo-abis failed, we would omit a
libsmpp build for no reason. So maybe it would also make sense to somehow build
the non-dependent libraries independently and later combine into a joint docker
image?? ...I would leave that up to your choices, just brain storming...

> A slightly different topic, did you think about pushing your docker
> images to hub.docker.com [2]?
> 
> In my opinion this will be a step forward to a transparent and local
> reproducible build environment. Afair Harald was quite clear about
> this requirement?

No idea / not aware of it / no opinion so far :)
I was once invited to design a logo for the reproducible builds project, but
haven't found time for that (yet?). That's about all I know about the topic...

> >> and "Wiki" is an ancient African word for "hopelessly outdated"...
> Thanks for pointing this out, usual I am hesitating to correct
> something to avoid being called nit-picky.

My middle name is "Nit-pick" :P
It's sometimes really hard for me to ignore a small detail that is obviously
wrong for the benefit of moving ahead faster...

In the Wiki's case, there is no drawback of correcting mistakes -- no noise in
code review or mailing lists is generated, so if you have the time, just do it,
as much as you like.

> >> >> [2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
> >> This sounds like we want to use it!
> 
> Alright, so I will work on the following migrations:
>      - "as is" -> "JobDSL"
>      - "as is" -> "Pipeline" (probably just for Max)

hehe

Just to be clear, I'm merely one person here, just because I like the sound of
Job-DSL, it doesn't mean everyone else does. I think it would be good to see an
example of an openbsc build using it and then let everyone decide.

> >> so far that doesn't look like an improvement of
> >> #!/bin/sh
> >> ./contrib/jenkins.sh
> 
> Agreed, the biggest advantages of Pipeline are:
> 
> -  the "eye-candy" (but who cares? :)
> 
> - easy artifacts sharing across different steps, which run on
> different nodes (but afaics you don't even use the "Copy Artifacts
> Plugin", which makes this argument pointless).

But if we introduce docker images as artifacts, this could become useful?

> - the duration/sub-steps can be easily seen (as mentioned by Max), but
> this can be achieved which some simple python scripts/plugins +
> influxDB + grafana as well.

I personally mostly care about the overall time a build takes, so that I get
V+1/V-1 on my gerrit patches faster.

> Neels, can you please help me fixing the following build error [3]:
> 
> 01:59:20 checking dbi/dbd.h usability... no
> 01:59:20 checking dbi/dbd.h presence... no
> 01:59:20 checking for dbi/dbd.h... no
> 01:59:20 configure: error: DBI library is not installed

Install libdbi-dev and libdbd-sqlite3.

> I already added the following package to the image, by:
> 
> docker run ........ osmocom:amd64 /bin/bash -c sudo apt-get install -y
> r-cran-dbi; /build/contrib/jenkins.sh

r-cran-dbi?? what is that? :)

> Why is this build dependency not part of the Docker image?

That's a good question, it should be, right?
According to the dockerfile, libdbi-dev is actually installed there.
So is libdbd-sqlite3.

Hard to say why configure tells you that that is missing. It should work.

I've attached the Dockerfile config used to generate the docker image for
openbsc. There's some stuff in there not needed for openbsc in particular, e.g.
the smalltalk things near the end. And it should probably use the osmo-ci git
repos instead of echoing the osmo-deps.sh to /usr/local/bin manually. Actually,
our openbsc-gerrit build job has osmo-ci in ~/bin and links that to /build_bin
in the docker build, also adding /build_bin to the PATH ... that's a bit
cumbersome and is my fault from when I wasn't too familiar with docker yet.
These things could be streamlined.

docker run --rm=true -e HOME=/build -e MAKE=make -e PARALLEL_MAKE="$PARALLEL_MAKE" \
-e IU="$IU" -e SMPP="$SMPP" -e MGCP="$MGCP" -e PATH="$PATH:/build_bin" \
-e OSMOPY_DEBUG_TCP_SOCKETS="1" -w /build -i -u build -v "$PWD:/build" \
-v "$HOME/bin:/build_bin" osmocom:amd64 /build/contrib/jenkins.sh


OT: does your nick "Blobb" indicate affinity to Binary Large Ob(b)jects?
... like docker images? ;)

~N

-- 
- Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Gesch?ftsf?hrer / Managing Directors: Harald Welte
-------------- next part --------------
FROM debian:jessie

RUN dpkg --add-architecture i386 && \
    DEBIAN_FRONTEND=noninteractive apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get upgrade -y && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends wget make

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends gcc g++ make git
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends sudo
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends unzip bzip2  python

# match the outside user
RUN useradd --uid=1000 build
#RUN echo "build ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/build

RUN mkdir /build
RUN chown build:build /build

# still generic
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends doxygen git asciidoc rsync coccinelle

# for GNU smalltalk
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends flex bison libsigsegv-dev libffi-dev texinfo

# libosmo-sccp/abis/etc
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends libortp-dev libpcsclite-dev libsctp-dev libfftw3-dev libsnmp-dev  libusb-1.0-0-dev libtalloc-dev libgnutls28-dev

# OsmocomBB
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends gcc-arm-none-eabi

# building
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends libtool pkg-config automake autoconf

# Linux kernel
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y bc

# and all
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y doxygen g++ libtalloc-dev libpcsclite-dev make gcc pkgconf libtool autoconf autoconf-archive automake libortp-dev asciidoc mscgen git libsctp-dev libpcap-dev osc libc-ares-dev libgps-dev  libsofia-sip-ua-glib-dev  libssl-dev libsqlite3-dev libusb-dev libffi-dev libfftw3-dev flex bison libdbi-dev  libsnmp-dev libncurses5-dev libgsm1-dev python-minimal python3 libdbd-sqlite3 cppcheck htop  libgmp-dev gawk texinfo flex bison bc  libsigsegv-dev libffi-dev libusb-1.0-0-dev  libreadline-dev debhelper devscripts gcc-arm-none-eabi git-buildpackage dh-systemd dh-autoreconf bc openssh-client 

RUN git clone git://git.osmocom.org/python/osmo-python-tests && cd osmo-python-tests && python2 ./setup.py install # 2017-03-06


RUN echo "#!/bin/sh \n\
 \n\
if ! test -d \$1; \n\
then \n\
  git clone git://git.osmocom.org/\$1 \$1 \n\
fi \n\
\n\
cd \$1 \n\
git fetch origin \n\
git reset --hard origin/master" > /usr/local/bin/osmo-deps.sh
RUN chmod +x /usr/local/bin/osmo-deps.sh


RUN git clone http://git.savannah.gnu.org/r/smalltalk.git
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends flex libsigsegv-dev bison libgmp-dev texinfo zip libltdl-dev 
RUN cd smalltalk && autoreconf --install --force && ./configure && make install
RUN rm -rf smalltalk
RUN git clone https://github.com/zecke/petitparser.git && cd petitparser && gst-package package.xml
RUN git clone https://github.com/zecke/petitparser-tests.git && cd petitparser-tests && gst-package package.xml
RUN git clone git://git.osmocom.org/smalltalk/osmo-st-logging && cd osmo-st-logging && gst-package package.xml
RUN git clone git://git.osmocom.org/smalltalk/osmo-st-core && cd osmo-st-core && gst-package package.xml
RUN git clone git://git.osmocom.org/smalltalk/osmo-st-network && cd osmo-st-network && gst-package package.xml
RUN git clone git://git.osmocom.org/smalltalk/osmo-st-gsm && cd osmo-st-gsm && gst-package --test package.xml
RUN git clone git://git.osmocom.org/smalltalk/osmo-st-openbsc-test && cd osmo-st-openbsc-test/fakebts && gst-package --test package.xml
RUN rm -rf petitparser petitparser-tests osmo-st-logging ost-st-core osmo-st-network osmo-st-gsm osmo-st-openbsc-test
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python-pip
RUN pip install timeout_decorator
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170308/a9775747/attachment.bin>

From nhofmeyr at sysmocom.de  Wed Mar  8 17:23:20 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Wed, 8 Mar 2017 18:23:20 +0100
Subject: New Defects reported by Coverity Scan for Osmocom
In-Reply-To: <58c033ab49f3b_41212f732070232@ss1435.mail>
References: <58c033ab49f3b_41212f732070232@ss1435.mail>
Message-ID: <20170308172320.GE31335@my.box>

From libosmo-abis beb10ef02a10d73537a97f6f21aad36664c9b266
"add basic unixsocket support":

On Wed, Mar 08, 2017 at 08:39:07AM -0800, scan-admin at coverity.com wrote:
> *** CID 163920:    (TAINTED_SCALAR)
> /source-Osmocom/libosmo-abis/src/input/unixsocket.c: 90 in unixsocket_read_cb()
> 84     	uint8_t controldata;
> 85     	int ret;
> 86     
> 87     	if (!msg)
> 88     		return -ENOMEM;
> 89     
> >>>     CID 163920:    (TAINTED_SCALAR)
> >>>     Calling function "read" taints argument "msg->data".
> 90     	ret = read(bfd->fd, msg->data, UNIXSOCKET_ALLOC_SIZE - 16);

What this seems to complain about is that we're writing arbitrary data to
msg->data, and in lapd_receive() pass this to a LOGP that prints a hexdump of
it. In osmo_hexdump, we of course do hex_chars[buf[i] >> 4] which we know to
always work out to one of '0'..'f' for *any* data. This point seems to be
missed by covertiy scan and it believes we may end up with an invalid index to
hex_chars[]. So this is a false positive. Marked it as such.

> *** CID 163919:  Security best practices violations  (STRING_OVERFLOW)
> /source-Osmocom/libosmo-abis/src/input/unixsocket.c: 236 in unixsocket_line_update()
> 230     	struct unixsocket_line *config;
> 231     	char sock_path[PATH_MAX];
> 232     	int ret = 0;
> 233     	int i;
> 234     
> 235     	if (line->sock_path)
> >>>     CID 163919:  Security best practices violations  (STRING_OVERFLOW)
> >>>     Note: This defect has an elevated risk because the source argument is a parameter of the current function.
> 236     		strcpy(sock_path, line->sock_path);

Please always use osmo_strlcpy(), here with sizeof(sock_path).

> 237     	else
> 238     		sprintf(sock_path, "%s%d", UNIXSOCKET_SOCK_PATH_DEFAULT,
> 239     			line->num);
> 240     
> 241     	LOGP(DLINP, LOGL_NOTICE, "line update (line=%p)\n", line);
> 
> ** CID 163918:  Memory - illegal accesses  (BUFFER_SIZE_WARNING)
> /source-Osmocom/openbsc/openbsc/src/libbsc/bsc_subscriber.c: 83 in bsc_subscr_set_imsi()
> 
> 


The next one is from me, in 6d804b1a7e375213cb4b3e437c2b9b8c68872164
"add struct bsc_subscr, separating libbsc from gsm_subscriber":

> ________________________________________________________________________________________________________
> *** CID 163918:  Memory - illegal accesses  (BUFFER_SIZE_WARNING)
> /source-Osmocom/openbsc/openbsc/src/libbsc/bsc_subscriber.c: 83 in bsc_subscr_set_imsi()
> 77     }
> 78     
> 79     void bsc_subscr_set_imsi(struct bsc_subscr *bsub, const char *imsi)
> 80     {
> 81     	if (!bsub)
> 82     		return;
> >>>     CID 163918:  Memory - illegal accesses  (BUFFER_SIZE_WARNING)
> >>>     Calling strncpy with a maximum size argument of 16 bytes on destination array "bsub->imsi" of size 16 bytes might leave the destination string unterminated.
> 83     	strncpy(bsub->imsi, imsi, sizeof(bsub->imsi));

Again, I should use osmo_strlcpy(), which correctly handles sizeof(buf), other
than strncpy which might leave the string unterminated.

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170308/94bb8dd3/attachment.bin>

From laforge at gnumonks.org  Wed Mar  8 17:24:11 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Wed, 8 Mar 2017 18:24:11 +0100
Subject: RFC: jenkins pipeline (containers and license compliance)
In-Reply-To: <20170308164826.GD31335@my.box>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
 <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
 <20170308164826.GD31335@my.box>
Message-ID: <20170308172411.nchtvqshpybyrc5u@nataraja>

Sorry for the slight detour:

Docker (and other related) container images pose still largely
unresolved license compliance conflicts.  We as Osmocom project are on
the safe side as long as we distribute only source code of our projects,
or binaries built from our source code, where the binaries include the
respective license texts as well as an indication where the
corresponding source for that build can be found.

If we distribute images with binaries in them, we need to be 100% sure
that we can ensure license compliance for everything inside such an
image, for each and every version of that image, and also make sure that
the source code (if not includeD) can be later produced for a given
build even if somebody asks three years later.

So in general, i have big reservations against containers and the way
how people in that area seem to ignore FOSS (and other?) license
compliance.  Maybe that's just my pre-occupation, and they have this all
sorted out.  But to me, so far, it seems like this technology is just
inviting people to commit license infringements.

Before anyone uploads/publishes any containers on our servers or on
hub.docker.com or any other site, please think twice and thrice about
how you ensure compliance withe every license of every software in such
an image. Thanks.

Regards,
	Harald

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From axilirator at gmail.com  Wed Mar  8 18:44:15 2017
From: axilirator at gmail.com (Vadim Yanitskiy)
Date: Thu, 9 Mar 2017 01:44:15 +0700
Subject: libosmocore build failure cascade
Message-ID: <CAPEnp=bB0VUrsdJ6z6xOknd0TLOHH8q-iuKR0+U=f6NE+Ur0Mw@mail.gmail.com>

Hi Neels, Max and others,

Sorry, I didn't expect such build failures. It seems, some of them was
already fixed before I noticed this mail. I was a bit busy with OsmoBTS
testing, so let me some time to go through this issues and let me know,
if I should fix something else.

Before submitting a new change, I do the following:

 - make check
 - make distcheck

Should I test something else in the future?

With best regards,
Vadim Yanitskiy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170309/7619bdee/attachment.htm>

From nhofmeyr at sysmocom.de  Wed Mar  8 22:02:39 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Wed, 8 Mar 2017 23:02:39 +0100
Subject: libosmocore build failure cascade
In-Reply-To: <CAPEnp=bB0VUrsdJ6z6xOknd0TLOHH8q-iuKR0+U=f6NE+Ur0Mw@mail.gmail.com>
References: <CAPEnp=bB0VUrsdJ6z6xOknd0TLOHH8q-iuKR0+U=f6NE+Ur0Mw@mail.gmail.com>
Message-ID: <20170308220239.GI31335@my.box>

On Thu, Mar 09, 2017 at 01:44:15AM +0700, Vadim Yanitskiy wrote:
> Should I test something else in the future?

Basically the Gerrit tests should ensure those things get caught.
I have patches in line to add a build-outside-source-tree check for openbsc.
Still missing is the debian build check, and the same checks for the other libs.

This would be how to check whether the debian build still works:
https://osmocom.org/projects/cellular-infrastructure/wiki/Build_from_Source#Build-debian-packages

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170308/012f4e01/attachment.bin>

From laforge at gnumonks.org  Thu Mar  9 01:08:23 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 9 Mar 2017 02:08:23 +0100
Subject: Thoughts on a next generation MIB/VTY replacement
Message-ID: <20170309010823.pxefxd5ik2vre76p@nataraja>

Hi All,

Using zebra/quagga VTY code was a great idea back then and has served us
nicely in all those years.  I like the general style of the command-line
based interface with its various nodes, the strict syntax checking, the
tab completion and the interactive context-sensivive help.

However, it feels a bit 20ieth-century-ish to have to manually write
code to parse and to save the respective values.  This is not a
productive way to spend our development resources, and it is error
prone.  The "save" can be forgotten, resulting in non-saveable config
parameters.  The save can store values that the "parse" function will
not be able to read again, etc.

Furthermore, the VTY is inherently a human user interface, not intended
for programmatic consumption.  For programmatic access, we have
developed the control interface.  In reality though, only 1% of the
parameters available in the VTY are exported via the control interface.
And rather than adding all the missing bits and pieces with hand-crafted
code to the control interface, we should have a generic way to define a
new configuration value, and that value should then be automatically
parse- and storable via VTY as well as a programmatic interface.

The next "problem' is that the current telnet connections live within
the process of the application.  This means that a user can effectively
"stall" the main application by performing I/O intensive operation on
the VTY, or even on as many VTY/telnet sessions as he wants.  There
shouldn't be any need for this, at least not for something as mundane as
performing configuration changes.  Of course the VTY has different use
cases such as runtime introspection of system state as well as logging.
But still.

Yet another concern is "VTY and telnet port proliferation".
Particularly with the conversion from NITB to BSC+MSC+HLR, we are yet
again getting more network elements with their own telnet ports.
Remembering the port numbers is clumsy.  It would be more convenient if
a single command line interface could provide access to the
configuration and the state of multiple different processes / network
elements.

Last, but not least, the current implementation is fixed to telnet,
without any form of authentication, and without a path to migrate e.g.
to something like SSL or SSH.   I don't think it is a major concern (you
can always SSH to the system and then telnet locally).

So what I had in mind for quite some time (actually since netconf 1.2
about one year ago), is to have some kind of an external "VTY/MIB
daemon" (or even separate daemons for each) which maintains a
hierarchical database of configuration values.  The MIB deamon simply
offers an API (via client library) to GET or SET the individual values,
or to NOTIFY an application about a changed value.  This API is both
used by the actual Osmocom programs to obtain their configuration (and
obtain updates to it during runtime), as well as by the "VTY daemon"
providing interactive shell access to it.  Finally, other external
applications could use the same interface/client library to do the same.
A proxy to SNMP or REST interfaces can be imagined, for even more
interfacing with the outside world.

What I don't like about many existing MIBs I've used (as a
sysadmin/user, not a developer) is their simple type system.  You can
often specify any random value to such a MIB, even one that is
completely useless.  A simple INT or STRING type is not sufficient, we
really want the ability to have ENUM types, to have integers with
ranges, etc. - just like we have in the VTY.

Also, the MIBs I've used typically are nothing more than a hierarchical
key-value store.  They do not contain the information required for
interactive, context-sensitive help needed for the "VTY" feel :(  This
is btw also the problem I have with OpenWRT's uci: It just has keys and
values, with no way to constrain those values to something that's
reasonable to the given program/context that is being configured, and
there's no help or other information associated with those keys and
values.

So what I would like to see in this context, is some way to have a
machine-parseable description of a "MIB/config item", together with
syntax, ranges, enum values, help texts, etc (ideally in the C source
code, possibly in comments?) which is used by some kind of MIB compiler
to build up the hierarchical structure of the MIB and all of its keys as
well as their permitted values and syntax reference.  This information
is then available to the VTY/telnet connections, irrespective of whether
a given application [using those values] is running at all. 

Once an application starts, it can query for the configuration values it
is interested in and populate its internal data structures from it.
Ideally, one would even be able to generate a 'struct' from the MIB
compiler, so that there is no need to explicitly call a "get" function
on each single configuration value, but one simply gets a C-language
'struct' with all the values filled in by the client library.

As stated above, there should be some way how he MIB can notify
applications about changes to certain nodes in the MIB tree, so the
application[s] can react to that.  I don't have a clear picture yet how
transaction logic (like changing multiple values and then committing
them at once) would work, or whether applications should even be able to
reject/revert a modification?

In either case, I just wanted to share my thoughts on this.  I know it
sounds rather complex, but I think the investment in some kind of
unified configuration database system would save us of a lot of
boilerplate code and subtle bugs and inconsistencies.

I've created https://osmocom.org/issues/1975 with above text to keep
track of it, but I think at this point it's more of a mailing list
discussion rather than something we can put into an issue.  I guess the
progress would rather be
* first discuss it here and/or at OsmoDevCon
* create something like a specification in the wiki
* then follow-up with actual tickets on the work items

All of this is brainstorming and vaporware, but I can't help to think of
better ways of doing this.  If somebody has experience with any existing
systems and wants to share if and how they might fit, or what other
ideas are useful to borrow: Please share!

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From alexander.chemeris at gmail.com  Thu Mar  9 07:58:08 2017
From: alexander.chemeris at gmail.com (Alexander Chemeris)
Date: Thu, 9 Mar 2017 10:58:08 +0300
Subject: RFC: jenkins pipeline (containers and license compliance)
In-Reply-To: <20170308172411.nchtvqshpybyrc5u@nataraja>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
 <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
 <20170308164826.GD31335@my.box> <20170308172411.nchtvqshpybyrc5u@nataraja>
Message-ID: <CABmJbFViWm8wu474RNP-O0WEkO_TFuCqbCg_vU-WjmuKYyi6VA@mail.gmail.com>

Hi Harald,


On Mar 8, 2017 8:24 PM, "Harald Welte" <laforge at gnumonks.org> wrote:

Sorry for the slight detour:

Docker (and other related) container images pose still largely
unresolved license compliance conflicts.  We as Osmocom project are on
the safe side as long as we distribute only source code of our projects,
or binaries built from our source code, where the binaries include the
respective license texts as well as an indication where the
corresponding source for that build can be found.

If we distribute images with binaries in them, we need to be 100% sure
that we can ensure license compliance for everything inside such an
image, for each and every version of that image, and also make sure that
the source code (if not includeD) can be later produced for a given
build even if somebody asks three years later.

So in general, i have big reservations against containers and the way
how people in that area seem to ignore FOSS (and other?) license
compliance.  Maybe that's just my pre-occupation, and they have this all
sorted out.  But to me, so far, it seems like this technology is just
inviting people to commit license infringements.

Before anyone uploads/publishes any containers on our servers or on
hub.docker.com or any other site, please think twice and thrice about
how you ensure compliance withe every license of every software in such
an image. Thanks.


Could you elaborate what kind of license infringement does Docker have and
what license issues might arise from publishing eg Osmocom images at some
repository?

I was confident it's ok and we did publish some code to create Docker
containers in the past. I would like to understand potential issues with
this.

Eg what's wrong with images built from publicly available sources without
any binary blobs? And are the issues with Docker itself?

Please excuse typos. Written with a touchscreen keyboard.

--
Regards,
Alexander Chemeris
CTO/Founder Fairwaves, Inc.
https://fairwaves.co
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170309/d105a417/attachment.htm>

From msuraev at sysmocom.de  Thu Mar  9 12:07:09 2017
From: msuraev at sysmocom.de (Max)
Date: Thu, 9 Mar 2017 13:07:09 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
 <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
Message-ID: <12e0893e-b3bf-91bb-9465-a1e6f1a43d1a@sysmocom.de>

Hi!


On 06.03.2017 16:15, Andr? Boddenberg wrote:
>
> Agreed, the biggest advantages of Pipeline are:
>
> -  the "eye-candy" (but who cares? :)
>
> - easy artifacts sharing across different steps, which run on
> different nodes (but afaics you don't even use the "Copy Artifacts
> Plugin", which makes this argument pointless).
>
> - the duration/sub-steps can be easily seen (as mentioned by Max), but
> this can be achieved which some simple python scripts/plugins +
> influxDB + grafana as well.

I don't think that installing and maintaining 2 additional software 
packages + writing custom code is as easy as enabling plugin in jenkins 
and configuring it. Being able to immediately see how long each step in 
the pipeline took is not only eye-candy - it's a great hint as to where 
our potential optimization targets are.

Having said that, if similar visibility could be achieved without using 
pipeline than by all means - go for it.

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From msuraev at sysmocom.de  Thu Mar  9 12:42:32 2017
From: msuraev at sysmocom.de (Max)
Date: Thu, 9 Mar 2017 13:42:32 +0100
Subject: libosmocore build failure cascade
In-Reply-To: <CAPEnp=bB0VUrsdJ6z6xOknd0TLOHH8q-iuKR0+U=f6NE+Ur0Mw@mail.gmail.com>
References: <CAPEnp=bB0VUrsdJ6z6xOknd0TLOHH8q-iuKR0+U=f6NE+Ur0Mw@mail.gmail.com>
Message-ID: <f6ad03d6-2051-3611-9ed0-3e36827a4315@sysmocom.de>

On a related note - are you working on moving OsmoBTS to libosmocoding? 
Are there some missing pieces before this can be done?

Looking forward to your patches.

On 08.03.2017 19:44, Vadim Yanitskiy wrote:
>  I was a bit busy with OsmoBTS
> testing, so let me some time to go through this issues and let me know,
> if I should fix something else.

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From nhofmeyr at sysmocom.de  Thu Mar  9 13:41:37 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 9 Mar 2017 14:41:37 +0100
Subject: Thoughts on a next generation MIB/VTY replacement
In-Reply-To: <20170309010823.pxefxd5ik2vre76p@nataraja>
References: <20170309010823.pxefxd5ik2vre76p@nataraja>
Message-ID: <20170309134137.GA2194@my.box>

On Thu, Mar 09, 2017 at 02:08:23AM +0100, Harald Welte wrote:
> And rather than adding all the missing bits and pieces with hand-crafted
> code to the control interface, we should have a generic way to define a
> new configuration value, and that value should then be automatically
> parse- and storable via VTY as well as a programmatic interface.

+1

> "stall" the main application by performing I/O intensive operation on
> the VTY

I once used a script that connects to nitb's vty, prints the lchan summary and
exits. I ran that once a second. It often managed to crash the NITB! ... I
never actually investigated it, but would have been good to do so.

> Remembering the port numbers is clumsy.  It would be more convenient if
> a single command line interface could provide access to the
> configuration and the state of multiple different processes / network
> elements.
[...]
> This API is both
> used by the actual Osmocom programs to obtain their configuration (and
> obtain updates to it during runtime), as well as by the "VTY daemon"
> providing interactive shell access to it.

So would the central interface simply remember the port numbers for us? No,
you're talking about a "config server" that the individual applications connect
to. On the one hand nice, on the other yet another osmo-entity. Still, +1.

Would osmo-programs require the config server to run?
Would they be able to launch the config daemon automatically -- portably?

> GET SET NOTIFY
> ENUM, integers with ranges, etc.
> machine-parseable description of a "MIB/config item", together with
> syntax, ranges, enum values, help texts, etc (ideally in the C source
> code, possibly in comments?)

Add to the wishlist: an indicator whether changing the value immediately
changes the application's behavior. A boolean unfortunately doesn't cut it,
e.g. some settings take effect when a specific BTS' OML link is restarted.
It could have the values "yes, immediate", "no, needs restart" and "please see
comment" with a separate comment provided for humans.


With global structs like gsm_network, I could imagine a completely automagic
code generation from API comments. The description could contain the global
gsm_network instance's symbol name, and the machine built code could provide
the NITB's side as well as the config server's side, in the NITB storing those
values directly in the global gsm_network instance. Or maybe let the automagic
code create a replacement for the config part of the gsm_network struct. That's
all easy for singleton items.

The default NOTIFY could invoke a simple GET (sometimes storing a value in a
struct is enough to change behavior immediately), but a manual on_notify() can
be specified (presence of on_notify() doesn't guarantee immediate effect).


More complex would be things like fetching a subscriber with a given IMSI first
and applying changes there. Arbitrary complexity of actions to update a value
are thinkable, so we probably won't get around writing it manually? Semi?

But hold on, there are different cases: we don't want to store individual IMSIs
in the config server (that's what the HLR is for), while we do want to store
individual BTS configurations centrally.

For GET/SET/NOTIFY of a BTS element, we would need the API to have an array
index argument, analogous to 'bts 0', 'bts 1'. For 'trx 0' etc we even need
nested array indexes. With a path scheme? GET('bts[2].trx[0].arfcn').

But for operations on a subscriber, we would instead merely send instructions
to the HLR to do/store/say things -- if we want to allow managing subscribers
from the central "vty" application, and if we want to allow querying the
current state of entities (like 'show lchan summary'), we need another class of
items that don't GET/SET things from/to the central config server, but tell an
application to do or say something. Call them COMMAND, and they would feature
arbitrary arguments. In the VTY we just implemented whatever we want to
happen, in this scheme we should explicitly differentiate config items from
commands.

When we have separated BSC from NITB for good, centrally issued COMMANDs could
also make up for the removed information match-up: in the NITB we always knew
which BTS a subscriber was calling from with all of the detailed BTS state.
With a separate BSC we don't anymore -- but the central "vty" could run queries
with MSC and BSC to match up that information conveniently again.


> All of this is brainstorming and vaporware

It would be nice to dabble up some hacky code to discuss about, to sharpen our
teeth on; without spending too much time on it and having in mind to discard it
for something better anytime?

Sounds a bit like re-inventing dbus. (I don't know much about dbus though,
would it make sense to use dbus?? Is it a giant kraken of dependencies?)

Seems to me that with all the wishlist features we have collected now, the
implementation of such a system would actually be quite complex and not a small
effort.

I think the most important feature is that all our config items become a
machine and human API at the same time -- and that is a killer feature for all
those folks asking for web interfaces.

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170309/94e0855b/attachment.bin>

From keith at rhizomatica.org  Thu Mar  9 15:14:45 2017
From: keith at rhizomatica.org (Keith Whyte)
Date: Thu, 09 Mar 2017 15:14:45 +0000
Subject: Thoughts on a next generation MIB/VTY replacement
In-Reply-To: <20170309134137.GA2194@my.box>
References: <20170309010823.pxefxd5ik2vre76p@nataraja>
 <20170309134137.GA2194@my.box>
Message-ID: <D47E45C3-4581-48ED-A45B-A35D90C6CBA6@rhizomatica.org>



On 9 March 2017 14:41:37 CET, Neels Hofmeyr <nhofmeyr at sysmocom.de> wrote:


I once used a script that connects to nitb's vty, prints the lchan
summary and
exits. I ran that once a second. It often managed to crash the NITB!


I do that often, once even forgot and left it running in a screen a few days.
It didn't crash. I cat show lchan to nc and have to make it wait for 1 second otherwise it sometimes misses to vty output, (minimum wait is 1 second with nc) Sometimes i would like to poll lchan faster.
Anyway, not an ideal solution for monitoring the lchans, as you know.
Fairwaves have something using the control port, but maybe expanding the meas feed is a better way to do this kind of thing?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170309/e5ed48e4/attachment.htm>

From k at rhizomatica.org  Thu Mar  9 15:12:04 2017
From: k at rhizomatica.org (k at rhizomatica.org)
Date: Thu, 09 Mar 2017 15:12:04 +0000
Subject: Thoughts on a next generation MIB/VTY replacement
In-Reply-To: <20170309134137.GA2194@my.box>
References: <20170309010823.pxefxd5ik2vre76p@nataraja>
 <20170309134137.GA2194@my.box>
Message-ID: <D5B66F2B-D879-4E64-8E7A-B61A2E6DF4D5@rhizomatica.org>



On 9 March 2017 14:41:37 CET, Neels Hofmeyr <nhofmeyr at sysmocom.de> wrote:

>
>I once used a script that connects to nitb's vty, prints the lchan
>summary and
>exits. I ran that once a second. It often managed to crash the NITB!

I do that often, once even forgot and left it running in a screen a few days.
It didn't crash. I cat show lchan to nc and have to make it wait for 1 second otherwise it sometimes misses to vty output, (minimum wait is 1 second with nc) Sometimes i would like to poll lchan faster.
Anyway, not an ideal solution for monitoring the lchans, as you know.
Fairwaves have something using the control port, but maybe expanding the meas feed is a better way to do this kind of thing?


-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


From alexander.chemeris at gmail.com  Thu Mar  9 19:20:11 2017
From: alexander.chemeris at gmail.com (Alexander Chemeris)
Date: Thu, 9 Mar 2017 22:20:11 +0300
Subject: Thoughts on a next generation MIB/VTY replacement
In-Reply-To: <20170309010823.pxefxd5ik2vre76p@nataraja>
References: <20170309010823.pxefxd5ik2vre76p@nataraja>
Message-ID: <CABmJbFW75nYrxzxuHSZEi+SesmJgZKki7+5PLuFkdmvSB4jA7Q@mail.gmail.com>

Hi Harald,

A very interesting topic indeed.

On Mar 9, 2017 4:15 AM, "Harald Welte" <laforge at gnumonks.org> wrote:

So what I had in mind for quite some time (actually since netconf 1.2
about one year ago), is to have some kind of an external "VTY/MIB
daemon" (or even separate daemons for each) which maintains a
hierarchical database of configuration values.  The MIB deamon simply
offers an API (via client library) to GET or SET the individual values,
or to NOTIFY an application about a changed value.  This API is both
used by the actual Osmocom programs to obtain their configuration (and
obtain updates to it during runtime), as well as by the "VTY daemon"
providing interactive shell access to it.  Finally, other external
applications could use the same interface/client library to do the same.
A proxy to SNMP or REST interfaces can be imagined, for even more
interfacing with the outside world.


Right now VTY and control interface serve three functions:
1. Configuration ("configure terminal")
2. Object manipulation (eg "subscriber IMSI .. active ...")
3. Information query with polling and traps (eg "show ..." and control
interface)

A centralized configuration DB massively works with use case (1), but
doesn't really fit use cases (2) and (3) IMHO. You can do it, but then
it'll serve just as a proxy to the actual application and may become a
quite bloated and complicated piece of code.

Running another instance of yet another daemon is also something I
personally would really like to avoid looking from a user/admin
perspective. Number of processes we're already running on a NITB BTS is
already getting out of hand :(

So stealing an idea from FreeSWITCH and building on top of it:

Why don't we keep the configuration DB inside each app, but expose a
machine readable API only? Then VTY becomes just one of many applications
talking to this API on par with web-ui, scripts and whatnot.

This API could be something like JSON or something more binary (Erlang
terms binary format, BERT, etc). We can even make it more REST-like with
only GET/SET/NOTIFY or more RPC like with a richer set of primitives. There
are pros and cons in all these approaches, but the key point is that:
1. This works with all 3 use cases above
2. It's less intrusive into the current code base (I think)
3. Allows to have a single CLI/VTY app talking to any Osmocom app. Btw, the
app can know default ports of standard apps and allow connection by neg let
name like:
  $ osmo-cli -n osmo-bts
4. The app can still do TAB completion and help in a VTY way - FreeSWITCH
handles this perfectly with its fs_cli.

What do you think? It's a completely different approach than a config DB,
so I wanted to through this idea in without too much details to see if it
sticks.

Please excuse typos. Written with a touchscreen keyboard.

--
Regards,
Alexander Chemeris
CTO/Founder Fairwaves, Inc.
https://fairwaves.co
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170309/5f238118/attachment.htm>

From laforge at gnumonks.org  Thu Mar  9 20:11:02 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 9 Mar 2017 21:11:02 +0100
Subject: Thoughts on a next generation MIB/VTY replacement
In-Reply-To: <CABmJbFW75nYrxzxuHSZEi+SesmJgZKki7+5PLuFkdmvSB4jA7Q@mail.gmail.com>
References: <20170309010823.pxefxd5ik2vre76p@nataraja>
 <CABmJbFW75nYrxzxuHSZEi+SesmJgZKki7+5PLuFkdmvSB4jA7Q@mail.gmail.com>
Message-ID: <20170309201102.ahpoingpvemot4tq@nataraja>

On Thu, Mar 09, 2017 at 10:20:11PM +0300, Alexander Chemeris wrote:

> Right now VTY and control interface serve three functions:
> 1. Configuration ("configure terminal")
> 2. Object manipulation (eg "subscriber IMSI .. active ...")
> 3. Information query with polling and traps (eg "show ..." and control
> interface)

> A centralized configuration DB massively works with use case (1), but
> doesn't really fit use cases (2) and (3) IMHO. You can do it, but then
> it'll serve just as a proxy to the actual application and may become a
> quite bloated and complicated piece of code.

I have some vague ideas here but need to think further about it before I
think it's ready for an actual proposal.

> Running another instance of yet another daemon is also something I
> personally would really like to avoid looking from a user/admin
> perspective. Number of processes we're already running on a NITB BTS is
> already getting out of hand :(

I don't see the problem.  On modern systems it is simply a matter of a
systemd job configuring that daemon as a dependency for the other
daemons.  And if you prefer some other system like upstart, it is
basically the same.  I don't think any user cares about the number of
processes, at least not until you're talking about hundreds of them.

> Why don't we keep the configuration DB inside each app, but expose a
> machine readable API only? Then VTY becomes just one of many applications
> talking to this API on par with web-ui, scripts and whatnot.

It's another idea, indeed.  I guess to a large extent the same work is
required for both my or your proposal: You need some data structures
that describe the syntax of the configuration values, their permitted
ranges, types, help texts, etc.  The difference is then just where this
configuration is stored / "owned".  I like the idea to have it external,
as some paramters can then easily be shared between processes.  Let's
say the 'gsm_network' configuration, i.e. things like MCC, MNC.  You
curently have to enter that on both the SGSN and the MSC, but this could
be stored just as one 'network' config object that multiple processes
refer to.

> What do you think? It's a completely different approach than a config DB,
> so I wanted to through this idea in without too much details to see if it
> sticks.

Well, see above, I think it's more or less the same code, but running
inside the same process vs. running it externaly?

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From laforge at gnumonks.org  Thu Mar  9 20:18:39 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 9 Mar 2017 21:18:39 +0100
Subject: RFC: jenkins pipeline (containers and license compliance)
In-Reply-To: <CABmJbFViWm8wu474RNP-O0WEkO_TFuCqbCg_vU-WjmuKYyi6VA@mail.gmail.com>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
 <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
 <20170308164826.GD31335@my.box>
 <20170308172411.nchtvqshpybyrc5u@nataraja>
 <CABmJbFViWm8wu474RNP-O0WEkO_TFuCqbCg_vU-WjmuKYyi6VA@mail.gmail.com>
Message-ID: <20170309201839.bon3fyyacvixdpwb@nataraja>

On Thu, Mar 09, 2017 at 10:58:08AM +0300, Alexander Chemeris wrote:
> Could you elaborate what kind of license infringement does Docker have and
> what license issues might arise from publishing eg Osmocom images at some
> repository?

The issue is not with docker itself.  The issue is that people are
likely distributing large sets of pre-compiled binaries, which typically
has all kinds of obligations under copyleft licenses.

Distributing source is always easy.  As soon as you distribute binaries,
you need to either include the complete and corresponding source code
(under GPL/LGPL/AGPL), or you need to provide a written offer as to how
the exact corresponding source code for that particular software can be
obtained.

So doing something like a "Ubuntu Live derivative" or something like a
VM image, container or other image means you have to provude the *exact*
corresponding source code, up to three years later, for that given
image.  And if you update the image once per month, you have to keep a
record of all those source bases.

Passing on a written offer (like saying: Go to Ubuntu/Debian/... and
download the source there) is permitted in non-commercial distribution,
and relies on the fact that nobody else will distribute such an image in
acommercial context, ...  Also, can you guarantee that this third party
(UBuntu, Debian, whoever) will have those exact source versions around
for yeasr into the future?  What if not?

> I was confident it's ok and we did publish some code to create Docker
> containers in the past. I would like to understand potential issues with
> this.

Code to create docker images (or VM images) is perfectly fine.  That's
not distributing actual binaries of programs.

> Eg what's wrong with images built from publicly available sources without
> any binary blobs? 

see above, the fact that source is available (at the time of build?)
somewhere else is insufficient for compliance with LGPL, GPL and AGPL,
at least as soon as you ever distribute such an image in a commercial
context.

> And are the issues with Docker itself?

not that I'm aware of.  I just have some preconception about people who
work a lot with containers without having properly solved their copyleft
license compliance first.  And it might be that there now are solutions
for this - I just happen to hear horrors about public websites /
repositories full of binary images without any of them providing the
complete and corresponding source code to all the programs they have
packaged...

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From laforge at gnumonks.org  Thu Mar  9 21:14:48 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 9 Mar 2017 22:14:48 +0100
Subject: Thoughts on a next generation MIB/VTY replacement
In-Reply-To: <20170309134137.GA2194@my.box>
References: <20170309010823.pxefxd5ik2vre76p@nataraja>
 <20170309134137.GA2194@my.box>
Message-ID: <20170309211448.6rnmkq7ovb3csoxn@nataraja>

Hi Neels,

On Thu, Mar 09, 2017 at 02:41:37PM +0100, Neels Hofmeyr wrote:

> I once used a script that connects to nitb's vty, prints the lchan summary and
> exits. I ran that once a second. It often managed to crash the NITB! ... I
> never actually investigated it, but would have been good to do so.

this should not happen, and I don't see a reason why it would.  There's
no concurrency and no race conditions, everything runs synchronously in
one thread.  I've not observed such behavior so far (luckily).  but it
should be a relatively trivial scriptable test case: "vty load
generation" ;)

> So would the central interface simply remember the port numbers for us? No,
> you're talking about a "config server" that the individual applications connect
> to. On the one hand nice, on the other yet another osmo-entity. Still, +1.

I don't really see where the problem lies with 'yet another process'.
People have been using zebra+quagga for some 20 years by now, and they
always came as a suite of processes.  And that was before systemd took
care of re-spawning them ;)

I mean seriously, on my laptop I have 195 kernel threads cluttering my
'ps' output already, why would I care about a hand ful of osmocom
processes?

> Would osmo-programs require the config server to run?

yes.

> Would they be able to launch the config daemon automatically -- portably?

I would probably not bother but simply print a meaningful error message
and assume that people get their systemd rules right in terms of the
dependencies?

> > GET SET NOTIFY
> > ENUM, integers with ranges, etc.
> > machine-parseable description of a "MIB/config item", together with
> > syntax, ranges, enum values, help texts, etc (ideally in the C source
> > code, possibly in comments?)
> 
> Add to the wishlist: an indicator whether changing the value immediately
> changes the application's behavior. 

We could, but I would rather use such a massive change to do it
properly, i.e. implement the 'immediate reaction' to value changes in
the respective programs.

> A boolean unfortunately doesn't cut it,
> e.g. some settings take effect when a specific BTS' OML link is restarted.

Yes, those are bugs / shortcomings of the code and should be
adressed by people caring sufficiently about it?  We can re-generate the
BCCH/SACCH filling at any time and update them in the BTS.  We can also
re-configure the OML Managed Objects at runtime.  The protoocl permits
all of that.

> With global structs like gsm_network, I could imagine a completely automagic
> code generation from API comments. The description could contain the global
> gsm_network instance's symbol name, and the machine built code could provide
> the NITB's side as well as the config server's side, in the NITB storing those
> values directly in the global gsm_network instance. Or maybe let the automagic
> code create a replacement for the config part of the gsm_network struct. That's
> all easy for singleton items.

Yes, I would go for separate config structs, and hand a newly-allocated
one to the application, so the application can check for differences
between old and new config, if neded, before applying the new config.

> More complex would be things like fetching a subscriber with a given IMSI first
> and applying changes there. Arbitrary complexity of actions to update a value
> are thinkable, so we probably won't get around writing it manually? Semi?

That's not configuration data, is it?  I would not try to do subscriber
management within the MIB.

> But hold on, there are different cases: we don't want to store individual IMSIs
> in the config server (that's what the HLR is for), while we do want to store
> individual BTS configurations centrally.

exactly.

> For GET/SET/NOTIFY of a BTS element, we would need the API to have an array
> index argument, analogous to 'bts 0', 'bts 1'. For 'trx 0' etc we even need
> nested array indexes. With a path scheme? GET('bts[2].trx[0].arfcn').

yes.  That's one part that OpenWRT's uci does ver nicely, but it has
weak types (or none at all?) and is unsuitable in many other ways.
Also, I think uci has limited path depth, which isn't nice either.

> But for operations on a subscriber, we would instead merely send instructions
> to the HLR to do/store/say things -- if we want to allow managing subscribers
> from the central "vty" application, and if we want to allow querying the
> current state of entities (like 'show lchan summary'), we need another class of
> items that don't GET/SET things from/to the central config server, but tell an
> application to do or say something. Call them COMMAND, and they would feature
> arbitrary arguments. In the VTY we just implemented whatever we want to
> happen, in this scheme we should explicitly differentiate config items from
> commands.

yes. There are many details to be resolved, though.  We could simply
require each application to provide pretty-print functions for all of
its 'introspectable structs's and then build a library/plugin that the
"VTY application" can load to print even the "raw" binary structs.  We'd
just have to prefix them with some kind of type annotation (in the
protocol, not at runtime in all processes).  This way even the printf()
effort is kept out of the respective applications, further reducing load
that adminstrative interfaces can put on them.

> When we have separated BSC from NITB for good, centrally issued COMMANDs could
> also make up for the removed information match-up: in the NITB we always knew
> which BTS a subscriber was calling from with all of the detailed BTS state.
> With a separate BSC we don't anymore -- but the central "vty" could run queries
> with MSC and BSC to match up that information conveniently again.
> 
possibly, but that sounds rather more comples.

> > All of this is brainstorming and vaporware
> 
> It would be nice to dabble up some hacky code to discuss about, to sharpen our
> teeth on; without spending too much time on it and having in mind to discard it
> for something better anytime?

Yes, but not now [TM].  I at least think I need to spend lots of more
time thinking about this (in the bathtub or elsewhere).

> Sounds a bit like re-inventing dbus. (I don't know much about dbus though,
> would it make sense to use dbus?? Is it a giant kraken of dependencies?)

I don't think dbus is a configuration management system, but an IPC
system.  And yes, we'd need some kind of IPC, but I would ignore/defer
the exact IPC method/transport for now, as it is not the key "idea"
under discuission.

> Seems to me that with all the wishlist features we have collected now, the
> implementation of such a system would actually be quite complex and not a small
> effort.

yes. The question is if we can find something that is possible to do in
stages/milestones, rather than changing everything at once.  Or some
elegant hack that provide at least some of the features we need at
limited effort.

> I think the most important feature is that all our config items become a
> machine and human API at the same time -- and that is a killer feature for all
> those folks asking for web interfaces.

exactly. Whether we do this still inside the processes or in an external
central process is a minor detail.

Also, what I'd really like is to see the generalized read/parse and save
code, avoiding all the possible manual mistakes we have at places.
Finally, being able to do away with the "vty-additions.xml" files but
having more documentation at the same place where the config items are
defined/declared is better and will make it feasible to have more
documentation in the vty reference manuals.

What I find very surprising is that nobody seems to have come up with
something like this already.  But what can be found is miles far away
from what we're thinking of.  There's libconfuse, uci, libcli, clish,
... - all of them seem to have some interesting ideas, but fall quite
short of a proper system.  And then there's things like SNMP and
net-snmp which are ugly and complex.

The IETF has come up with OpenConfig (and NETCONF and RESTCONF
protocols) which goes very much into the direction I've been thinking
of.  But it seems they've mostly been working on architecture as well as
OpenConfig models for regular IP-type network element configuration,
(writtein in YANG, a language they invented for that purpose) and
protocols, but I haven't seen any FOSS implemenataions yet?
http://jedelman.com/home/openconfig-data-models-and-apis/ is a good
intro into this part of the networking universe.  Fully buzzword
compatible ;)  And again very heavy and complex.  If there were many
implementations out there, it might be worth investigating - but there
aren't.

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Fri Mar 10 03:37:39 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Fri, 10 Mar 2017 04:37:39 +0100
Subject: about reusing subscriber connections
Message-ID: <20170310033739.GB2194@my.box>

Testing with the nano3G, e.g. when sending a large SMS, I see that multiple CM
Service Requests may come in on the same IuCS connection. The code so far
assumed that a CM Service Request would always come in on a fresh,
unauthenticated connection (after iuRelease): the ProcessAccessReQuest FSM
always starts out with establishing identities, authentication, ciphering.

Now I've added code to accept a CM Service Request on an already accepted
connection, by sending back an immediate CM Service Accept without even
invoking the PARQ FSM. But that might not be such a good idea:

I face the same for a Location Updating. A phone may choose to send a periodic
LU on an already open and accepted connection (I guess). So we should allow
that by sending back a LU Accept right away, without another authentication
dance.  But we also want to tell the HLR about the LU. The VLR FSMs may also
have other actions in mind that the MSC doesn't know or care about. So:

To not re-invent the wheel outside of libvlr, I would go on and add a flag to
the LU and PARQ FSMs to indicate that the request is coming in on an already
accepted connection, to jump directly to the post-ciphering FSM state.

But I'd like to hear whether you guys have some thoughts or experience on this.


For the case of the large multi-part SMS, we usually are fast enough in
releasing the conn after each SMS fragment. For the next fragment, another
initialUE message with a CM Service Request starts the PARQ FSM, auth+ciph is
re-established, and the next part goes through. But sometimes the UE has
already started to send another CM Service Request to initiate the next SMS
fragment before it acked the previous one -- meaning that before we get as far
as being done and issuing an iuRelease from the CN, the UE already sends
another fresh CM Service Request on the same conn. That means we don't have to
re-establish auth, quite nice actually. To allow cascades like these, we might
also decide to iuRelease slightly later, giving the UE another second to re-use
the open connection, so that we have less message overhead and use less auth
tuples -- I wouldn't actually give it much more than a second; after about five
seconds, the UE usually does an iuRelease itself.

That sounds sane, but actually https://osmocom.org/issues/1816 holds against
that: when I send USSD requests *from the phone* rapidly without leaving time
to iuRelease, only the first USSD goes through from the UE to the CN. The
second one is never received on the IuCS wire. That would indicate that the
phone expects a conn to close right away from the CN, always. When I wait for
iuRelease before sending another USSD, the MO USSD gets its own conn
established and all is well. This behaves identically for both the femto cell
kinds I have for testing. But, today, seeing multiple CM Service Requests on
the same conn for SMS makes me think that #1816 might be a bug in the USSD
dispatch of my phone. If MO SMS can do CM Service Requests on an already open
conn, why should MO USSD not be able to do the same?

I think in the old, pre-VLR code, we would accept multiple CM Service Requests
on the same conn without problems, because the semantics were "re-entrant",
calling secure_connection() in multiple places, which was skipped when some
flag that we're already authenticated was set. The VLR paradigm is different:
it "locks down" a new conn until that is accepted (authenticated and ciphered,
according to config). Once a conn is accepted, the rest of the code does not
need to bother with asking for securing a connection. So far receiving a CM
Service Request is semantically identical with starting a new conn. That's why
I would explicitly add a parameter to FSM invocation to skip over auth+ciph in
case the conn is already accepted, but still do any HLR actions as needed.

What still bugs me is: when the phone knows that a conn is open, why should it
bother with a CM Service Request, it could just start sending the real meat
right away.


Re-using from the CN side:

In https://osmocom.org/issues/1712#note-7 I noted that the code should not page
when a connection is already open.
https://osmocom.org/issues/1862 asks for not attempting to send USSD from the
CN to the UE when a conn is there but not authenticated/ciphered yet.
When the conn is accepted, we can immediately send the MT messages -- however,
if the conn is there but still busy authenticating, we must neither page nor
can we send messages right away. We should then add an item to the queue of
actions to carry out after a paging response, without dispatching a paging
request; and ensure that the queue is also worked off in case we receive a CM
Service Request or LU Request instead of a Paging Response ... actually, this
might already work, but I should test for it.

P.S.: Also wondering, could it happen that we receive a Paging Response on an
already accepted connection? Wouldn't make much sense, but might be possible to
provoke ;)


Any facts/thoughts on this stuff are welcome.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170310/250f0202/attachment.bin>

From msuraev at sysmocom.de  Fri Mar 10 09:11:14 2017
From: msuraev at sysmocom.de (Max)
Date: Fri, 10 Mar 2017 10:11:14 +0100
Subject: Thoughts on a next generation MIB/VTY replacement
In-Reply-To: <20170309010823.pxefxd5ik2vre76p@nataraja>
References: <20170309010823.pxefxd5ik2vre76p@nataraja>
Message-ID: <67b970e5-b66a-40b6-abb1-8847a6021d3e@sysmocom.de>

Hi.

I won't claim in-depth experience with MIB frameworks, just want to 
contribute few buzzwords into brainstorming:
NETCONF - rfc 6241 https://tools.ietf.org/html/rfc6241
YANG - rfc rfc 6020 https://tools.ietf.org/html/rfc6020

The former is IETF protocol for configuring network devices [1]. The 
latter is data modeling language describing device's configuration and 
state, manipulated by netconf [2].

 From brief overview those seems to match well with Harald's outline. 
There might be few pieces still missing but it seems doable. I see 
following advantages in adopting netconf & yang:
- it's ietf standard
- there are open source (including GPL) implementations we can play with 
[3, 4]
- it's already widely supported by telecom vendors so chances are in 
operators using Osmocom stack, will be some people familiar with it
- it's SDN/NFV compatible which makes it future-proof
- there's proposed extension RESTCONF which make integration with creepy 
web things like SOAP much easier [5]
- and most important for me personally - there's Emacs mode for it [6] :-)

We're not the first to feel the need for proper MIB/configuration 
subsystem, and the task indeed looks huge. So the more we can rely on 
work already done in this field, the better. Even if we have to write 
some things, by relying on open and widely adopted standards we increase 
our chances of getting external contributions which is always nice.

[1] https://trac.ietf.org/trac/netconf
[2] http://www.yang-central.org/twiki/bin/view/Main/WebHome
[3] http://ensuite.sourceforge.net/
[4] https://github.com/CESNET/libnetconf
[5] https://tools.ietf.org/html/draft-bierman-netconf-restconf-04
[6] https://www.emacswiki.org/emacs/yang-mode.el

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From sebastian.stumpf87 at googlemail.com  Fri Mar 10 13:36:09 2017
From: sebastian.stumpf87 at googlemail.com (Sebastian Stumpf)
Date: Fri, 10 Mar 2017 14:36:09 +0100
Subject: SMS not received in mobile - Virtual layer1 (osmo-nitb + osmo-bts +
 osmocom-bb/mobile)
Message-ID: <CALFVOecysW2gsyHN+r2bg-UXRz5ubRRk9eO2ahx=SwW5Wg=gEg@mail.gmail.com>

Hi,

Some of my sent SMS are not received by my mobile and I try to figure out
if this is a problem with my virtual layer 1, configuration or
something in the layers 2+.

I use:
osmo-nitb: 1 subscriber in hlr (id 1, ext 017519191919)
osmo-bts: virt-phy, arfcn 666, TS0=CCCH+SDCCH4, TS1=SDCCH8
MS layer23 app mobile: is the subscriber from osmo-nitb
MS layer1 virt-phy: virtual gsmtap layer between bts and mobile

I want:
Send a sms to subscriber 017519191919.
osmo-nitb VTY:
 OpenBSC# subscriber id 1 sms sender id 1 send Test

I have 2 outcomes, first one is the failure. See
https://www.dropbox.com/s/eunpfioq518t09a/mobile--ms-virt--bts-virt--bsc-nitb_170310%3Afailed_sms_rec_lapdm.pcapng?dl=0
for the pcap. Do not wonder about the duplicated gsmtap messages from mobile,
the ones to ip 225.0.0.1 are over virt-phy layer, the ones to localhost from
the gsmtap logging option from mobile. Filter them out via "gsmtap &&
(ip.dst != 127.0.0.1)".

602-613: RR channel establishment with RA and IA as reaction to paging in 590
627,631: SABM from BTS to establish async balanced mode on SAPI3 (SMS)
and the ACK from MS
656,684,699: I-Data frames on SAPI3 containing the sms
660,688,703: ReceiveReady's from MS (That ACK all I-frames before
N(R)-1 as far as I understood)

For me, that looks good and I really do not understand why the MS will
not answer with the CP-ACK/RP-ACK
after receiving the SMS Data in 699.
The LAPDM link will stay up for some more time and be used by the
network for retransmission of the sms (1149),
but MS will never react to it on the SMS layer (CP-ACK, RP-ACK).

Here is the console log for the mobile
(https://www.dropbox.com/s/6gug5cd4up5qv7y/mobile--ms-virt--bts-virt--bsc-nitb_170310%3Afailed_sms_rec_lapdm?dl=0).
The paging is answered in line 378:
 Fri Mar 10 10:42:06 2017 DLLAPD <0012> lapd_core.c:793 SABM(E)
received in state LAPD_STATE_IDLE

Second outcome is a successfully sent sms from network to ms. See
https://www.dropbox.com/s/ghdn7pc0j4vifbe/mobile--ms-virt--bts-virt--bsc-nitb_170310%3Asuc_sms_rec_lapdm.pcapng?dl=0
for the pcap. What is different here? There is no paging, but the sms
is sent within the dedicated
channel used for the location update initiated by the phone started
up. Why there is another outcome, I don't know.

169-181: RR channel establishment with RA and IA as part of mibiles
startup routine to make location update
188,416: Location udpdate procedure on SAPI0
230,234: SABM from BTS to establish async balanced mode on SAPI3 (SMS)
and the ACK from MS
396,444: I-Data frames on SAPI3 containing the sms
400,449: RR's from MS
450: CP-ACK for BTS
482,499: RP-ACK I-Frames for Network (osmo-nitb)

Again, the console log of the mobile
(https://www.dropbox.com/s/cujsn40d43g6wxk/mobile--ms-virt--bts-virt--bsc-nitb_170310%3Asuc_sms_rec_lapdm?dl=0).
The SABM on SAPI3 on downlink is received in line 206:
 Fri Mar 10 11:06:13 2017 DLLAPD <0012> lapd_core.c:793 SABM(E)
received in state LAPD_STATE_IDLE

The signaling looks quite similar in both cases, but one time the
mobiles sms handler responds to CP-DATA msg containing the SMS, the
other time not.
Can someone find an error in the signaling I did not see?
Is there a known Bug? To be honest, I last merged my branch with
master some time ago (~ Jan 18 2017) and thus did not update osmo-nitb
and
the libraries since then to avoid compatibility problems. Have there
been recent changes that could cause that behavior?

Thank you and Kind Regards,
Sebastian


From holger at freyther.de  Fri Mar 10 14:19:28 2017
From: holger at freyther.de (Holger Freyther)
Date: Fri, 10 Mar 2017 15:19:28 +0100
Subject: MNCC ABI changes 
Message-ID: <2EFB6320-67DB-4CEF-B99C-AE9B538B23E6@freyther.de>

Hi,

as part of looking into early media (and at some point handover handling) I would like to use MNCC_RTP_CONNECT and supply "recvonly", "sendonly" attributes. The gsm_mncc_rtp structure does not carry this information though.

struct gsm_mncc_rtp {
        uint32_t        msg_type;
        uint32_t        callref;
        uint32_t        ip;
        uint16_t        port;
        uint32_t        payload_type;
        uint32_t        payload_msg_type;
}; 

The content of payload_msg_type currently holds anything between 0x300 and 0x03ff and I would like to split this field into two uint16_t. The reason is not to change the size of the structure (and maybe not dump the protocol version). It would be a backward compatible change.

Is anyone open for such tricks? Otherwise I will bump protocol version and change the ABI (and try to update the various active users of this protocol).


holger

From laforge at gnumonks.org  Fri Mar 10 15:40:10 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Fri, 10 Mar 2017 16:40:10 +0100
Subject: MNCC ABI changes
In-Reply-To: <2EFB6320-67DB-4CEF-B99C-AE9B538B23E6@freyther.de>
References: <2EFB6320-67DB-4CEF-B99C-AE9B538B23E6@freyther.de>
Message-ID: <20170310154010.xp6yvp6ptkymxoer@nataraja>

Hi Holger,

On Fri, Mar 10, 2017 at 03:19:28PM +0100, Holger Freyther wrote:
> as part of looking into early media (and at some point handover
> handling) I would like to use MNCC_RTP_CONNECT and supply "recvonly",
> "sendonly" attributes. 

It would make sense to have a bitmask i.e. have one bit for rx and one
for tx, as opposed to an 'arbitrary' enum.

> The content of payload_msg_type currently holds anything between 0x300
> and 0x03ff and I would like to split this field into two uint16_t. The
> reason is not to change the size of the structure (and maybe not dump
> the protocol version). It would be a backward compatible change.
>
> Is anyone open for such tricks? Otherwise I will bump protocol version
> and change the ABI (and try to update the various active users of this
> protocol).

I don't mind.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From alexander.huemer at xx.vu  Fri Mar 10 23:24:36 2017
From: alexander.huemer at xx.vu (Alexander Huemer)
Date: Sat, 11 Mar 2017 00:24:36 +0100
Subject: Unknown HFC multiport controller
Message-ID: <20170310232436.GA612@yade.chaostreff.at>

Hi,

I have a Siemens BS-11 and a HFC IOB1E1 PCI card.
The linux kernel is not very happy with the card, I get

        Unknown HFC multiport controller (vendor:1397 device:30b1
        subvendor:1397 subdevice:b543)
        Please contact the driver maintainer for support.

The same error is mentioned in [1] but with different subdevice id, 
which I believe does not matter.
I am not sure why vendor ID 1397 (PCI_VENDOR_ID_CCD) and device ID 30b1 
(PCI_DEVICE_ID_CCD_HFCE1) are not valid. Consulting [2] I was under the 
impression that this is the expectation of the driver.
[1] does not provide an answer as far as I can see.
Would anybody know what the problem is here?

-Alex

[1] http://lists.osmocom.org/pipermail/openbsc/2009-August/001838.html
[2] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/isdn/hardware/mISDN/hfcmulti.c#n5445
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170311/7d9b6a53/attachment.bin>

From alexander.huemer at xx.vu  Fri Mar 10 23:42:32 2017
From: alexander.huemer at xx.vu (Alexander Huemer)
Date: Sat, 11 Mar 2017 00:42:32 +0100
Subject: BS-11 Serial Interface
Message-ID: <20170310234232.GB612@yade.chaostreff.at>

Hi,

I have a Siemens BS-11 with the factory cable for the LMT that has a 
DSUB-9 plug on the non-BTS end.
When I connect a USB<-->RS232 adapter and run either $ cu -l /dev/ttyUSB 
or bs11_config -p /dev/ttyUSB0 I do not get any output when I switch on 
the BTS.  I tried with nullmodem adapter and without just to be sure 
since I often mix up the pinouts.
I was not able to measure any voltage on any of the DSUB-9 pins when the 
BTS is powered up, but believe that on either RX or TX there should be 
the voltage level of the serial line since RS232 is active low.
Has somebody seen this behavior?

-Alex
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170311/d5b4c914/attachment.bin>

From axilirator at gmail.com  Sat Mar 11 03:43:34 2017
From: axilirator at gmail.com (Vadim Yanitskiy)
Date: Sat, 11 Mar 2017 10:43:34 +0700
Subject: libosmocore build failure cascade
Message-ID: <CAPEnp=YMNm+0ypHXzpn20eCW9q9bsdOwrKVqWs3igNrhc7HW9w@mail.gmail.com>

Hi Max,

> On a related note - are you working on moving OsmoBTS to libosmocoding?

Not a whole OsmoBTS :)
GSM 05.03 transcoding routines only.
Right now, I am testing dynamic TS switching on UmTRX.

> Are there some missing pieces before this can be done?

Almost done, now we need to switch OsmoBTS to use libosmocore's
implementation and remove the old code.

With best regards,
Vadim Yanitskiy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170311/86150c82/attachment.htm>

From spaar at mirider.augusta.de  Sat Mar 11 08:37:00 2017
From: spaar at mirider.augusta.de (Dieter Spaar)
Date: Sat, 11 Mar 2017 08:37:00 
Subject: BS-11 Serial Interface
Message-ID: <58c3b72c.mirider@mirider.augusta.de>

Hello Alexander,

On Sat, 11 Mar 2017 00:42:32 +0100, "Alexander Huemer" <alexander.huemer at xx.vu> wrote:
>
> I was not able to measure any voltage on any of the DSUB-9 pins when the=20
> BTS is powered up, but believe that on either RX or TX there should be=20
> the voltage level of the serial line since RS232 is active low.
> Has somebody seen this behavior?

Maybe there is something wrong with the cable? This is from my notes
about the BS-11 serial interface:

  The four E1 BNC connectors are here.

    5 6 7 8
    x x x x
    x x x x
    1 2 3 4

    2: LMT Tx (out)
    6: LMT Rx (in)
    4: +5V
    8: GND

There are two more 8-pin connectors below, you have to connect the cable
to the top one (below the E1 BNC connectors). The same with some pictures
is also here:

  https://osmocom.org/projects/openbsc/wiki/BS11LMT

Best regards,
  Dieter
-- 
Dieter Spaar, Germany                           spaar at mirider.augusta.de


From dr.blobb at gmail.com  Sat Mar 11 21:47:41 2017
From: dr.blobb at gmail.com (=?UTF-8?Q?Andr=C3=A9_Boddenberg?=)
Date: Sat, 11 Mar 2017 22:47:41 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <20170308164826.GD31335@my.box>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
 <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
 <20170308164826.GD31335@my.box>
Message-ID: <CALQ9Oh3qqNUry8e6WsBV4_saygyjObpSozuXmXXco0tf1+pONA@mail.gmail.com>

>> It was good meeting you in person, Andr? :)

It was a pleasure to meet you all and to speak with you in person
about jenkins.osmocom! I am looking forward to see you all on the
OsmoCon :)

>>How about each built library extends a docker image produced by a previous
>>dependency's last successful build? e.g. if libosmocore rebuilds, that updates
>>the libosmocore docker image, then libosmo-abis is triggered and adds itself,
>>producing a last-successfully-built libosmo-abis docker image, and so on, down
>>to openbsc re-using a docker image that already holds all its dependencies?

In general that's my thought too, but I am afraid of the complexity of
handling/maintaining all these build images.
Afaics right now only two docker images have to be (re)build
(osmobuild:amd64/32bit), when using docker images and its inheritance
model for dependency handling one would end up building a lot more.

I need to tinker around more to understand better.

> Just to be clear, I'm merely one person here, just because I like the sound of
> Job-DSL, it doesn't mean everyone else does. I think it would be good to see an
> example of an openbsc build using it and then let everyone decide.

No worry, right now I am just sharing my experience about the topic
"CI as Code" with you. Whether jobDSL will be used by the osmocom
projects is a decision taken by you, not me :)

That's what have been created so far:

osmo-seed [2]: holds inline jobDSL script for [3][4], normally this
jobs polls a repo holding all jobs and deploy them after a change has
been detected.

openBSC_jobDSL [3]: the openBSC build job as a script deployed by the
seed. Basically looks the same, the configuration can be still changed
in the web-ui. Manually changes will be marked [4].

Furthermore this wiki [5] is a good entrypoint to jobDSL. Afterwards,
these sites [6][7] are helpful imho for a first and second hands-on.


>>>> Why is this build dependency not part of the Docker image?
>>
>> That's a good question, it should be, right?

In general yes, but huge dependencies e.g. an Android SDK are often
mounted from the local file system and not backed into a docker image,
so there's not a clear general answer.

Can I may ask how you rebuild your docker images? I'd assume that an
image is rebuild after a patch submission to osmo-ci, which introduced
a change to its Dockerfile. At least according to the Dockerfile
attached in this mail thread in which the omso-ci repo is backed in.

>> I've attached the Dockerfile config used to generate the docker image for openbsc.

Thanks a lot for your support and the Dockerfile. I'm just wondering
why the osmo-ci repo [8] doesn't hold the latest state of the
Dockerfile is it used for builds?

>>> And it should probably use the osmo-ci git
>> repos instead of echoing the osmo-deps.sh to /usr/local/bin manually. Actually,
>> our openbsc-gerrit build job has osmo-ci in ~/bin and links that to /build_bin
>> in the docker build, also adding /build_bin to the PATH ... that's a bit
>> cumbersome and is my fault from when I wasn't too familiar with docker yet.
>> These things could be streamlined.

hehe, I recognized that osmo-ci must live in your slave's home
directory. I mounted each script to /build_bin/$script to work around
it.

>> OT: does your nick "Blobb" indicate affinity to Binary Large Ob(b)jects?... like docker images? ;)

Hehe, this assumption is made by a lot of coders/techies, but it
origins from the game blobby volley [1]. It may sounds weird, but I am
called blobb(y), because I am looking similar to the characters in the
game. Maybe it's time to cut another letter at the end? ;)


blobb

[1] https://sourceforge.net/projects/blobby/
[2] https://jenkins.blobb.me/view/osmocom/job/omso-seed/       (add
"configure-readonly/" to read config)
[3] https://jenkins.blobb.me/view/osmocom/job/openBSC_jobDSL/
[4] https://jenkins.blobb.me/view/osmocom/job/openBSC_jobDSL_manually_changed/
[5] https://github.com/jenkinsci/job-dsl-plugin/wiki
[6] https://github.com/sheehan/job-dsl-gradle-example
[7] http://job-dsl.herokuapp.com/
[8] http://git.osmocom.org/osmo-ci/tree/docker/Dockerfile.deb8_amd64


2017-03-08 17:48 GMT+01:00 Neels Hofmeyr <nhofmeyr at sysmocom.de>:
> FYI, Blobb visited us in the sysmocom office yesterday, and we had a personal
> conversation on the jenkins build setup. It was good meeting you in person,
> Andr? :)  For the sake of this ML, let me answer briefly here as well.
>
> On Mon, Mar 06, 2017 at 04:15:20PM +0100, Andr? Boddenberg wrote:
>> When we speak about the "easier docker integration", we mean that
>> everyone would be happy if not every matrix-configuration axis builds
>> all deps like libosmocore, libosmo-netif etc pp, right?
>
> We're rebuilding everything simply because no-one has found time to make this
> more efficient yet. There is no need to rebuild all of libosmocore through to
> to libsmpp34 for every matrix cell.
>
> It would also be useful to build all libosmo* only once for each update of the
> master branch, and in the dependent builds (openbsc, openbsc-gerrit, ...)
> always re-use the last successfully built binaries of the dependencies.
>
>> To address this issue I'd like to create a local temporary docker
>> image which extends osmocom:amd64  and holds all deps. This image can
>> then be used for all openBSC builds and the temporary local docker
>> image can be removed as a "post-build"-step (which should get
>> triggered regardless the build result).
>
> How about each built library extends a docker image produced by a previous
> dependency's last successful build? e.g. if libosmocore rebuilds, that updates
> the libosmocore docker image, then libosmo-abis is triggered and adds itself,
> producing a last-successfully-built libosmo-abis docker image, and so on, down
> to openbsc re-using a docker image that already holds all its dependencies?
>
> Even though our libraries don't necessarily have a linear dependency, it could
> make sense to artificially define such a linear line of dependencies (e.g. even
> though libsmpp doesn't need libosmo-abis, we make libsmpp build on top of the
> libosmo-abis docker to collect it into the dependecies docker image).  But,
> then again, if libosmocore succeeded and libosmo-abis failed, we would omit a
> libsmpp build for no reason. So maybe it would also make sense to somehow build
> the non-dependent libraries independently and later combine into a joint docker
> image?? ...I would leave that up to your choices, just brain storming...
>
>> A slightly different topic, did you think about pushing your docker
>> images to hub.docker.com [2]?
>>
>> In my opinion this will be a step forward to a transparent and local
>> reproducible build environment. Afair Harald was quite clear about
>> this requirement?
>
> No idea / not aware of it / no opinion so far :)
> I was once invited to design a logo for the reproducible builds project, but
> haven't found time for that (yet?). That's about all I know about the topic...
>
>> >> and "Wiki" is an ancient African word for "hopelessly outdated"...
>> Thanks for pointing this out, usual I am hesitating to correct
>> something to avoid being called nit-picky.
>
> My middle name is "Nit-pick" :P
> It's sometimes really hard for me to ignore a small detail that is obviously
> wrong for the benefit of moving ahead faster...
>
> In the Wiki's case, there is no drawback of correcting mistakes -- no noise in
> code review or mailing lists is generated, so if you have the time, just do it,
> as much as you like.
>
>> >> >> [2] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
>> >> This sounds like we want to use it!
>>
>> Alright, so I will work on the following migrations:
>>      - "as is" -> "JobDSL"
>>      - "as is" -> "Pipeline" (probably just for Max)
>
> hehe
>
> Just to be clear, I'm merely one person here, just because I like the sound of
> Job-DSL, it doesn't mean everyone else does. I think it would be good to see an
> example of an openbsc build using it and then let everyone decide.
>
>> >> so far that doesn't look like an improvement of
>> >> #!/bin/sh
>> >> ./contrib/jenkins.sh
>>
>> Agreed, the biggest advantages of Pipeline are:
>>
>> -  the "eye-candy" (but who cares? :)
>>
>> - easy artifacts sharing across different steps, which run on
>> different nodes (but afaics you don't even use the "Copy Artifacts
>> Plugin", which makes this argument pointless).
>
> But if we introduce docker images as artifacts, this could become useful?
>
>> - the duration/sub-steps can be easily seen (as mentioned by Max), but
>> this can be achieved which some simple python scripts/plugins +
>> influxDB + grafana as well.
>
> I personally mostly care about the overall time a build takes, so that I get
> V+1/V-1 on my gerrit patches faster.
>
>> Neels, can you please help me fixing the following build error [3]:
>>
>> 01:59:20 checking dbi/dbd.h usability... no
>> 01:59:20 checking dbi/dbd.h presence... no
>> 01:59:20 checking for dbi/dbd.h... no
>> 01:59:20 configure: error: DBI library is not installed
>
> Install libdbi-dev and libdbd-sqlite3.
>
>> I already added the following package to the image, by:
>>
>> docker run ........ osmocom:amd64 /bin/bash -c sudo apt-get install -y
>> r-cran-dbi; /build/contrib/jenkins.sh
>
> r-cran-dbi?? what is that? :)
>
>> Why is this build dependency not part of the Docker image?
>
> That's a good question, it should be, right?
> According to the dockerfile, libdbi-dev is actually installed there.
> So is libdbd-sqlite3.
>
> Hard to say why configure tells you that that is missing. It should work.
>
> I've attached the Dockerfile config used to generate the docker image for
> openbsc. There's some stuff in there not needed for openbsc in particular, e.g.
> the smalltalk things near the end. And it should probably use the osmo-ci git
> repos instead of echoing the osmo-deps.sh to /usr/local/bin manually. Actually,
> our openbsc-gerrit build job has osmo-ci in ~/bin and links that to /build_bin
> in the docker build, also adding /build_bin to the PATH ... that's a bit
> cumbersome and is my fault from when I wasn't too familiar with docker yet.
> These things could be streamlined.
>
> docker run --rm=true -e HOME=/build -e MAKE=make -e PARALLEL_MAKE="$PARALLEL_MAKE" \
> -e IU="$IU" -e SMPP="$SMPP" -e MGCP="$MGCP" -e PATH="$PATH:/build_bin" \
> -e OSMOPY_DEBUG_TCP_SOCKETS="1" -w /build -i -u build -v "$PWD:/build" \
> -v "$HOME/bin:/build_bin" osmocom:amd64 /build/contrib/jenkins.sh
>
>
> OT: does your nick "Blobb" indicate affinity to Binary Large Ob(b)jects?
> ... like docker images? ;)
>
> ~N
>
> --
> - Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
> =======================================================================
> * sysmocom - systems for mobile communications GmbH
> * Alt-Moabit 93
> * 10559 Berlin, Germany
> * Sitz / Registered office: Berlin, HRB 134158 B
> * Gesch?ftsf?hrer / Managing Directors: Harald Welte



-- 
Gru?
Blobb


From nhofmeyr at sysmocom.de  Mon Mar 13 00:46:38 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 13 Mar 2017 01:46:38 +0100
Subject: RFC: jenkins pipeline
In-Reply-To: <CALQ9Oh3qqNUry8e6WsBV4_saygyjObpSozuXmXXco0tf1+pONA@mail.gmail.com>
References: <CALQ9Oh0ntFyoAp+sbp+6vfM+eV4GW10Rqb4B2NpTCv+t5EGALg@mail.gmail.com>
 <20170306135339.GB12874@my.box>
 <CALQ9Oh0+Puadz9bivSOXGu=30Gz4SPhxsUN_J-BJAvErXsxLAg@mail.gmail.com>
 <20170308164826.GD31335@my.box>
 <CALQ9Oh3qqNUry8e6WsBV4_saygyjObpSozuXmXXco0tf1+pONA@mail.gmail.com>
Message-ID: <20170313004638.GA25910@my.box>

On Sat, Mar 11, 2017 at 10:47:41PM +0100, Andr? Boddenberg wrote:
> Can I may ask how you rebuild your docker images?

Manually. Basically so far changes to the osmo-python-tests were the only
events that needed a rebuild of that image. So when I changed something on
osmo-python-tests, I login on the build server, trivially bump the dockerfile
and launch a rebuild. All other changes, libosmocore thru openbsc, are built
for each job, so all we need there is a basis for building.

osmo-ci is pulled in from the docker commandline.

Would probably be good to have the/a docker image in osmo-ci and use that.  One
could autogenerate the tail of the dockerfile to add 'git checkout's of the
current HEAD hashes and trigger a rebuild as an easy way to re-use.

> I'd assume that an image is rebuild after a patch submission to osmo-ci

I agree that this would be a good idea.

> Thanks a lot for your support and the Dockerfile. I'm just wondering
> why the osmo-ci repo [8] doesn't hold the latest state of the
> Dockerfile is it used for builds?

osmo-ci is pulled in from the docker commandline, 'mounted' at /build_bin.
I believe I explained that here:

> >> our openbsc-gerrit build job has osmo-ci in ~/bin and links that to /build_bin
> >> in the docker build, also adding /build_bin to the PATH ... that's a bit
> >> cumbersome and is my fault from when I wasn't too familiar with docker yet.

If there's another osmo-ci in the dockerfile, we're not using that.

> [2] https://jenkins.blobb.me/view/osmocom/job/omso-seed/       (add

lol 'omso'

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170313/2552353c/attachment.bin>

From nhofmeyr at sysmocom.de  Mon Mar 13 01:44:40 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 13 Mar 2017 02:44:40 +0100
Subject: FYI: Build failure of network:osmocom:nightly/osmo-pcu in
 xUbuntu_16.10/x86_64
In-Reply-To: <58c456bb33a45_6b03efc102634e5@build.opensuse.org>
References: <build-a48978cf87c1debc5b7c789594e2bf7c@build.opensuse.org>
 <58c456bb33a45_6b03efc102634e5@build.opensuse.org>
Message-ID: <20170313014440.GD25910@my.box>

fyi, we're still getting daily failures by the opensuse builds, which should be
fixed once https://gerrit.osmocom.org/2007 is merged.

~N

On Sat, Mar 11, 2017 at 07:57:26PM +0000, OBS Notification wrote:
> Visit https://build.opensuse.org/package/live_build_log/network:osmocom:nightly/osmo-pcu/xUbuntu_16.10/x86_64
> 
> Package network:osmocom:nightly/osmo-pcu failed to build in xUbuntu_16.10/x86_64
> 
> Check out the package for editing:
>   osc checkout network:osmocom:nightly osmo-pcu
> 
> Last lines of build log:
> [  111s]                  from bts.cpp:21:
> [  111s] /usr/include/c++/6/cstdlib:185:3: note: candidate: __int128 std::abs(__int128)
> [  111s]    abs(__GLIBCXX_TYPE_INT_N_0 __x) { return __x >= 0 ? __x : -__x; }
> [  111s]    ^~~
> [  111s] /usr/include/c++/6/cstdlib:180:3: note: candidate: long long int std::abs(long long int)
> [  111s]    abs(long long __x) { return __builtin_llabs (__x); }
> [  111s]    ^~~
> [  111s] /usr/include/c++/6/cstdlib:172:3: note: candidate: long int std::abs(long int)
> [  111s]    abs(long __i) { return __builtin_labs(__i); }
> [  111s]    ^~~
> [  111s] Makefile:778: recipe for target 'bts.lo' failed
> [  111s] make[2]: *** [bts.lo] Error 1
> [  111s] make[2]: Leaving directory '/usr/src/packages/BUILD/src'
> [  111s] Makefile:403: recipe for target 'all-recursive' failed
> [  111s] make[1]: *** [all-recursive] Error 1
> [  111s] make[1]: Leaving directory '/usr/src/packages/BUILD'
> [  111s] dh_auto_build: make -j1 returned exit code 2
> [  111s] debian/rules:12: recipe for target 'build' failed
> [  111s] make: *** [build] Error 2
> [  111s] dpkg-buildpackage: error: debian/rules build gave error exit status 2
> [  111s] 
> [  111s] lamb16 failed "build osmo-pcu_0.3.20170311.dsc" at Sat Mar 11 19:57:22 UTC 2017.
> [  111s] 
> [  111s] ### VM INTERACTION START ###
> [  115s] [  102.127065] reboot: Power down
> [  115s] ### VM INTERACTION END ###
> [  115s] 
> [  115s] lamb16 failed "build osmo-pcu_0.3.20170311.dsc" at Sat Mar 11 19:57:26 UTC 2017.
> [  115s] 
> 
> -- 
> Configure notifications at https://build.opensuse.org/user/notifications
> openSUSE Build Service (https://build.opensuse.org/)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170313/5455ce9c/attachment.bin>

From keith at rhizomatica.org  Mon Mar 13 22:01:57 2017
From: keith at rhizomatica.org (Keith)
Date: Mon, 13 Mar 2017 23:01:57 +0100
Subject: NITB: high cpu usage and "crossed" messages when SMS table grows
Message-ID: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>

Hi all,

I temporarily disabled a cron job we run at rhizomatica that purges the
hlr SMS table of sent messages every day.

After a few days I noticed slightly sluggish behaviour in the VTY, and
sure enough, the nitb was consuming 100% cpu, not always, but presumably
whenever it does a queue run. I also just heard that in the last few
days, we got a number of reports from users, some confirmed by photos of
the phones, about messages being delivered to the wrong destination.

Apologies if this has been touched on before. I am not finding where I
can directly search this list archives. Anyway, it might not be so bad
to bring it up, as the cron job purging via cron job calling sqlite3 is
not ideal, and anyway.. this crossed messages shouldn't happen, right? I
imagine we'd like to track this one down.

Keith.






From nhofmeyr at sysmocom.de  Tue Mar 14 00:04:53 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Tue, 14 Mar 2017 01:04:53 +0100
Subject: NITB: high cpu usage and "crossed" messages when SMS table grows
In-Reply-To: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
References: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
Message-ID: <20170314000453.GB1419@my.box>

On Mon, Mar 13, 2017 at 11:01:57PM +0100, Keith wrote:
> I temporarily disabled a cron job we run at rhizomatica that purges the
> hlr SMS table of sent messages every day.
> 
> After a few days I noticed slightly sluggish behaviour in the VTY, and
> sure enough, the nitb was consuming 100% cpu, not always, but presumably
> whenever it does a queue run.

Hmm, that's a very vague indicator. How performant is the hardware? For how
long does this load endure? Does the process hang otherwise, is service
disrupted?

From how I got to know the SMS code, it appears to have sound safeguards in
place, e.g.  limits the number of SMS to be delivered per queue run, and only
attempts deliveries of SMS for actually attached subscribers ... But in fact we
don't have load testing in place. It would be good to find out where
unproportional CPU load is coming from -- SQlite? The NITB sms code? From a
theoretical standpoint I'd also expect the SMS database to discard messages
that are past a certain age, not sure though, as I'm not that deeply familiar
with that (yet).

Would be good to know: how many SMS are pending, for how many subscribers, of
which how many are currently attached? How often are SMS deliveries being
retried and end in failure? ... and anything else you can think of.

> I also just heard that in the last few days, we got a number of reports from
> users, some confirmed by photos of the phones, about messages being delivered
> to the wrong destination.

Whoa! That should absolutely not happen. I can't see how this is even possible.

> I imagine we'd like to track this one down.

Optimally, we would want to be able to reproduce the failure. Do you have any
edge data on the scenario in which this situation comes up?

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170314/cdde768d/attachment.bin>

From alexander.chemeris at gmail.com  Tue Mar 14 08:49:52 2017
From: alexander.chemeris at gmail.com (Alexander Chemeris)
Date: Tue, 14 Mar 2017 11:49:52 +0300
Subject: NITB: high cpu usage and "crossed" messages when SMS table grows
In-Reply-To: <20170314000453.GB1419@my.box>
References: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
 <20170314000453.GB1419@my.box>
Message-ID: <CABmJbFVYNY=EHZueQiy7FFzhiu686nGVWVJa_Zu=K-fKO-xtAw@mail.gmail.com>

Hi Neels,

On Mar 14, 2017 03:05, "Neels Hofmeyr" <nhofmeyr at sysmocom.de> wrote:

On Mon, Mar 13, 2017 at 11:01:57PM +0100, Keith wrote:
> I temporarily disabled a cron job we run at rhizomatica that purges the
> hlr SMS table of sent messages every day.
>
> After a few days I noticed slightly sluggish behaviour in the VTY, and
> sure enough, the nitb was consuming 100% cpu, not always, but presumably
> whenever it does a queue run.

Hmm, that's a very vague indicator. How performant is the hardware? For how
long does this load endure? Does the process hang otherwise, is service
disrupted?

>From how I got to know the SMS code, it appears to have sound safeguards in
place, e.g.  limits the number of SMS to be delivered per queue run, and
only
attempts deliveries of SMS for actually attached subscribers ... But in
fact we
don't have load testing in place. It would be good to find out where
unproportional CPU load is coming from -- SQlite? The NITB sms code? From a
theoretical standpoint I'd also expect the SMS database to discard messages
that are past a certain age, not sure though, as I'm not that deeply
familiar
with that (yet).

Would be good to know: how many SMS are pending, for how many subscribers,
of
which how many are currently attached? How often are SMS deliveries being
retried and end in failure? ... and anything else you can think of.


We looked into this a couple years ago, but didn't come up with a good
solution context switched to something else.

Just add a few thousand SMS to the DB (DB should 10Mb or more roughly) and
start OsmoNITB. You'll notice it's agony immediately.

It's been a while, but IIRC the issue is that the DB didn't have proper
indexes for the kinds of queries we're running on it, so it's getting super
inefficient.

Regarding removing SMS - it should be fine based on validity time of the
SMS and it was completely broken. I had a patch set which fixed validity
time handling, but IIRC it wasn't merged. We can probably dug it up, but I
don't have much time to rebase / adapt it to the new codebase right now. If
there are any volunteers, that would be great.

Please excuse typos. Written with a touchscreen keyboard.

--
Regards,
Alexander Chemeris
CTO/Founder Fairwaves, Inc.
https://fairwaves.co
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170314/d9516a03/attachment.htm>

From daniel at totalueberwachung.de  Tue Mar 14 09:13:10 2017
From: daniel at totalueberwachung.de (Daniel Willmann)
Date: Tue, 14 Mar 2017 10:13:10 +0100
Subject: BS-11 Serial Interface
In-Reply-To: <20170310234232.GB612@yade.chaostreff.at>
 (sfid-20170311_004237_816660_D2BDA91B)
References: <20170310234232.GB612@yade.chaostreff.at>
 (sfid-20170311_004237_816660_D2BDA91B)
Message-ID: <1489482790.25309.51.camel@totalueberwachung.de>

On Sat, 2017-03-11 at 00:42 +0100, Alexander Huemer wrote:
> Hi,
> 
> I have a Siemens BS-11 with the factory cable for the LMT that has a 
> DSUB-9 plug on the non-BTS end.
> When I connect a USB<-->RS232 adapter and run either $ cu -l /dev/ttyUSB 
> or bs11_config -p /dev/ttyUSB0 I do not get any output when I switch on 
> the BTS.  I tried with nullmodem adapter and without just to be sure 
> since I often mix up the pinouts.

I remember that the LMT did not behave well either with USB serial
adapters or with a real serial port. So just to be sure if you have
access to a real serial port try it with that.


Regards,
Daniel



From nhofmeyr at sysmocom.de  Tue Mar 14 11:21:17 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Tue, 14 Mar 2017 12:21:17 +0100
Subject: NITB: high cpu usage and "crossed" messages when SMS table grows
In-Reply-To: <CABmJbFVYNY=EHZueQiy7FFzhiu686nGVWVJa_Zu=K-fKO-xtAw@mail.gmail.com>
References: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
 <20170314000453.GB1419@my.box>
 <CABmJbFVYNY=EHZueQiy7FFzhiu686nGVWVJa_Zu=K-fKO-xtAw@mail.gmail.com>
Message-ID: <20170314112117.GB1914@ass40.sysmocom.de>

On Tue, Mar 14, 2017 at 11:49:52AM +0300, Alexander Chemeris wrote:
> Just add a few thousand SMS to the DB (DB should 10Mb or more roughly) and
> start OsmoNITB. You'll notice it's agony immediately.
> 
> It's been a while, but IIRC the issue is that the DB didn't have proper
> indexes for the kinds of queries we're running on it, so it's getting super
> inefficient.

Ah, so that part is SQlite DB related. I've been very active in Osmocom
for the past 15 or so months thanks to sysmocom, but still learning new
aspects of the code base regularly :)

> Regarding removing SMS - it should be fine based on validity time of the
> SMS and it was completely broken. I had a patch set which fixed validity
> time handling, but IIRC it wasn't merged. We can probably dug it up, but I
> don't have much time to rebase / adapt it to the new codebase right now. If
> there are any volunteers, that would be great.

You could push it as a private branch without bothering to pull it up to
recent code and tell this list about it, maybe someone will be interested.

For Osmocom's future plans in general, we are moving away from having an
SQLite database in the OsmoNITB (and the new OsmoMSC). With the new VLR,
the database is only used for SMS, no longer for subscribers, and there
has been talk about implementing a proper SMSC, i.e. a separate process to
handle SMS. So we would probably not want to spend effort on optimizing
the old SMS storage "just before" we go on to get rid of it altogether.

Sending SMS to the wrong recipient though is probably worth fixing. That
should really not happen.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170314/edbfe553/attachment.bin>

From keith at rhizomatica.org  Tue Mar 14 13:05:04 2017
From: keith at rhizomatica.org (Keith)
Date: Tue, 14 Mar 2017 14:05:04 +0100
Subject: IMSI: Inf ?
Message-ID: <d12b1c08-9a88-40ec-63a0-6c83fb95624a@rhizomatica.org>

This is a bit vague again...

I have found across our sites several entries with a value of Inf for
IMSI on a create-on-demand subscriber:

For example:

sqlite> select * from Subscriber where id=284054;
284054|2017-03-08 21:19:40|2017-03-08 21:19:40|Inf||20649|0||0|

I also have some Inf values for IMEI in Equipment:

sqlite> select id,created,imei from Equipment where id = 95108;
95108|2014-03-13 03:18:06|Inf

Not a big deal for operations, just thought I'd mention it as I noticed
while restoring backups, (sqlite3 doesn't like to reimport the Inf value
it writes itself with .dump)







From keith at rhizomatica.org  Tue Mar 14 13:15:24 2017
From: keith at rhizomatica.org (Keith)
Date: Tue, 14 Mar 2017 14:15:24 +0100
Subject: NITB: high cpu usage and "crossed" messages when SMS table grows
In-Reply-To: <20170314000453.GB1419@my.box>
References: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
 <20170314000453.GB1419@my.box>
Message-ID: <874a1ff6-4fa0-0021-8d10-72167e38fed7@rhizomatica.org>



On 14/03/2017 01:04, Neels Hofmeyr wrote:
> On Mon, Mar 13, 2017 at 11:01:57PM +0100, Keith wrote:
>> the nitb was consuming 100% cpu, not always, but presumably
>> whenever it does a queue run.
> Hmm, that's a very vague indicator. 
I know, I know.. :-/
> Would be good to know: how many SMS are pending, for how many subscribers, of
> which how many are currently attached? How often are SMS deliveries being
> retried and end in failure? ... and anything else you can think of.
Yesterday I did
 DELETE FROM SMS WHERE sent IS NOT NULL
and the cpu usage problem goes away.

Previous to that, On the site where I most noticed it:

Total entries in table: 114231
Pending (not sent): 1662
Distinct number of subscribers with pending SMS: 501

So as Alexander says, It seems to have to do with simply the amount of
entries in the table.
Alex, if you dig out those patches, I'm happy to submit them for Code
Review.
>> messages being delivered
>> to the wrong destination.
> Whoa! That should absolutely not happen. I can't see how this is even possible.
That's exactly the reaction that happened when I mentioned it at
OsmoDevCon last year. :-)
I have also been somewhat incredulous of these reports, putting it down
to user error, or possibly something in our SMPP->kannel->python
stuff->kannel->SMPP->Osmo chain, and after working a little on that
code, the problem ceased to be reported, but it's quite telling that at
the same time that I stop purging the SMS table and we grow above
100,000 entries in SMS table, we get reports from at least 4 communities
of these "crossed" messages.

k/




From msuraev at sysmocom.de  Tue Mar 14 13:20:12 2017
From: msuraev at sysmocom.de (Max)
Date: Tue, 14 Mar 2017 14:20:12 +0100
Subject: autoconf and .deb question
Message-ID: <d77d876d-7ea0-8d4e-e41d-e4fdb86bff6e@sysmocom.de>

Hi.

There's strange thing about the way autoconf and debian/rules interact 
for OpenBSC:
- in openbsc/include/openbsc/Makefile.am gsm_data_shared.h and 
common_cs.h are in noinst_HEADERS
- in debian/openbsc-dev.install they are marked for installation

And this actually works! If you run "dpkg-buildpackage -uc -us -tc" 
you'll get openbsc-dev*.deb with those headers properly installed.

It's nice but feels a bit too much like magic. Any ideas how dh_install 
(or smth else?) manages to work around it?
Should we be worried that it might break some time in future? Should we 
move those headers away from noinst_* section?

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From keith at rhizomatica.org  Tue Mar 14 13:37:00 2017
From: keith at rhizomatica.org (Keith)
Date: Tue, 14 Mar 2017 14:37:00 +0100
Subject: NITB: high cpu usage and "crossed" messages when SMS table grows
In-Reply-To: <20170314112117.GB1914@ass40.sysmocom.de>
References: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
 <20170314000453.GB1419@my.box>
 <CABmJbFVYNY=EHZueQiy7FFzhiu686nGVWVJa_Zu=K-fKO-xtAw@mail.gmail.com>
 <20170314112117.GB1914@ass40.sysmocom.de>
Message-ID: <33556465-1029-4169-c51f-95c85630ff3a@rhizomatica.org>



On 14/03/2017 12:21, Neels Hofmeyr wrote:
> Ah, so that part is SQlite DB related. I've been very active in Osmocom
> for the past 15 or so months thanks to sysmocom, but still learning new
> aspects of the code base regularly :)
Yep, there are some big FIXME message in openbsc/src/libmsc/db.c:
db_sms_store()
>> Regarding removing SMS - it should be fine based on validity time of the
I am mildly concerned about concurrent write access to the sqlite hlr,
although I saw the SMS table corrupted once, that prevented the nitb
strting, I'm not even sure it was caused by concurrent writes. I read up
about locking and how sqlite handles this, it should be OK.
>> SMS and it was completely broken. I had a patch set which fixed validity
>> time handling, but IIRC it wasn't merged. We can probably dug it up, but I
>> don't have much time to rebase / adapt it to the new codebase right now. If
>> there are any volunteers, that would be great.
>
>
> For Osmocom's future plans in general, we are moving away from having an
> SQLite database in the OsmoNITB (and the new OsmoMSC). 
>  So we would probably not want to spend effort on optimizing
> the old SMS storage "just before" we go on to get rid of it altogether.

Yep.. how long is just before? (in ms please) :-)
There isn't really an uncomplicated (out-of-the-box) SMSC solution at
this time.
We should probably fix this up and create some internal purging of sent SMS?
I wanted to write routines to view more info about the SMS queue from
the vty. Maybe this part is not worth it, especially as I've done it in
python.

>
> Sending SMS to the wrong recipient though is probably worth fixing. That
> should really not happen.
No, it makes people rather irate. :-(



From ruben.undheim at gmail.com  Tue Mar 14 13:38:44 2017
From: ruben.undheim at gmail.com (Ruben Undheim)
Date: Tue, 14 Mar 2017 14:38:44 +0100
Subject: autoconf and .deb question
In-Reply-To: <d77d876d-7ea0-8d4e-e41d-e4fdb86bff6e@sysmocom.de>
References: <d77d876d-7ea0-8d4e-e41d-e4fdb86bff6e@sysmocom.de>
Message-ID: <CA+ChNyXh0LnB8y9yjnzaB+pAPhxDD-g=+sBKJEM=TNa7OF+47Q@mail.gmail.com>

Hi Max,

When using two columns for a line in a debian/*.install file, you
actually pick from the source tree - not from the installed files! The
first column specify which files in the source tree to pick, the
second column specify where it will be installed. See
https://www.debian.org/doc/manuals/maint-guide/dother.en.html#install

> Should we be worried that it might break some time in future? Should we move those headers away from noinst_* section?

So this should always work - as long as the files are not moved to
another place in the source tree.. No need to worry.


Cheers
Ruben



2017-03-14 14:20 GMT+01:00 Max <msuraev at sysmocom.de>:
> Hi.
>
> There's strange thing about the way autoconf and debian/rules interact for
> OpenBSC:
> - in openbsc/include/openbsc/Makefile.am gsm_data_shared.h and common_cs.h
> are in noinst_HEADERS
> - in debian/openbsc-dev.install they are marked for installation
>
> And this actually works! If you run "dpkg-buildpackage -uc -us -tc" you'll
> get openbsc-dev*.deb with those headers properly installed.
>
> It's nice but feels a bit too much like magic. Any ideas how dh_install (or
> smth else?) manages to work around it?
> Should we be worried that it might break some time in future? Should we move
> those headers away from noinst_* section?
>
> --
> Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
> =======================================================================
> * sysmocom - systems for mobile communications GmbH
> * Alt-Moabit 93
> * 10559 Berlin, Germany
> * Sitz / Registered office: Berlin, HRB 134158 B
> * Geschaeftsfuehrer / Managing Director: Harald Welte
>


From msuraev at sysmocom.de  Tue Mar 14 17:41:20 2017
From: msuraev at sysmocom.de (Max)
Date: Tue, 14 Mar 2017 18:41:20 +0100
Subject: osmo-bts-trx channel activation
Message-ID: <a6c5f7bf-8eec-be8a-3f16-5e21c4385f37@sysmocom.de>

Hi.

This has been asked last year but to no avail, let's try again.

How logical channels are (de)activated in osmo-bts-trx? For example in 
all the other models there's sapi_deactivate_cb() callback but 
osmo-bts-trx doesn't seem to be using sapi_cmd at all.

To put this into perspective - while working on 
https://projects.osmocom.org/issues/1575 I got to activate/deactivate 
lchans when SI3 is received. This already works for sysmobts but I'd 
like to do the same for osmo-bts-trx as well.

Is there any documentation somewhere describing how it works there? Or 
maybe someone more familiar with the code could help to shed the light 
on it?

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From nhofmeyr at sysmocom.de  Tue Mar 14 20:21:05 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Tue, 14 Mar 2017 21:21:05 +0100
Subject: osmo-bts-trx channel activation
In-Reply-To: <a6c5f7bf-8eec-be8a-3f16-5e21c4385f37@sysmocom.de>
References: <a6c5f7bf-8eec-be8a-3f16-5e21c4385f37@sysmocom.de>
Message-ID: <20170314202105.GA5538@my.box>

On Tue, Mar 14, 2017 at 06:41:20PM +0100, Max wrote:
> How logical channels are (de)activated in osmo-bts-trx? For example in all
> the other models there's sapi_deactivate_cb() callback but osmo-bts-trx
> doesn't seem to be using sapi_cmd at all.

You could take a look at the dynamic timeslot implementation, i.e.
bts_model_ts_disconnect() and bts_model_ts_connect(). The way it looks is that
all you need to do is to set the new config of the channel. After all,
osmo-bts-trx has no lower level hardware to be instructed to do anything, it's
all in the structs in RAM: bts_model_ts_disconnect() for trx is empty.
bts_model_ts_connect() basically just sets the new TS config using
trx_set_ts_as_pchan(). I asked myself the same question back then, and in the
end found out that simply setting the new config works fine.

I hope that helps ... and is even related to your question :)

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170314/62bc086c/attachment.bin>

From nhofmeyr at sysmocom.de  Wed Mar 15 13:22:23 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Wed, 15 Mar 2017 14:22:23 +0100
Subject: NITB: high cpu usage and "crossed" messages when SMS table grows
In-Reply-To: <33556465-1029-4169-c51f-95c85630ff3a@rhizomatica.org>
References: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
 <20170314000453.GB1419@my.box>
 <CABmJbFVYNY=EHZueQiy7FFzhiu686nGVWVJa_Zu=K-fKO-xtAw@mail.gmail.com>
 <20170314112117.GB1914@ass40.sysmocom.de>
 <33556465-1029-4169-c51f-95c85630ff3a@rhizomatica.org>
Message-ID: <20170315132223.GB1357@my.box>

On Tue, Mar 14, 2017 at 02:37:00PM +0100, Keith wrote:
> We should probably fix this up and create some internal purging of sent SMS?

> > Sending SMS to the wrong recipient though is probably worth fixing. That
> > should really not happen.
> No, it makes people rather irate. :-(

The social dimension of it is potentially quite destructive :P
"Wait what!? *Who* is *Mandy*!?!?"

Plus the IMSI == Inf thing.

I fully agree and would love to jump in and help out, but unfortunately I can't
divert my attention from the current commitment, at least not now...
Switching to an external SMSC is probably also going to take Inf ms.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170315/260b5242/attachment.bin>

From laforge at gnumonks.org  Wed Mar 15 13:43:38 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Wed, 15 Mar 2017 14:43:38 +0100
Subject: proper e-mail quoting is important (was Re: NITB: high cpu usage and
 "crossed" messages when SMS table grows)
In-Reply-To: <CABmJbFVYNY=EHZueQiy7FFzhiu686nGVWVJa_Zu=K-fKO-xtAw@mail.gmail.com>
References: <c16a4efb-efc3-5736-0ea5-da341242b544@rhizomatica.org>
 <20170314000453.GB1419@my.box>
 <CABmJbFVYNY=EHZueQiy7FFzhiu686nGVWVJa_Zu=K-fKO-xtAw@mail.gmail.com>
Message-ID: <20170315134338.vo76stojphpnbc65@nataraja>

Hi Alexander,

I would really appreciate if you could fix your quoting in your e-mails.

My time is (sorry) too limited to have to read very carefully through
every line and then try to figure out if it is something you quoted from
somebody else's mail, or it is something you actually added to the
discussion.  This means reading such a mail takes a multitude of reading
properly formatted mails.  Thanks for your attention.

In fact, I had already started to not read through your most recent
mails anymore due to the lack of proper quoting, and I thought it would
be better to actually let you know rather than silently disregarding the
mails.  I apologize for my impulsive response to this, and use this
message as a general reminder to everyone on this list on how important
proper quoting is.

Regards,
	Harald

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Thu Mar 16 00:53:58 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 16 Mar 2017 01:53:58 +0100
Subject: Build failure of network:osmocom:nightly/libosmocore in
 xUbuntu_16.10/i586
In-Reply-To: <58c99bb922be5_6b03efc1094313f@build.opensuse.org>
References: <build-bf35add6a68a4e1911237b342a5f2dc5@build.opensuse.org>
 <58c99bb922be5_6b03efc1094313f@build.opensuse.org>
Message-ID: <20170316005358.GC1357@my.box>

should be fixed by https://gerrit.osmocom.org/2089

On Wed, Mar 15, 2017 at 07:52:33PM +0000, OBS Notification wrote:
> Visit https://build.opensuse.org/package/live_build_log/network:osmocom:nightly/libosmocore/xUbuntu_16.10/i586
> 
> Package network:osmocom:nightly/libosmocore failed to build in xUbuntu_16.10/i586
> 
> Check out the package for editing:
>   osc checkout network:osmocom:nightly libosmocore
> 
> Last lines of build log:
> [  143s]  RAND:	6a61050765caa32c90371370e5d6dc2d
> [  143s] -AUTN:	afb993e4f4b8000069cdeebb4a4b5b58
> [  144s] +AUTN:	504693e4f4b80000e3963072c50f0b91
> [  144s]  IK:	c348c2fe2f3e1fb37a7ae1638163bd98
> [  144s]  CK:	e740c156278705a14e1a99ba6d31334f
> [  144s]  RES:	7c04e86a67967fcd
> [  144s]  SRES:	1b9297a7
> [  144s]  Kc:	10687b71e4eb94c5
> [  144s] -SQN:	281474976710655
> [  144s] +SQN:	4294967295
> [  144s]  
> [  144s]  
> [  144s]  > osmo-auc-gen -3 -a milenage -r 39fa2f4e3d523d8619a73b4f65c3e14d -k EB215756028D60E3275E613320AEC880 -o FB2A3D1B360F599ABAB99DB8669F8308 -A 979498b1f72d3e28c59fa2e72f9c
> [  144s] 40. testsuite.at:253: 40. osmo-auc-gen (testsuite.at:253): FAILED (testsuite.at:257)
> [  144s] debian/rules:26: recipe for target 'override_dh_auto_test' failed
> [  144s] make[1]: *** [override_dh_auto_test] Error 1
> [  144s] make[1]: Leaving directory '/usr/src/packages/BUILD'
> [  144s] debian/rules:15: recipe for target 'build' failed
> [  144s] make: *** [build] Error 2
> [  144s] dpkg-buildpackage: error: debian/rules build gave error exit status 2
> [  144s] 
> [  144s] build85 failed "build libosmocore_0.9.6.20170315.dsc" at Wed Mar 15 19:52:15 UTC 2017.
> [  144s] 
> [  144s] ### VM INTERACTION START ###
> [  146s] [  136.282415] reboot: Power down
> [  147s] ### VM INTERACTION END ###
> [  147s] 
> [  147s] build85 failed "build libosmocore_0.9.6.20170315.dsc" at Wed Mar 15 19:52:19 UTC 2017.
> [  147s] 
> 
> -- 
> Configure notifications at https://build.opensuse.org/user/notifications
> openSUSE Build Service (https://build.opensuse.org/)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170316/3195154a/attachment.bin>

From nhofmeyr at sysmocom.de  Thu Mar 16 13:30:24 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 16 Mar 2017 14:30:24 +0100
Subject: [MERGED] openbsc[master]: gprs_sgsn.c: initialize ptmsi with
 0xdeadbeef
In-Reply-To: <20170314133416.8EAF42E4C2@lists.osmocom.org>
References: <gerrit.1486169569048.Ibc2d585c5db899e6af20104211e32faf3822633a@gerrit.osmocom.org>
 <20170314133416.8EAF42E4C2@lists.osmocom.org>
Message-ID: <20170316133024.GA1586@ass40.sysmocom.de>

> -	uint32_t ptmsi;
> +	uint32_t ptmsi = 0xdeadbeef;

GSM_RESERVED_TMSI would make more sense here I guess (the "invalid TMSI"
value).

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170316/68311ce9/attachment.bin>

From laforge at gnumonks.org  Thu Mar 16 18:47:58 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 16 Mar 2017 19:47:58 +0100
Subject: OsmoTRX runtime CPU optimization detection
Message-ID: <20170316184758.jru3yznbgjgorpz7@nataraja>

FYI, Philipp has been working on some patches to do the OsmoTRX cpu
instruction set extension detection at runtime rather than compile-time.

See "Bug #1869: osmo-trx binary cannot be moved to similar CPU"
https://osmocom.org/issues/1869#change-3394

As not everyone is following each redmine ticket or gerrit patch:

I would like the OsmoTRX folks (Tom, Alexander, ...) to have a look at
this.  For convenience, the latest update to the ticket and the
associated links to the gerrit patches below:

---

The matching implementation is now selected dynamically during runtime.
In order to be sure that the convolution part did not break, I wrote a
small test program to compute some testvectors. I compared the results
before and after my changes. They match up. I made only minimal changes
to the conversion code, so I skipped testing that as well.

The SSE relevant sources are compiled with $(SIMD_FLAGS) now. The
sources only contain the SSE implementation and the decision logic to
defer to the correct implementation during runtime. That should run fine
on non SSE3 / SSE4.1 CPUs, since the decision logic is not
vectorize-able. However, we can divide this up further as discussed.

https://gerrit.osmocom.org/2098 buildenv: Turn off native architecture builds
https://gerrit.osmocom.org/2099 cosmetic: Make parameter lists uniform
https://gerrit.osmocom.org/2100 ssedetect: Add runtime CPU detection
https://gerrit.osmocom.org/2101 cosmetic: Add info about SSE support
https://gerrit.osmocom.org/2102 buildenv: Make build CPU invariant
https://gerrit.osmocom.org/2103 cosmetic: remove code duplication
https://gerrit.osmocom.org/2104 Add test program to verify convolution implementation

---

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Thu Mar 16 19:31:57 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 16 Mar 2017 20:31:57 +0100
Subject: coverity scan for vlr_2G and vlr_3G
Message-ID: <20170316193157.GB1586@ass40.sysmocom.de>

It seems that we won't be merging vlr_2G or vlr_3G into openbsc.git's
master, at least not soon. Instead we're likely to move to a new git
repository altogether to mark the changed setup.

We could switch 'Osmocom' over to vlr_3G and assume that it includes the
others, but that's not accurate at all. The VLR stuff cuts away half the
code from openbsc.git's master and 3G the other half. well, almost :)

So I thought we could repurpose two of the unused coverity scan projects
we have for the vlr_2G and vlr_3G branches. Or maybe one for vlr_3G would
be enough.

It seems we can't change the name of a coverity project.  Maybe we can
just use the OpenBSC one for vlr_3G without changing the name?

We could possibly also include a separate build within the tar sent to
the Osmocom job, I can also check that out.

Or I can try and register a brand new one.

Any preferences?

~N

-- 
- Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Gesch?ftsf?hrer / Managing Directors: Harald Welte
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170316/aa2cac09/attachment.bin>

From laforge at gnumonks.org  Thu Mar 16 22:06:28 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 16 Mar 2017 23:06:28 +0100
Subject: coverity scan for vlr_2G and vlr_3G
In-Reply-To: <20170316193157.GB1586@ass40.sysmocom.de>
References: <20170316193157.GB1586@ass40.sysmocom.de>
Message-ID: <20170316220628.7phipnq655gu7zqy@nataraja>

Hi Neels,

On Thu, Mar 16, 2017 at 08:31:57PM +0100, Neels Hofmeyr wrote:
> It seems that we won't be merging vlr_2G or vlr_3G into openbsc.git's
> master, at least not soon. Instead we're likely to move to a new git
> repository altogether to mark the changed setup.

yes, I think basically as soon as the naming is clear.  Do you have any
ideas?

The [post-VLR] repository currently contains the following main programs
(excluding various utilities like abisip-find, isdnsync, measurement
stuff, ... :

* osmo-bsc
* osmo-bsc_nat
* osmo-msc (NITB without HLR and BSC)

On the GPRS side we have
* osmo-sgsn
* osmo-gbproxy
* osmo-gtphub

So in terms of clean-up, one of the questions is: what exactly do the
circuit switched programs share with the packet-switched ones?  Maybe
there's a chance for even splitting up those two unrelated parts?

But whether CS+PS are still in one repo, or whether they will continue
in two repositories: The question of naming is the difficult one.

Getting rid of the OpenBSC name is good.  It served us well initially,
but it is a mis-nomer for a long time.  Osmocom is very generic and
covers all the various projects, even those not 3GPP related.

In redmine we have the 'cellular infrastructure' project as a parent to
all the 2G/3G related projects.  Maybe we should call it
'osmo-cell-infra', 'osmo-2g3g' or even more generic 'osmo-cellular'?
that's actually again a bit too generic, as the BTS and PCU are prime
example of other cellular infrastructure projects that are not part of
that repo.  Also, there's the HLR which lives in a separate repo -
simply because it shares no code with the other projects.  Still, it
might actually be worth to consider merging it into the same repo as the
MSC.

Maybe a non-descriptive name should be used?  Or, in the great tradition
of the telecom world, yet another acronym?

In general, when we do the "fork" to the new repo, I would like to use
the opportunity to get rid of the extra 'openbsc' sub directory, which
currently really only exists because everyone's spec file / OE recipe /
debian-rules expects it to be that way.

In terms of your question about coverity:

> Any preferences?

I really don't care ;)

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From msuraev at sysmocom.de  Thu Mar 16 22:21:25 2017
From: msuraev at sysmocom.de (Max)
Date: Thu, 16 Mar 2017 23:21:25 +0100
Subject: coverity scan for vlr_2G and vlr_3G
In-Reply-To: <20170316220628.7phipnq655gu7zqy@nataraja>
References: <20170316193157.GB1586@ass40.sysmocom.de>
 <20170316220628.7phipnq655gu7zqy@nataraja>
Message-ID: <a5c0019f-5a2c-8436-613f-60a1ff769681@sysmocom.de>

OsmoXSC or Osmo-SC perhaps? :)

16.03.2017 23:06, Harald Welte ?????:
> The question of naming is the difficult one.

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
======================================================================= 
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93 
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B 
* Geschaeftsfuehrer / Managing Directors: Holger Freyther, Harald Welte 






From laforge at gnumonks.org  Fri Mar 17 20:17:19 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Fri, 17 Mar 2017 21:17:19 +0100
Subject: git.osmocom.org cgit improvements
Message-ID: <20170317201719.r3d6n3ab35wke4mu@nataraja>

Hi all,

today I've deployed some cgit improvements on https://git.osmocom.org/,
in the hope that it makes this tool even more useful:

1) syntax highlighting of source code (requested by Hoernchen)

The source code is now highlighted by pygments.  I don't really
understand why somebody would want to look at source code a lot in a
browser, but well, it was as easy as to enable the existing pygments
based filter plugin.


2) rendering of "about" page from README.md

As you might have noticed, I've introduced a README.md in a number of
repositoires, and cgit is now rendering an about page for every
repository, e.g. at http://git.osmocom.org/libosmo-abis/about/


3) gerrit change-ID hyperlink generation

All gerrit Change-IDs in commit messages are now automatically converted
to hyperlinks to the respective gerrit change, see e.g. the below
example:
http://git.osmocom.org/openbsc/commit/?id=6dd0fc685b7149f67a5fe17a5bce55c446aa563c

Please note that this works for the "Change-Id" line of the actual
change, but also for change-ids in the free text (e.g. "this depends on
change-id ... in libosmocore")


4) Osmocom ticket/issue hyperlink generation

Any Line that matches the "^((Relate|Close|Fixe)[ds]):" prefix is
scanned for occurrences of "OS#(\d+)" which are then amended with
hyperlinks to the respective issue on osmocom.org

Please note the OS# prefix is mandatory, so things like "OS#1614, 1615"
will not work, as can be seen at
http://git.osmocom.org/osmo-pcu/commit/?id=0a8fae8d141c2cfa4387ffe9b35402d5b8cc85cd

Please format your commit messages accordingly.

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Sat Mar 18 11:49:20 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Sat, 18 Mar 2017 12:49:20 +0100
Subject: nightly packages failures
Message-ID: <20170318114920.GA3517@my.box>

openggsn failed because README was renamed to README.md without adjusting the debian installation rules (possibly the same change occured in other repositories, Harald?)

the osmo-auc-gen_test.sh still fails on 32bit systems, to be fixed by https://gerrit.osmocom.org/2089 (please review)

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170318/9168fb21/attachment.bin>

From nhofmeyr at sysmocom.de  Sat Mar 18 15:04:10 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Sat, 18 Mar 2017 16:04:10 +0100
Subject: coverity scan for vlr_2G and vlr_3G
In-Reply-To: <20170316220628.7phipnq655gu7zqy@nataraja>
References: <20170316193157.GB1586@ass40.sysmocom.de>
 <20170316220628.7phipnq655gu7zqy@nataraja>
Message-ID: <20170318150410.GA3272@my.box>

On Thu, Mar 16, 2017 at 11:06:28PM +0100, Harald Welte wrote:
> The [post-VLR] repository currently contains the following main programs
> (excluding various utilities like abisip-find, isdnsync, measurement
> stuff, ... :
> 
> * osmo-bsc
> * osmo-bsc_nat
> * osmo-msc (NITB without HLR and BSC)

To clarify for anyone else reading along: the VLR, i.e. libvlr, sits in the
OsmoNITB, closely tied to libmsc, on the vlr_2G and vlr_3G branches. See also
the block diagrams towards the end of https://osmocom.org/news/67

> So in terms of clean-up, one of the questions is: what exactly do the
> circuit switched programs share with the packet-switched ones?

The answer to this *should* be: anything found in libcommon. For things shared
among the circuit-switched parts, we have libcommon-cs, so the rest should be
relevant for cs + ps ... However, that's not accurate.

Taking a look at libcommon:

Used everywhere but makes sense to separate instead:
  bsc_version.c  -- just the copyright string
  common_vty.c   -- vty_go_parent() combined for various VTY trees
  debug.c        -- logging categories and filters, combined

Only related to gprs because above debug.c combines unrelated scopes:
  gsm_subscriber_base.c -- debug.c filter_fn uses msc/bsc subscr_get()/subscr_put()

Completely unrelated to gprs:
  gsm_data.c     -- except tall_bsc_ctx, gprs should have its own.
  gsm_data_shared.c
  talloc_ctx.c
  socket.c       -- actually only used by osmo-bsc_nat and ipaccess-proxy,
                    should probably use libosmocore socket code instead?

Used across gprs and cs:
  gsup_client.c       -- not yet, but used by CS on vlr_2G branch
  gsup_test_client.c
  oap_client.c

These are proven / illustrated by a playful branch that moves things to
libcommon-cs: https://git.osmocom.org/openbsc/log/?h=neels/libcommon-cs

In fact it looks like rather than splitting off libcommon-cs, we should have
created a libosmo-gsupclient and left libcommon to be used by cs programs only,
separating the minor generic bits to gprs/.

Then there's also libiu, which is shared across OsmoSGSN and OsmoMSC on the 3G
branch to provide IuPS and IuCS, respectively.


In summary: only gsup_client and libiu are shared across cs and ps, both of
which are also just needed by libmsc/libvlr and OsmoSGSN. (not by osmo-bsc*,
not by gb_proxy nor by any utils.)

If we wanted to completely separate cs from ps, we could technically move
gsup_client to libosmocore and libiu to osmo-iuh. That would allow having
separate osmo-bsc+msc and osmo-sgsn repositories. Actually, our plans for
separating MSC and BSC also calls for separating gsm_subscriber_connection and
gsm_network, and at that point it would also make sense to have separate
osmo-bsc and osmo-msc repositories. The names would then come naturally.  We
could install libbsc from osmo-bsc and link it in from osmo-msc to build
OsmoNITB while we still need to; and when the A-interface comes along stop
linking libbsc from osmo-msc.

If we keep bsc, msc and gprs combined, we need to come up with a logical name.
Maybe the fact that we still haven't found one is an indicator that we should
rather separate openbsc.git up by the familiar names as above?

> In redmine we have the 'cellular infrastructure' project as a parent to
> all the 2G/3G related projects.  Maybe we should call it
> 'osmo-cell-infra', 'osmo-2g3g' or even more generic 'osmo-cellular'?
> that's actually again a bit too generic, as the BTS and PCU are prime
> example of other cellular infrastructure projects that are not part of
> that repo.  Also, there's the HLR which lives in a separate repo -
> simply because it shares no code with the other projects.  Still, it
> might actually be worth to consider merging it into the same repo as the
> MSC.

With steve, the name Osmocom-CN has kind of stuck, for osmocom core network.
But again the HLR is separate and it's not really accurate. Osmocom-CN would
make sense if we have a joint BSC, MSC, SGSN and HLR repository.

> Maybe a non-descriptive name should be used?  Or, in the great tradition
> of the telecom world, yet another acronym?

Nondescriptive... osmo-kitchensink? :P

Weird that they should have so many acronyms like SIGTRAN, NAS, GERAN/UTRAN,
and none of them match our intended grouping :)

But what *is* our intended grouping?  I (naively?) tend to like complete
separation into separate BSC, separate MSC, separate SGSN, because it matches
the separation that the code is moving towards, and our redmine project tree.

> In general, when we do the "fork" to the new repo, I would like to use
> the opportunity to get rid of the extra 'openbsc' sub directory

+1

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170318/a5e9927d/attachment.bin>

From nhofmeyr at sysmocom.de  Sat Mar 18 15:09:31 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Sat, 18 Mar 2017 16:09:31 +0100
Subject: git.osmocom.org cgit improvements
In-Reply-To: <20170317201719.r3d6n3ab35wke4mu@nataraja>
References: <20170317201719.r3d6n3ab35wke4mu@nataraja>
Message-ID: <20170318150931.GB3272@my.box>

On Fri, Mar 17, 2017 at 09:17:19PM +0100, Harald Welte wrote:
> 2) rendering of "about" page from README.md
> 
> As you might have noticed, I've introduced a README.md in a number of
> repositoires, and cgit is now rendering an about page for every
> repository, e.g. at http://git.osmocom.org/libosmo-abis/about/

It seems like these need some follow-ups in the debian packaging instructions
-- i.e. update to the new .md suffix or mark as installation (or ignore, but I
guess rather installation) file.

> 4) Osmocom ticket/issue hyperlink generation
> 
> Any Line that matches the "^((Relate|Close|Fixe)[ds]):" prefix is
> scanned for occurrences of "OS#(\d+)" which are then amended with
> hyperlinks to the respective issue on osmocom.org

Ah, so when OS#123 appears in the commit log's free text, it isn't linked?
Can we just scan everything for "\<OS#\d+\>"?

All in all these are excellent improvements!
Holger, what's the status of your promise, made one day in a chat, to make
redmine catch the OS#123s?

~N

-- 
- Neels Hofmeyr <nhofmeyr at sysmocom.de>          http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Gesch?ftsf?hrer / Managing Directors: Harald Welte
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170318/c20a9250/attachment.bin>

From laforge at gnumonks.org  Sat Mar 18 17:06:09 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Sat, 18 Mar 2017 18:06:09 +0100
Subject: coverity scan for vlr_2G and vlr_3G
In-Reply-To: <20170318150410.GA3272@my.box>
References: <20170316193157.GB1586@ass40.sysmocom.de>
 <20170316220628.7phipnq655gu7zqy@nataraja>
 <20170318150410.GA3272@my.box>
Message-ID: <20170318170609.3wurhi6p2lelxff3@nataraja>

Hi Neels,

On Sat, Mar 18, 2017 at 04:04:10PM +0100, Neels Hofmeyr wrote:
> With steve, the name Osmocom-CN has kind of stuck, for osmocom core network.
> But again the HLR is separate and it's not really accurate. Osmocom-CN would
> make sense if we have a joint BSC, MSC, SGSN and HLR repository.

No, it does not make sense seven in that case, because the BSC is not
part of the Core Network.

> But what *is* our intended grouping?  I (naively?) tend to like complete
> separation into separate BSC, separate MSC, separate SGSN, because it matches
> the separation that the code is moving towards, and our redmine project tree.

yes, but what do we do until that is reached?  It will be quite some
time until there is a stand-alone MSC with A interface.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From holger at freyther.de  Sun Mar 19 20:57:57 2017
From: holger at freyther.de (Holger Freyther)
Date: Sun, 19 Mar 2017 21:57:57 +0100
Subject: git.osmocom.org cgit improvements
In-Reply-To: <20170318150931.GB3272@my.box>
References: <20170317201719.r3d6n3ab35wke4mu@nataraja>
 <20170318150931.GB3272@my.box>
Message-ID: <9437479C-B7C9-4603-B01D-EC775130F164@freyther.de>


> On 18 Mar 2017, at 16:09, Neels Hofmeyr <nhofmeyr at sysmocom.de> wrote:
> 
> 

Hi,


> All in all these are excellent improvements!
> Holger, what's the status of your promise, made one day in a chat, to make
> redmine catch the OS#123s?

lol, I love how we go from a private chat to this CC list. I have not looked at it but maybe this is something for an afternoon at Osmodevcon. It should be fairly simple with redmine.

holger

From nhofmeyr at sysmocom.de  Mon Mar 20 04:30:52 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 20 Mar 2017 05:30:52 +0100
Subject: gerrit openid login: osmocom.org vs. projects.osmocom.org
Message-ID: <20170320043052.GB12441@my.box>

Has someone made osmocom.org login redirect to projects.osmocom.org?
Because since recently I observe this:


I go to osmocom.org, click on "Sign in".
It *redirects* me to projects.osmocom.org/login.
I log in.

I go to gerrit, enter, as always

  https://osmocom.org/openid

I get another login screen, this time on osmocom.org without 'projects.'
Interesting, there seem to be two realms, maybe from cookie rules.

Ok then, I add projects to the openid, for gerrit login:

  https://projects.osmocom.org/openid

and it works, nice.

However, now I seem to be logged in as a kind of ghost of my user. I'm logged
in as 'nhofmeyr at sysmocom.de', but no patches are on my page and I don't have
the voting nor admin permissions I normally have.

When instead of clicking on "Sign in" on osmocom.org redmine, I *manually* enter
  https://osmocom.org/login
(omitting projects.), I can login on osmocom.org and my gerrit user works out.

I notice that with projects.osmocom.org I am user ID 1000073,
while with osmocom.org I am 1000005.

In the gerrit user database, I see distinct user IDs:

? ssh go 'gerrit gsql -c "select * from account_external_ids where account_id = 1000073 or account_id = 1000005"'
 ACCOUNT_ID | EMAIL_ADDRESS         | EXTERNAL_ID
 -----------+-----------------------+--------------------------------------------
 1000005    | nhofmeyr at sysmocom.de  | https://osmocom.org/openid/user/91
 1000005    | NULL                  | username:neels
 1000073    | nhofmeyr at sysmocom.de  | https://projects.osmocom.org/openid/user/91

When I manually patch up the 1000073 to 1000005 in the last row, both openid
URLs work out to the correct user.


So gerrit potentially gets confused by one and the same user, fails to match
the email addresses rather than the openid provider.

Looking at the other registered users, most use the osmocom.org and not
projects.osmocom.org, so you all may be susceptible to the same issue.

I also see that four have entered http:// as openid, without SSL, which seems
to me is something we should rather not allow.

For example, laforge's user is shadowed in the same way just because of the non-https:

 1000004    | laforge at gnumonks.org  | https://osmocom.org/openid/user/7
 1000021    | laforge at gnumonks.org  | http://osmocom.org/openid/user/7

If redirecting to projects.o.o is intentional and the way to go (TM), I should
probably pre-empt problems for existing users by creating external ids with
'projects' in the openid url, pointing at the proper existing users.

Otherwise we should avoid magical forwarding of osmocom.org logins to
projects.osmocom.org.

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170320/8b677aea/attachment.bin>

From nhofmeyr at sysmocom.de  Mon Mar 20 04:36:31 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 20 Mar 2017 05:36:31 +0100
Subject: git.osmocom.org cgit improvements
In-Reply-To: <9437479C-B7C9-4603-B01D-EC775130F164@freyther.de>
References: <20170317201719.r3d6n3ab35wke4mu@nataraja>
 <20170318150931.GB3272@my.box>
 <9437479C-B7C9-4603-B01D-EC775130F164@freyther.de>
Message-ID: <20170320043631.GC12441@my.box>

On Sun, Mar 19, 2017 at 09:57:57PM +0100, Holger Freyther wrote:
> > Holger, what's the status of your promise, made one day in a chat, to make
> > redmine catch the OS#123s?
> 
> lol, I love how we go from a private chat to this CC list.

Hey, if you love it that much, I can come up with plenty more private
conversations between us I could Cc around... ;)

In other words, sorry about disclosing. I kind of remembered it was a public
commitment. And it doesn't need to stick, either, anyone could take a look if
you're busy.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170320/99ba8078/attachment.bin>

From laforge at gnumonks.org  Mon Mar 20 05:51:20 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Mon, 20 Mar 2017 06:51:20 +0100
Subject: nightly packages failures
In-Reply-To: <20170318114920.GA3517@my.box>
References: <20170318114920.GA3517@my.box>
Message-ID: <20170320055120.r6keyzolvb4vz5gz@nataraja>

On Sat, Mar 18, 2017 at 12:49:20PM +0100, Neels Hofmeyr wrote:
> openggsn failed because README was renamed to README.md without
> adjusting the debian installation rules (possibly the same change
> occured in other repositories, Harald?)

The same change occurred in other repositories, but it seems OpenGGSN is
the only repo where the README file was explicitly referred-to by the
Debian package rules.  I fixed this over the weekend and the build seems
to have recovered.

We still have plenty of build failures on i586 recently, but those are
unrelated to my README related changes...

> the osmo-auc-gen_test.sh still fails on 32bit systems, to be fixed by
> https://gerrit.osmocom.org/2089 (please review)

reviewed+merged now.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From msuraev at sysmocom.de  Mon Mar 20 11:37:36 2017
From: msuraev at sysmocom.de (Max)
Date: Mon, 20 Mar 2017 12:37:36 +0100
Subject: test failures in current OpenBSC
Message-ID: <5a89476c-d018-58f7-ab95-d568fa39f28a@sysmocom.de>

Hi.

A little heads-up - the current master of OpenBSC is failing at 'make 
check' stage - see 
http://jenkins.osmocom.org/jenkins/job/OpenBSC-gerrit/2019/IU=--disable-iu,MGCP=--disable-mgcp-transcoding,SMPP=--disable-smpp,label=linux_amd64_debian8/console 
for example.

Until this is fixed, any patches for OpenBSC won't be accepted by 
jenkins. If you hit this - please retrigger corresponding failed build 
in jenkins once it's fixed.

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From nhofmeyr at sysmocom.de  Mon Mar 20 12:23:48 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 20 Mar 2017 13:23:48 +0100
Subject: test failures in current OpenBSC
In-Reply-To: <5a89476c-d018-58f7-ab95-d568fa39f28a@sysmocom.de>
References: <5a89476c-d018-58f7-ab95-d568fa39f28a@sysmocom.de>
Message-ID: <20170320122348.GA2052@my.box>

On Mon, Mar 20, 2017 at 12:37:36PM +0100, Max wrote:
> the current master of OpenBSC is failing at 'make check'

That failure has actually been fixed 5 days ago, the gerrit job only sees it
because the corresponding patch is based on an outdated openbsc master.

> stage - see http://jenkins.osmocom.org/jenkins/job/OpenBSC-gerrit/2019/IU=--disable-iu,MGCP=--disable-mgcp-transcoding,SMPP=--disable-smpp,label=linux_amd64_debian8/console

The one-stop shop for asserting master branch status is
http://jenkins.osmocom.org/jenkins/job/OpenBSC/
Best always look there to confirm before heads-upping.

Currently that's red because of a 'broken pipe' related sporadic error, which
we can't seem to get rid of >:(
So you also need to look whether the entire matrix failed or just one of them.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170320/575a2a49/attachment.bin>

From holger at freyther.de  Wed Mar 22 07:11:18 2017
From: holger at freyther.de (Holger Freyther)
Date: Wed, 22 Mar 2017 08:11:18 +0100
Subject: gerrit openid login: osmocom.org vs. projects.osmocom.org
In-Reply-To: <20170320043052.GB12441@my.box>
References: <20170320043052.GB12441@my.box>
Message-ID: <B61B008A-FB1F-4027-8A35-BB44425ECEB9@freyther.de>


> On 20 Mar 2017, at 05:30, Neels Hofmeyr <nhofmeyr at sysmocom.de> wrote:
> 
> Has someone made osmocom.org login redirect to projects.osmocom.org?
> Because since recently I observe this:
> 
> 
> I go to osmocom.org, click on "Sign in".
> It *redirects* me to projects.osmocom.org/login.
> I log in.


hmm... I did this https://osmocom.org/issues/1728 could you try with curl -v?


> If redirecting to projects.o.o is intentional and the way to go (TM), I should
> probably pre-empt problems for existing users by creating external ids with
> 'projects' in the openid url, pointing at the proper existing users.
> 
> Otherwise we should avoid magical forwarding of osmocom.org logins to
> projects.osmocom.org.

No. that is not intended. I tried to block opening /login as http though. So maybe $server_name in nginx doesn't work as it should(tm).

$ curl -v http://osmocom.org/login/ 2>&1 | grep Location
< Location: https://projects.osmocom.org/login/

I changed $server_name to $host. It looks better?

holger

From nhofmeyr at sysmocom.de  Wed Mar 22 11:05:56 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Wed, 22 Mar 2017 12:05:56 +0100
Subject: gerrit openid login: osmocom.org vs. projects.osmocom.org
In-Reply-To: <B61B008A-FB1F-4027-8A35-BB44425ECEB9@freyther.de>
References: <20170320043052.GB12441@my.box>
 <B61B008A-FB1F-4027-8A35-BB44425ECEB9@freyther.de>
Message-ID: <20170322110556.GA1628@ass40.sysmocom.de>

On Wed, Mar 22, 2017 at 08:11:18AM +0100, Holger Freyther wrote:
> I changed $server_name to $host. It looks better?

Ah, I noticed a detail:

httpS://osmocom.org/login doesn't redirect to projects,

but httP://osmocom.org/login still does to httpS://PROJECTS.osmocom.org/login

When I pull up osmocom.org I end up at httP, then clicking 'Sign in'
goes to httpS projects.

So technically this could be fixed for me with the https-everywhere
browser plugin, which I have active, but seems to have outdated/incomplete
osmocom.org rules :)

I also have access to the osmocom nginx ... I'm probably not very adept at
it, so don't mind at all if you do it, just let me know if I should try to
fix it myself instead; now that I know the intention.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170322/675918b7/attachment.bin>

From jsteiger at sysmocom.de  Wed Mar 22 17:09:58 2017
From: jsteiger at sysmocom.de (Joachim Steiger)
Date: Wed, 22 Mar 2017 18:09:58 +0100
Subject: weekly test report (w11 2017)
Message-ID: <cf9806b3-d397-c48c-8856-c93fe383aabc@sysmocom.de>

Hi,

this is the BTS test report for week11 2017.

BTS models which have been tested with OsmoNITB are:

 - sysmoBTS1002
 - octasic OCTBTS3500
 - ettus USRP B200
 - nanoBTS model:165G

All test cases passed except for one for one config of the octasic bts:
3 TCH/H, 3 PDCH
in this config the gprs did not work as expected. sorry no logs this time.

For additional information related to test cases, timeslots
configuration and voice codecs used, please visit:

https://osmocom.org/projects/cellular-infrastructure/wiki/Weekly_BTS_Tests

if you are interested in the details, drop me a mail and i'll forward
the whole report.

kind regards

-- 
- Joachim Steiger <jsteiger at sysmocom.de>      http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Harald Welte


From osmocom at alteholz.de  Wed Mar 22 21:21:19 2017
From: osmocom at alteholz.de (Thorsten Alteholz)
Date: Wed, 22 Mar 2017 22:21:19 +0100 (CET)
Subject: python on FreeBSD in libosmocore
Message-ID: <alpine.DEB.2.02.1703222214290.418@jupiter.server.alteholz.net>

Hi,

as python2 will disappear sooner or later, I tried to replace all python2 
references in libosmocore by the generic "python" [1].
Unfortunately the build fails on FreeBSD[2]. Google didn't want to explain 
me why there might be /usr/bin/python2 but no /usr/bin/python, so do you 
have any idea what is wrong here?

   Thorsten


[1] https://gerrit.osmocom.org/#/c/2158/
[2] http://jenkins.osmocom.org/jenkins/job/libosmocore-gerrit/903/label=FreeBSD_amd64/



From gerrit-no-reply at lists.osmocom.org  Thu Mar 23 07:29:56 2017
From: gerrit-no-reply at lists.osmocom.org (Holger Freyther)
Date: Thu, 23 Mar 2017 07:29:56 +0000
Subject: osmo-pcap[master]: debian: Make a new release with the new feature
In-Reply-To: <gerrit.1488925126540.Ibe86b761b494e0fb78bbbc78e3c1982e44185750@gerrit.osmocom.org>
References: <gerrit.1488925126540.Ibe86b761b494e0fb78bbbc78e3c1982e44185750@gerrit.osmocom.org>
Message-ID: <mailman.521.1641744211.42898.openbsc@lists.osmocom.org>


Patch Set 1: Code-Review+2

-- 
To view, visit https://gerrit.osmocom.org/1999
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibe86b761b494e0fb78bbbc78e3c1982e44185750
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Max <msuraev at sysmocom.de>
Gerrit-HasComments: No


From gerrit-no-reply at lists.osmocom.org  Thu Mar 23 07:29:58 2017
From: gerrit-no-reply at lists.osmocom.org (Holger Freyther)
Date: Thu, 23 Mar 2017 07:29:58 +0000
Subject: [MERGED] osmo-pcap[master]: debian: Make a new release with the new
 feature
In-Reply-To: <gerrit.1488925126540.Ibe86b761b494e0fb78bbbc78e3c1982e44185750@gerrit.osmocom.org>
References: <gerrit.1488925126540.Ibe86b761b494e0fb78bbbc78e3c1982e44185750@gerrit.osmocom.org>
Message-ID: <mailman.522.1641744211.42898.openbsc@lists.osmocom.org>

Holger Freyther has submitted this change and it was merged.

Change subject: debian: Make a new release with the new feature
......................................................................


debian: Make a new release with the new feature

Change-Id: Ibe86b761b494e0fb78bbbc78e3c1982e44185750
---
M debian/changelog
1 file changed, 6 insertions(+), 0 deletions(-)

Approvals:
  Max: Looks good to me, but someone else must approve
  Jenkins Builder: Verified
  Holger Freyther: Looks good to me, approved



diff --git a/debian/changelog b/debian/changelog
index fd4259c..6e4df55 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+osmo-pcap (0.0.11) UNRELEASED; urgency=medium
+
+  * Add "source ip A.B.C.D" option to use specific address.
+
+ -- Holger Hans Peter Freyther <holger at moiji-mobile.com>  Tue, 17 Jan 2017 09:12:52 +0100
+
 osmo-pcap (0.0.10) unstable; urgency=medium
 
   * New release with new features

-- 
To view, visit https://gerrit.osmocom.org/1999
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ibe86b761b494e0fb78bbbc78e3c1982e44185750
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Max <msuraev at sysmocom.de>


From gerrit-no-reply at lists.osmocom.org  Thu Mar 23 07:30:03 2017
From: gerrit-no-reply at lists.osmocom.org (Holger Freyther)
Date: Thu, 23 Mar 2017 07:30:03 +0000
Subject: osmo-pcap[master]: debian: Add -dbg packages for the osmo-pcap-client
 and osmo-...
In-Reply-To: <gerrit.1488925126540.I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4@gerrit.osmocom.org>
References: <gerrit.1488925126540.I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4@gerrit.osmocom.org>
Message-ID: <mailman.523.1641744211.42898.openbsc@lists.osmocom.org>


Patch Set 1: Code-Review+2

-- 
To view, visit https://gerrit.osmocom.org/2000
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Max <msuraev at sysmocom.de>
Gerrit-HasComments: No


From gerrit-no-reply at lists.osmocom.org  Thu Mar 23 07:30:05 2017
From: gerrit-no-reply at lists.osmocom.org (Holger Freyther)
Date: Thu, 23 Mar 2017 07:30:05 +0000
Subject: [MERGED] osmo-pcap[master]: debian: Add -dbg packages for the
 osmo-pcap-client and osmo-...
In-Reply-To: <gerrit.1488925126540.I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4@gerrit.osmocom.org>
References: <gerrit.1488925126540.I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4@gerrit.osmocom.org>
Message-ID: <mailman.524.1641744211.42898.openbsc@lists.osmocom.org>

Holger Freyther has submitted this change and it was merged.

Change subject: debian: Add -dbg packages for the osmo-pcap-client and osmo-pcap-server
......................................................................


debian: Add -dbg packages for the osmo-pcap-client and osmo-pcap-server

Currently looking at a weird issue. Make it possible to install the
-dbg packages.

Change-Id: I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4
---
M debian/changelog
M debian/control
M debian/rules
3 files changed, 15 insertions(+), 0 deletions(-)

Approvals:
  Max: Looks good to me, but someone else must approve
  Jenkins Builder: Verified
  Holger Freyther: Looks good to me, approved



diff --git a/debian/changelog b/debian/changelog
index 6e4df55..13bce30 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,6 +1,7 @@
 osmo-pcap (0.0.11) UNRELEASED; urgency=medium
 
   * Add "source ip A.B.C.D" option to use specific address.
+  * Add osmo-pcap-client-dbg/osmo-pcap-server-dbg package
 
  -- Holger Hans Peter Freyther <holger at moiji-mobile.com>  Tue, 17 Jan 2017 09:12:52 +0100
 
diff --git a/debian/control b/debian/control
index 3364e34..2677bc0 100644
--- a/debian/control
+++ b/debian/control
@@ -17,3 +17,13 @@
 Architecture: any
 Depends: ${shlibs:Depends}, ${misc:Depends}
 Description: Collect traces from other systems.
+
+Package: osmo-pcap-client-dbg
+Architecture: any
+Depends: osmo-pcap-client (= ${binary:Version})
+Description: Debug symbols of osmo-pcap-client-dbg
+
+Package: osmo-pcap-server-dbg
+Architecture: any
+Depends: osmo-pcap-server (= ${binary:Version})
+Description: Debug symbols of osmo-pcap-server-dbg
diff --git a/debian/rules b/debian/rules
index 78ddc43..872097c 100755
--- a/debian/rules
+++ b/debian/rules
@@ -39,3 +39,7 @@
 
 	install -d -m 0755 $(CURDIR)/debian/osmo-pcap-server/etc/cron.daily/
 	install -m 0755 $(CURDIR)/contrib/osmo_pcap_clean_old $(CURDIR)/debian/osmo-pcap-server/etc/cron.daily/
+
+override_dh_strip:
+	dh_strip -posmo-pcap-client --dbg-package=osmo-pcap-client-dbg
+	dh_strip -posmo-pcap-server --dbg-package=osmo-pcap-server-dbg

-- 
To view, visit https://gerrit.osmocom.org/2000
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7d6c8e491be459151c1531b86f28bb1dc2ee8bb4
Gerrit-PatchSet: 1
Gerrit-Project: osmo-pcap
Gerrit-Branch: master
Gerrit-Owner: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Max <msuraev at sysmocom.de>


From laforge at gnumonks.org  Thu Mar 23 09:18:49 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 23 Mar 2017 10:18:49 +0100
Subject: python on FreeBSD in libosmocore
In-Reply-To: <alpine.DEB.2.02.1703222214290.418@jupiter.server.alteholz.net>
References: <alpine.DEB.2.02.1703222214290.418@jupiter.server.alteholz.net>
Message-ID: <20170323091849.rsykibpimoulw2f2@nataraja>

Hi,

On Wed, Mar 22, 2017 at 10:21:19PM +0100, Thorsten Alteholz wrote:

> as python2 will disappear sooner or later, I tried to replace all python2
> references in libosmocore by the generic "python" [1].
> Unfortunately the build fails on FreeBSD[2]. Google didn't want to explain
> me why there might be /usr/bin/python2 but no /usr/bin/python, so do you
> have any idea what is wrong here?

I don't know about FreeBSD policy on this, but those FreeBSD systems
that I work with alwys "only" had a python2 and/or python3 executable
and not a genric "python" one, which breaks a lot of scripts and
requires manual editing :/

I guess it's worth asking some FreeBSD list/forum about this, if it's
not a FAQ.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Thu Mar 23 11:29:52 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 23 Mar 2017 12:29:52 +0100
Subject: weekly test report (w11 2017)
In-Reply-To: <cf9806b3-d397-c48c-8856-c93fe383aabc@sysmocom.de>
References: <cf9806b3-d397-c48c-8856-c93fe383aabc@sysmocom.de>
Message-ID: <20170323112952.GA2489@my.box>

On Wed, Mar 22, 2017 at 06:09:58PM +0100, Joachim Steiger wrote:
> All test cases passed except for one for one config of the octasic bts:
> 3 TCH/H, 3 PDCH

Could you create a ticket for it and post the link here, thanks!
Would also be good to produce logs for it if you can spare the time.
(or next time is probably fine)

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170323/1912e005/attachment.bin>

From pmaier at sysmocom.de  Thu Mar 23 12:07:17 2017
From: pmaier at sysmocom.de (Philipp Maier)
Date: Thu, 23 Mar 2017 13:07:17 +0100
Subject: [PATCH 2/2] fix writing of ICCID for sysmo-usim-sjs1
In-Reply-To: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
Message-ID: <1490270837-28823-3-git-send-email-pmaier@sysmocom.de>

The programming procedure for sysmo-usim-sjs1 lacks
writing the ICCID. This commit adds the missing call
to update_binary()
---
 pySim/cards.py | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/pySim/cards.py b/pySim/cards.py
index fafc55f..925c5e6 100644
--- a/pySim/cards.py
+++ b/pySim/cards.py
@@ -434,20 +434,20 @@ class SysmoUSIMSJS1(Card):
 
 	def program(self, p):
 
+		# authenticate as ADM using default key (written on the card..)
+		if not p['pin_adm']:
+			raise ValueError("Please provide a PIN-ADM as there is no default one")
+		self._scc.verify_chv(0x0A, h2b(p['pin_adm']))
 
 		# select MF
 		r = self._scc.select_file(['3f00'])
 
+		# write EF.ICCID
+		data, sw = self._scc.update_binary('2fe2', enc_iccid(p['iccid']))
+
 		# select DF_GSM
 		r = self._scc.select_file(['7f20'])
 
-		# authenticate as ADM using default key (written on the card..)
-		if not p['pin_adm']:
-			raise ValueError("Please provide a PIN-ADM as there is no default one")
-
-		self._scc.verify_chv(0x0A, h2b(p['pin_adm']))
-
-
 		# set Ki in proprietary file
 		data, sw = self._scc.update_binary('00FF', p['ki'])
 
-- 
1.9.1



From pmaier at sysmocom.de  Thu Mar 23 12:07:16 2017
From: pmaier at sysmocom.de (Philipp Maier)
Date: Thu, 23 Mar 2017 13:07:16 +0100
Subject: [PATCH 1/2] Fix select control parameter
In-Reply-To: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
Message-ID: <1490270837-28823-2-git-send-email-pmaier@sysmocom.de>

sysmo-usim-sjs1 requires P2 to be set to 0x0C (request FCI) when
using the USIM application commands. The FCI is not used by pysim
anyway and might even cause problems with other cards.

This commit adds a pair of get/set methods to the SimCardCommands
class in order to set a default for the selection control
parameters (P1, P2). (Similar to the set/get methods for the class
byte)

The SysmoUSIMSJS1 class now calls the setter method for the
selection control parameters inside of its constructuor and sets
the selection control parameter default to "000C". This way we
can be sure that we only change the behaviour for sysmo-usim-sjs1
and do not break support for any other cards.
---
 pySim/cards.py    |  1 +
 pySim/commands.py | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/pySim/cards.py b/pySim/cards.py
index 23352a7..fafc55f 100644
--- a/pySim/cards.py
+++ b/pySim/cards.py
@@ -425,6 +425,7 @@ class SysmoUSIMSJS1(Card):
 	def __init__(self, ssc):
 		super(SysmoUSIMSJS1, self).__init__(ssc)
 		self._scc.cla_byte = "00"
+		self._scc.sel_ctrl = "000C"
 
 	@classmethod
 	def autodetect(kls, scc):
diff --git a/pySim/commands.py b/pySim/commands.py
index cb72a11..d8bd8f2 100644
--- a/pySim/commands.py
+++ b/pySim/commands.py
@@ -29,6 +29,7 @@ class SimCardCommands(object):
 	def __init__(self, transport):
 		self._tp = transport;
 		self._cla_byte = "a0"
+		self._sel_ctrl = "0000"
 
 	@property
 	def cla_byte(self):
@@ -37,11 +38,17 @@ class SimCardCommands(object):
 	def cla_byte(self, value):
 		self._cla_byte = value
 
+	@property
+	def sel_ctrl(self):
+		return self._sel_ctrl
+	@sel_ctrl.setter
+	def sel_ctrl(self, value):
+		self._sel_ctrl = value
 
 	def select_file(self, dir_list):
 		rv = []
 		for i in dir_list:
-			data, sw = self._tp.send_apdu_checksw(self.cla_byte + "a4000002" + i)
+			data, sw = self._tp.send_apdu_checksw(self.cla_byte + "a4" + self._sel_ctrl + "02" + i)
 			rv.append(data)
 		return rv
 
@@ -101,4 +108,4 @@ class SimCardCommands(object):
 
 	def verify_chv(self, chv_no, code):
 		fc = rpad(b2h(code), 16)
-		return self._tp.send_apdu_checksw(self.cla_byte + '2000' + ('%02X' % chv_no) + '08' + fc)
\ No newline at end of file
+		return self._tp.send_apdu_checksw(self.cla_byte + '2000' + ('%02X' % chv_no) + '08' + fc)
-- 
1.9.1



From pmaier at sysmocom.de  Thu Mar 23 12:07:15 2017
From: pmaier at sysmocom.de (Philipp Maier)
Date: Thu, 23 Mar 2017 13:07:15 +0100
Subject: [PATCH 0/2] fixes for pysim
Message-ID: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>

Hello,

I would like to propose the two changes for pysim, the first is for fixing
the sysmo-usim-sjs1 support and the second fixes the missing support for
writing the ICCID to a sysmo-usim-sjs1.

If there are no major concerns with that I would like to merge the two
patches to master.

Repo: https://git.osmocom.org/pysim
Branch: pmaier/fixfci

regards,
Philipp Maier

Philipp Maier (2):
  Fix select control parameter
  fix writing of ICCID for sysmo-usim-sjs1

 pySim/cards.py    | 15 ++++++++-------
 pySim/commands.py | 11 +++++++++--
 2 files changed, 17 insertions(+), 9 deletions(-)

-- 
1.9.1



From msuraev at sysmocom.de  Thu Mar 23 12:10:04 2017
From: msuraev at sysmocom.de (Max)
Date: Thu, 23 Mar 2017 13:10:04 +0100
Subject: [PATCH 0/2] fixes for pysim
In-Reply-To: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
Message-ID: <6fb32df0-6849-c7f6-ab19-1dce5d02c0e6@sysmocom.de>

On a related note: should we add pysim to 
https://gerrit.osmocom.org/#/admin/projects/ ?


On 23.03.2017 13:07, Philipp Maier wrote:
> Hello,
>
> I would like to propose the two changes for pysim, the first is for fixing
> the sysmo-usim-sjs1 support and the second fixes the missing support for
> writing the ICCID to a sysmo-usim-sjs1.
>
> If there are no major concerns with that I would like to merge the two
> patches to master.
>
> Repo: https://git.osmocom.org/pysim
> Branch: pmaier/fixfci
>
> regards,
> Philipp Maier
>
> Philipp Maier (2):
>    Fix select control parameter
>    fix writing of ICCID for sysmo-usim-sjs1
>
>   pySim/cards.py    | 15 ++++++++-------
>   pySim/commands.py | 11 +++++++++--
>   2 files changed, 17 insertions(+), 9 deletions(-)
>

-- 
Max Suraev <msuraev at sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From pmaier at sysmocom.de  Thu Mar 23 14:27:23 2017
From: pmaier at sysmocom.de (Philipp Maier)
Date: Thu, 23 Mar 2017 15:27:23 +0100
Subject: runtime cpu-detection for osmo-trx
Message-ID: <58D3DB4B.1090105@sysmocom.de>

Dear list members, dear Mr. Tsou,

we had a couple of problems with the SSE support in Transceiver52 of 
osmo-trx. The problem was that the -march=native, that is used when 
compiling the x86 code in Transceiver52M/x86/. This would create a 
platform dependent binary that can not be easily moved to another 
platform. (e.g. binary is compiled on a machine that does support 
SSE4.1, but used on a machine that only supports SSE3)

In order to solve the problem, we have added logic to detect the CPU 
type at runtime and to switch between the base implementation 
(Transceiver52M/common) and between the platform specific implementation.

Since there might be different compilers out there, which do not support 
the -msse3 and -msse4.1 options, we also modified the build system to 
detect if the compiler supports -msse3 and -msse4.1. If no support can 
be detected, there is a hard fallback to the generic implementation.

The patches are currently in the review process, you can find them below:
https://gerrit.osmocom.org/2098 buildenv: Turn off native architecture 
builds
https://gerrit.osmocom.org/2099 cosmetic: Make parameter lists uniform
https://gerrit.osmocom.org/2100 ssedetect: Add runtime CPU detection
https://gerrit.osmocom.org/2101 cosmetic: Add info about SSE support
https://gerrit.osmocom.org/2102 buildenv: Make build CPU invariant
https://gerrit.osmocom.org/2103 cosmetic: remove code duplication
https://gerrit.osmocom.org/2104 Add test program to verify convolution 
implementation
https://gerrit.osmocom.org/2134 buildenv: Split up SSE3 and SSE4.1 code

There is also a readmine ticket concerning this change:
https://osmocom.org/issues/1869

It would be very kind if you could have a look at the changes and, if 
possible, to approve them.

regards,
Philipp Maier

-- 
Philipp Maier <pmaier at sysmocom.de>              http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



From holger at freyther.de  Thu Mar 23 15:10:21 2017
From: holger at freyther.de (Holger Freyther)
Date: Thu, 23 Mar 2017 16:10:21 +0100
Subject: gerrit openid login: osmocom.org vs. projects.osmocom.org
In-Reply-To: <20170322110556.GA1628@ass40.sysmocom.de>
References: <20170320043052.GB12441@my.box>
 <B61B008A-FB1F-4027-8A35-BB44425ECEB9@freyther.de>
 <20170322110556.GA1628@ass40.sysmocom.de>
Message-ID: <0E715C08-6AAE-4DF3-9A27-39644E5FD395@freyther.de>


> On 22 Mar 2017, at 12:05, Neels Hofmeyr <nhofmeyr at sysmocom.de> wrote:
> 
> but httP://osmocom.org/login still does to httpS://PROJECTS.osmocom.org/login

curl -v httP://osmocom.org/login 2>&1 | grep Loca
< Location: https://osmocom.org/login

=> your browser has the redirect cached?

holger

From jsteiger at sysmocom.de  Thu Mar 23 15:19:22 2017
From: jsteiger at sysmocom.de (Joachim Steiger)
Date: Thu, 23 Mar 2017 16:19:22 +0100
Subject: weekly test report (w11 2017)
In-Reply-To: <20170323112952.GA2489@my.box>
References: <cf9806b3-d397-c48c-8856-c93fe383aabc@sysmocom.de>
 <20170323112952.GA2489@my.box>
Message-ID: <4518f59c-61d7-7c60-5264-9f3e2a31644e@sysmocom.de>

On 03/23/2017 12:29 PM, Neels Hofmeyr wrote:
> Could you create a ticket for it and post the link here, thanks!
> Would also be good to produce logs for it if you can spare the time.
> (or next time is probably fine)

will do.

since i could not reproduce the issue anymore i haven't created a
ticket. i guess it was some problem in my testsetup.

kind regards

-- 
- Joachim Steiger <jsteiger at sysmocom.de>      http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Harald Welte


From jsteiger at sysmocom.de  Thu Mar 23 15:25:07 2017
From: jsteiger at sysmocom.de (Joachim Steiger)
Date: Thu, 23 Mar 2017 16:25:07 +0100
Subject: weekly test report (w12 2017)
Message-ID: <e6a24d12-8273-71e1-a794-b54cce95ce80@sysmocom.de>

Hi,

this is the BTS test report for week12 2017.

BTS models which have been tested with OsmoNITB are:

 - sysmoBTS1002
 - octasic OCTBTS3500
 - ettus USRP B200
 - nanoBTS model:165G

All test cases have passed.

For additional information related to test cases, timeslots
configuration and voice codecs used, please visit:

https://osmocom.org/projects/cellular-infrastructure/wiki/Weekly_BTS_Tests

following are some git hashes/version strings of tested components:

libosmo-abis:       1e04d4b7a2c845589da20d891cc5e3c471cfe983
libosmocore:        62d6f2570358730965162f7dd756a6e2d07627b2
libosmo-netif:      3a060c59bec7a4a5b22849938b8b4c7b7ecb4c01
libosmo-sccp:       8e708d1f2da1b187f631bf08172a5194a85b1a23
libsmpp34:          cc0bcd6bc051d5ccaf32cdbbc28f073369900857
openbsc:            dd22a30d75090ebe2ba08056bfa04a92aa8d6ba1
openggsn:           19e19e3609508d121ba46c165e5ed1502a3cf9da
osmo-bts:           b609ee879eb798f8f3836223b4702745f3f6491e
osmo-bts-octphy:    b609ee879eb798f8f3836223b4702745f3f6491e
osmo-pcu:           0a8fae8d141c2cfa4387ffe9b35402d5b8cc85cd
osmo-trx:           14d13b67dcd4fa35b03cbbef0c5ddd2622b89155
sysmobts:	OsmoBTS version 0.4.0.410-b609
sysmobts:	Osmo-PCU version 0.2.896-0a8f
octbts:		(name='octsdr_gsm', desc='Software Define Radio',
ver='02.07.00-B1039', ver_hdr='02.07.00-B1039')
octbts:		(platform='Opus2', version='OCTSYS_VERSION=01.02.19.B1
OCTODK_VERSION=01.15.01-B1;OCTADF_VERSION=04.05.01-B2637;')
nanobts:	Equipment_Version='165g029_79'
nanobts:	Software_Version='168d462_v200b202d0'

if you are interested in the details, drop me a mail and i'll forward
the whole report.

kind regards

-- 
- Joachim Steiger <jsteiger at sysmocom.de>      http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Harald Welte



From tom at tsou.cc  Thu Mar 23 21:42:23 2017
From: tom at tsou.cc (Tom Tsou)
Date: Thu, 23 Mar 2017 14:42:23 -0700
Subject: runtime cpu-detection for osmo-trx
In-Reply-To: <58D3DB4B.1090105@sysmocom.de>
References: <58D3DB4B.1090105@sysmocom.de>
Message-ID: <CAATyM+YACZvOiA0pzPBxUhQ9E-7LD4QHxFU_gh8d2i2XKJUp9w@mail.gmail.com>

Hi Philipp,

On Thu, Mar 23, 2017 at 7:27 AM, Philipp Maier <pmaier at sysmocom.de> wrote:
> we had a couple of problems with the SSE support in Transceiver52 of
> osmo-trx. The problem was that the -march=native, that is used when
> compiling the x86 code in Transceiver52M/x86/. This would create a platform
> dependent binary that can not be easily moved to another platform. (e.g.
> binary is compiled on a machine that does support SSE4.1, but used on a
> machine that only supports SSE3)
>
> In order to solve the problem, we have added logic to detect the CPU type at
> runtime and to switch between the base implementation
> (Transceiver52M/common) and between the platform specific implementation.

Thank you for the contributions. I have been travelling recently and
am catching up with a number of recent osmo-trx patch submissions. I
am aware of the SSE concerns and will try to review and merge the
patches in a timely manner.

  -TT


From holger at freyther.de  Thu Mar 23 23:43:25 2017
From: holger at freyther.de (Holger Freyther)
Date: Fri, 24 Mar 2017 00:43:25 +0100
Subject: python on FreeBSD in libosmocore
In-Reply-To: <20170323091849.rsykibpimoulw2f2@nataraja>
References: <alpine.DEB.2.02.1703222214290.418@jupiter.server.alteholz.net>
 <20170323091849.rsykibpimoulw2f2@nataraja>
Message-ID: <A557C996-C6E2-4251-87B0-AD732E968A3E@freyther.de>


> On 23 Mar 2017, at 10:18, Harald Welte <laforge at gnumonks.org> wrote:
> 
> Hi,
> 
> On Wed, Mar 22, 2017 at 10:21:19PM +0100, Thorsten Alteholz wrote:
> 
>> as python2 will disappear sooner or later, I tried to replace all python2
>> references in libosmocore by the generic "python" [1].
>> Unfortunately the build fails on FreeBSD[2]. Google didn't want to explain
>> me why there might be /usr/bin/python2 but no /usr/bin/python, so do you
>> have any idea what is wrong here?
> 
> I don't know about FreeBSD policy on this, but those FreeBSD systems
> that I work with alwys "only" had a python2 and/or python3 executable
> and not a genric "python" one, which breaks a lot of scripts and
> requires manual editing :/
> 
> I guess it's worth asking some FreeBSD list/forum about this, if it's
> not a FAQ.

We can install the "python" package that adds a symlink from python2
to python. My main issues with this approach is:

* Some of our scripts don't work with python3
* Python2.7 will be around for a long time (and was recently forked)

So when the build breakage came up I preferred to be honest and say it
is a python2 script.

holger


From ereedsanchez at gmail.com  Fri Mar 24 14:49:42 2017
From: ereedsanchez at gmail.com (Edwin Reed-Sanchez)
Date: Fri, 24 Mar 2017 10:49:42 -0400
Subject: GSM Audio Problems and
Message-ID: <CAJCTxpNBkdPU=yCq-jJ97tLOftcGfWsowPyy2h3=fXRHo2Q2Sg@mail.gmail.com>

Dear Osmocom list

We are getting several audio problems and dropped calls.
The audio sounds like static.  We are also getting large frame correction
in the logs, and SACCH deactivation timeout.

Any ideas what could be the likely culprit?


2017-03-24_14:30:57.93546 <001a> rtp_proxy.c:295 Correcting frame
difference of 74518293571 frames
2017-03-24_14:31:11.45899 <001a> rtp_proxy.c:295 Correcting frame
difference of 1 frames
2017-03-24_14:31:11.51590 <001a> rtp_proxy.c:295 Correcting frame
difference of 2 frames
2017-03-24_14:31:11.65513 <001a> rtp_proxy.c:295 Correcting frame
difference of 1 frames

2017-03-24_14:32:08.95028 <0004> abis_rsl.c:1343 (bts=1,trx=0,ts=1,ss=0)
SACCH deactivation timeout.
2017-03-24_14:37:10.11844 <0004> abis_rsl.c:662 (bts=1,trx=0,ts=1,ss=0) is
back in operation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170324/2e5cfc65/attachment.htm>

From holger at freyther.de  Fri Mar 24 14:52:21 2017
From: holger at freyther.de (Holger Freyther)
Date: Fri, 24 Mar 2017 15:52:21 +0100
Subject: GSM Audio Problems and
In-Reply-To: <CAJCTxpNBkdPU=yCq-jJ97tLOftcGfWsowPyy2h3=fXRHo2Q2Sg@mail.gmail.com>
References: <CAJCTxpNBkdPU=yCq-jJ97tLOftcGfWsowPyy2h3=fXRHo2Q2Sg@mail.gmail.com>
Message-ID: <A2E5B08E-3DB5-4600-A21F-12584316DFA4@freyther.de>


> On 24 Mar 2017, at 15:49, Edwin Reed-Sanchez <ereedsanchez at gmail.com> wrote:
> 
> Dear Osmocom list 
> 
> We are getting several audio problems and dropped calls.
> The audio sounds like static.  We are also getting large frame correction in the logs, and SACCH deactivation timeout.
> 
> Any ideas what could be the likely culprit?  

classic radio interference?


> 2017-03-24_14:30:57.93546 <001a> rtp_proxy.c:295 Correcting frame difference of 74518293571 frames

looks like a change of SSRC? What does RTP look like?



> 2017-03-24_14:31:11.45899 <001a> rtp_proxy.c:295 Correcting frame difference of 1 frames
> 2017-03-24_14:31:11.51590 <001a> rtp_proxy.c:295 Correcting frame difference of 2 frames
> 2017-03-24_14:31:11.65513 <001a> rtp_proxy.c:295 Correcting frame difference of 1 frames
> 
> 2017-03-24_14:32:08.95028 <0004> abis_rsl.c:1343 (bts=1,trx=0,ts=1,ss=0) SACCH deactivation timeout.
> 2017-03-24_14:37:10.11844 <0004> abis_rsl.c:662 (bts=1,trx=0,ts=1,ss=0) is back in operation.



From ereedsanchez at gmail.com  Fri Mar 24 15:09:20 2017
From: ereedsanchez at gmail.com (Edwin Reed-Sanchez)
Date: Fri, 24 Mar 2017 11:09:20 -0400
Subject: GSM Audio Problems and
In-Reply-To: <A2E5B08E-3DB5-4600-A21F-12584316DFA4@freyther.de>
References: <CAJCTxpNBkdPU=yCq-jJ97tLOftcGfWsowPyy2h3=fXRHo2Q2Sg@mail.gmail.com>
 <A2E5B08E-3DB5-4600-A21F-12584316DFA4@freyther.de>
Message-ID: <CAJCTxpMnwfdzt1GLJQB3dd9ZhfiPx1vGyLxHtf4nXdnASmS0Tg@mail.gmail.com>

Holger,

classic radio interference?

- could this be caused by having amplifiers to high, and being close to
another BTS?  Should I attenuate the amplification in Osmotrx?

looks like a change of SSRC? What does RTP look like?

- where do I gather information for RTP.  Is it in the logs, or in the
configs.  We currently are not using
rtp jitter-buffer 0
in the osmobts config.

We are also getting radio link failures.

e2017-03-24_14:58:33.56803 <0004> abis_rsl.c:1035 (bts=1,trx=0,ts=1,ss=0)
CONNECTION FAIL: RELEASING state ACTIVE CAUSE=0x01(Radio Link Failure).

Will send you files in separate email.

Thanks

On Fri, Mar 24, 2017 at 10:52 AM, Holger Freyther <holger at freyther.de>
wrote:

>
> > On 24 Mar 2017, at 15:49, Edwin Reed-Sanchez <ereedsanchez at gmail.com>
> wrote:
> >
> > Dear Osmocom list
> >
> > We are getting several audio problems and dropped calls.
> > The audio sounds like static.  We are also getting large frame
> correction in the logs, and SACCH deactivation timeout.
> >
> > Any ideas what could be the likely culprit?
>
> classic radio interference?
>
>
> > 2017-03-24_14:30:57.93546 <001a> rtp_proxy.c:295 Correcting frame
> difference of 74518293571 frames
>
> looks like a change of SSRC? What does RTP look like?
>
>
>
> > 2017-03-24_14:31:11.45899 <001a> rtp_proxy.c:295 Correcting frame
> difference of 1 frames
> > 2017-03-24_14:31:11.51590 <001a> rtp_proxy.c:295 Correcting frame
> difference of 2 frames
> > 2017-03-24_14:31:11.65513 <001a> rtp_proxy.c:295 Correcting frame
> difference of 1 frames
> >
> > 2017-03-24_14:32:08.95028 <0004> abis_rsl.c:1343 (bts=1,trx=0,ts=1,ss=0)
> SACCH deactivation timeout.
> > 2017-03-24_14:37:10.11844 <0004> abis_rsl.c:662 (bts=1,trx=0,ts=1,ss=0)
> is back in operation.
>
>


-- 
Edwin Reed-Sanchez
__________________
itp.junglebrains.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170324/0f48731f/attachment.htm>

From holger at freyther.de  Sat Mar 25 18:08:49 2017
From: holger at freyther.de (Holger Freyther)
Date: Sat, 25 Mar 2017 19:08:49 +0100
Subject: GSM Audio Problems and
In-Reply-To: <CAJCTxpMnwfdzt1GLJQB3dd9ZhfiPx1vGyLxHtf4nXdnASmS0Tg@mail.gmail.com>
References: <CAJCTxpNBkdPU=yCq-jJ97tLOftcGfWsowPyy2h3=fXRHo2Q2Sg@mail.gmail.com>
 <A2E5B08E-3DB5-4600-A21F-12584316DFA4@freyther.de>
 <CAJCTxpMnwfdzt1GLJQB3dd9ZhfiPx1vGyLxHtf4nXdnASmS0Tg@mail.gmail.com>
Message-ID: <5346D680-A135-4FE7-9C87-02DCCF5ADBA1@freyther.de>


> On 24 Mar 2017, at 16:09, Edwin Reed-Sanchez <ereedsanchez at gmail.com> wrote:
> 
> Holger,
> 
> classic radio interference?
> 
> - could this be caused by having amplifiers to high, and being close to another BTS?  Should I attenuate the amplification in Osmotrx?

Difficult to say. You can look at the RSL measurement reports and
take a look if there is something obvious, e.g. rxlev/rxqual being
very bad



> 
> looks like a change of SSRC? What does RTP look like?
> 
> - where do I gather information for RTP.  Is it in the logs, or in the configs.  We currently are not using 
> rtp jitter-buffer 0  
> in the osmobts config. 


In Wireshark under Edit->Preferences->Protocol->RSL ("Use nanoBTS
defitnition"). Then you will see IPA CRCX/MDCX/DLCX to manage the
audio flow and Wireshark will be able to identify the RTP streams.

With Telephony->RTP->Show all Streams you can see how many streams
SRC ip/port, DST ip/port, SSRC exist. Then compare the timestamps
from the log with timestamp of the RTP packet. Is there another
stream/SSRC? Is this is just the first sequence number (which should
be random) and leads to a big jump?



holger

From nhofmeyr at sysmocom.de  Sun Mar 26 22:45:49 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 27 Mar 2017 00:45:49 +0200
Subject: gerrit openid login: osmocom.org vs. projects.osmocom.org
In-Reply-To: <0E715C08-6AAE-4DF3-9A27-39644E5FD395@freyther.de>
References: <20170320043052.GB12441@my.box>
 <B61B008A-FB1F-4027-8A35-BB44425ECEB9@freyther.de>
 <20170322110556.GA1628@ass40.sysmocom.de>
 <0E715C08-6AAE-4DF3-9A27-39644E5FD395@freyther.de>
Message-ID: <20170326224549.GB2441@my.box>

On Thu, Mar 23, 2017 at 04:10:21PM +0100, Holger Freyther wrote:
> 
> > On 22 Mar 2017, at 12:05, Neels Hofmeyr <nhofmeyr at sysmocom.de> wrote:
> > 
> > but httP://osmocom.org/login still does to httpS://PROJECTS.osmocom.org/login
> 
curl -v httP://osmocom.org/login 2>&1 | grep Loca < Location: https://osmocom.org/login
> 
> => your browser has the redirect cached?

Seems to have been the case, at least now I'm not redirected anymore.
Thanks!

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170327/4e9032c5/attachment.bin>

From nhofmeyr at sysmocom.de  Sun Mar 26 22:47:08 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 27 Mar 2017 00:47:08 +0200
Subject: [PATCH 0/2] fixes for pysim
In-Reply-To: <6fb32df0-6849-c7f6-ab19-1dce5d02c0e6@sysmocom.de>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
 <6fb32df0-6849-c7f6-ab19-1dce5d02c0e6@sysmocom.de>
Message-ID: <20170326224708.GC2441@my.box>

On Thu, Mar 23, 2017 at 01:10:04PM +0100, Max wrote:
> On a related note: should we add pysim to
> https://gerrit.osmocom.org/#/admin/projects/ ?

+1

I can do this if we have consensus. Anyone against it?

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170327/7be1ef79/attachment.bin>

From nhofmeyr at sysmocom.de  Sun Mar 26 22:49:13 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 27 Mar 2017 00:49:13 +0200
Subject: [PATCH 1/2] Fix select control parameter
In-Reply-To: <1490270837-28823-2-git-send-email-pmaier@sysmocom.de>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
 <1490270837-28823-2-git-send-email-pmaier@sysmocom.de>
Message-ID: <20170326224913.GD2441@my.box>

On Thu, Mar 23, 2017 at 01:07:16PM +0100, Philipp Maier wrote:
> -		return self._tp.send_apdu_checksw(self.cla_byte + '2000' + ('%02X' % chv_no) + '08' + fc)
> \ No newline at end of file
> +		return self._tp.send_apdu_checksw(self.cla_byte + '2000' + ('%02X' % chv_no) + '08' + fc)

This shows me that the patch is not based on the current master.
I fixed the missing newline in another patch.

...maybe let's wait a bit, if we're moving this to gerrit soon you won't need
to send another mail.

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170327/b343c5f1/attachment.bin>

From laforge at gnumonks.org  Mon Mar 27 08:27:12 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Mon, 27 Mar 2017 10:27:12 +0200
Subject: [PATCH 0/2] fixes for pysim
In-Reply-To: <20170326224708.GC2441@my.box>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
 <6fb32df0-6849-c7f6-ab19-1dce5d02c0e6@sysmocom.de>
 <20170326224708.GC2441@my.box>
Message-ID: <20170327082712.bkriijzphk3kclh5@nataraja>

Hi Neels,

On Mon, Mar 27, 2017 at 12:47:08AM +0200, Neels Hofmeyr wrote:
> On Thu, Mar 23, 2017 at 01:10:04PM +0100, Max wrote:
> > On a related note: should we add pysim to
> > https://gerrit.osmocom.org/#/admin/projects/ ?
> 
> I can do this if we have consensus. Anyone against it?

Thanks for the offer.  However, the "consensus" of so far two or three
people (who have very little involvement in pysim) doesn't weigh much in
context of the opinion of the original author and maintainer of the
software, which is in this case Sylvain.  

So it's primarily up to him to decide which tools he wants to use.

Please don't make any changes unless the "owner" of the respective
project has explicitly requested them.

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Mon Mar 27 12:20:02 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Mon, 27 Mar 2017 14:20:02 +0200
Subject: pysim on gerrit? -- was: [PATCH 0/2] fixes for pysim
In-Reply-To: <20170327082712.bkriijzphk3kclh5@nataraja>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
 <6fb32df0-6849-c7f6-ab19-1dce5d02c0e6@sysmocom.de>
 <20170326224708.GC2441@my.box>
 <20170327082712.bkriijzphk3kclh5@nataraja>
Message-ID: <20170327122002.GA1757@my.box>

On Mon, Mar 27, 2017 at 10:27:12AM +0200, Harald Welte wrote:
> > I can do this if we have consensus. Anyone against it?
> Please don't make any changes unless the "owner" of the respective
> project has explicitly requested them.

That's exactly what I meant by consensus, maybe the wrong word there...
tnt, what's your opinion on gerrit?

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170327/d42331df/attachment.bin>

From laforge at gnumonks.org  Mon Mar 27 13:14:10 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Mon, 27 Mar 2017 15:14:10 +0200
Subject: Updates on Osmocom Conference 2017
Message-ID: <20170327131410.rqtqkrqxqz2lwmwa@nataraja>

OsmoCon 2017 updates

There are some updates related to OsmoCon2017, the first Osmocom
Conference, held on April 21st, 2017 in Berlin, Germany.

See http://osmocom.org/news/68 for the web version of this announcement.

== Summary ==

Summary (for those too busy to read the full post):
* Schedule of talks has been released
  http://osmocom.org/projects/osmo-dev-con/wiki/OsmoCon2017#Schedule
* Travel Grants available for participants who are otherwise unable to
  travel to Berlin: http://osmocom.org/projects/osmo-dev-con/wiki/OsmoCon2017_TravelGrants
* Social Event details available, including menu:
  http://osmocom.org/projects/osmo-dev-con/wiki/OsmoCon2017_SocialEvent
* April 21st is approaching fast, make sure you get your Ticket in time.
  Limited number of seats available
  http://shop.sysmocom.de/products/ticket-for-osmocon-2017

== Details ==

=== Schedule has been released ===

The list of talks with their abstracts has been on the website for quite
some time, but now we actually have put together a schedule based on
those talks.

Please see
http://osmocom.org/projects/osmo-dev-con/wiki/OsmoCon2017#Schedule
for the schedule.

As you can see, the day is fully packed with talks about Osmocom
cellular infrastructure projects. We had to cut some talk slots short
(30min instead of 45min), but I'm confident that it is good to cover a
wider range of topics, while at the same time avoiding fragmenting the
audience with multiple tracks.

=== Travel Grants ===

We are happy to announce that we have received donations to permit for
providing travel grants!

This means that any attendee who is otherwise not able to cover their
travel to OsmoCon 2017 (e.g. because their interest in Osmocom is not
related to their work, or because their employer doesn't pay the travel
expenses) can now apply for such a travel grant.

For more details see
http://osmocom.org/projects/osmo-dev-con/wiki/OsmoCon2017_TravelGrants
and/or constact osmocon2017 at sysmocom.de.

=== Social Event ===

Tech Talks are nice and fine, but what many people enjoy even more at
conferences is the informal networking combined with good food. For
this, we have the social event at night, which is open to all attendees.

See more details about it at
http://osmocom.org/projects/osmo-dev-con/wiki/OsmoCon2017_SocialEvent


-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From 246tnt at gmail.com  Mon Mar 27 16:49:34 2017
From: 246tnt at gmail.com (Sylvain Munaut)
Date: Mon, 27 Mar 2017 18:49:34 +0200
Subject: pysim on gerrit? -- was: [PATCH 0/2] fixes for pysim
In-Reply-To: <20170327122002.GA1757@my.box>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
 <6fb32df0-6849-c7f6-ab19-1dce5d02c0e6@sysmocom.de>
 <20170326224708.GC2441@my.box>
 <20170327082712.bkriijzphk3kclh5@nataraja> <20170327122002.GA1757@my.box>
Message-ID: <CAHL+j0_yxs_xMvx+65yCuRg31akBiS3UFCLB7AavQNHsRfrzXQ@mail.gmail.com>

Hi,

> That's exactly what I meant by consensus, maybe the wrong word there...
> tnt, what's your opinion on gerrit?

I think I'm going crazy. I'm like 100% sure of having answered that a
week ago or so but I can't find any trace of it.

Anyway the gist of it was :

 - gettit is fine
 - If someone wants to take over the project, I'd be happy to see it
in better hands than mine. I don't use pysim on a regular basis (I
have a couple of test sims, but they're programmed and I already have
the parameters and no reason to change them) and don't have most of
the supported cards model to test changes anyway.

Cheers,

    Sylvain


From laforge at gnumonks.org  Mon Mar 27 18:58:55 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Mon, 27 Mar 2017 20:58:55 +0200
Subject: pysim on gerrit? -- was: [PATCH 0/2] fixes for pysim
In-Reply-To: <CAHL+j0_yxs_xMvx+65yCuRg31akBiS3UFCLB7AavQNHsRfrzXQ@mail.gmail.com>
References: <1490270837-28823-1-git-send-email-pmaier@sysmocom.de>
 <6fb32df0-6849-c7f6-ab19-1dce5d02c0e6@sysmocom.de>
 <20170326224708.GC2441@my.box>
 <20170327082712.bkriijzphk3kclh5@nataraja>
 <20170327122002.GA1757@my.box>
 <CAHL+j0_yxs_xMvx+65yCuRg31akBiS3UFCLB7AavQNHsRfrzXQ@mail.gmail.com>
Message-ID: <20170327185855.2sde47dpbllle7ek@nataraja>

Hi Sylvain,

On Mon, Mar 27, 2017 at 06:49:34PM +0200, Sylvain Munaut wrote:
> I think I'm going crazy. I'm like 100% sure of having answered that a
> week ago or so but I can't find any trace of it.

I know the feeling.  Has happened plenty of times before, and then the
mail was either in the drafts folder, or I never actually wrote it and
just thought of writing it.  Must be the age [at least in my case] ;)

> Anyway the gist of it was :
> 
>  - gettit is fine

fine.

>  - If someone wants to take over the project, I'd be happy to see it
> in better hands than mine. I don't use pysim on a regular basis (I
> have a couple of test sims, but they're programmed and I already have
> the parameters and no reason to change them) and don't have most of
> the supported cards model to test changes anyway.

I think this would be an ideal opportunity for one of the various new
contributors part of the accelerate3g5 project (most of whom should be
lurking on this list).  Let's see if somebody wants to step forward for
this.  It's a really small and self-contained tool, and you don't need
to be a low-level C guru, so I think it's actually a good candidate for
somebody interested in contributing in some way.

I'm happy to send anyone interested in maintaining this utility a couple
of different SIM cards (magic, sysmoSIM-GR1, sysmoUSIM-SJS1, ...).  I
don't think I ever had (or still have) all of the supported models,
though.

Regards,
	Harald
-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From nhofmeyr at sysmocom.de  Tue Mar 28 09:43:45 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Tue, 28 Mar 2017 11:43:45 +0200
Subject: pysim has moved to gerrit
Message-ID: <20170328094345.GA1835@ass40.sysmocom.de>

pysim patches are managed by gerrit from now on.

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170328/a2991271/attachment.bin>

From dr.blobb at gmail.com  Thu Mar 30 02:08:33 2017
From: dr.blobb at gmail.com (=?UTF-8?Q?Andr=C3=A9_Boddenberg?=)
Date: Thu, 30 Mar 2017 04:08:33 +0200
Subject: RFC: speed up gerrit verifications by using dependency artifacts
Message-ID: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>

Hi,

some weeks a go the topic about using dependency-artifacts to speed up
gerrit verifications was mentioned by Neels in the "RFC: jenkins
pipeline" thread [1].

I was curious and wrote a bash script (osmo-artifacts.sh) [2], which
enables build jobs to archive and fetch dependencies. Instead of
Docker it simply uses archives to store them. An artifact's name has
the following form:

    ( n(${project}.${branch}.${shortRev}_) ).tar.gz

Openbsc builds are 50 % faster when using artifacts. [3]

The jenkins.sh [2] script had to be slightly modified i.e. basically
split into two functions:

  1) buildProjectDeps()
  2) buildProject()

A third function to generate the artifact name has been added. The
template.sh [2] script illustrates this description.


What do you think about the result/artifactStore/approach in general?

A cleanUpArtifactStore.sh script running as a cron job is already on
my mind, but imo it's a good point to discuss the intermediate result.


blobb

[1] http://lists.osmocom.org/pipermail/openbsc/2017-March/010400.html
[2] git clone https://github.com/blobbsen/diy-artifacts.git
[3] https://jenkins.blobb.me/job/openBSC_multi-configuration_withArtifacts_testRefactoring/buildTimeTrend


From laforge at gnumonks.org  Thu Mar 30 08:17:45 2017
From: laforge at gnumonks.org (Harald Welte)
Date: Thu, 30 Mar 2017 10:17:45 +0200
Subject: RFC: speed up gerrit verifications by using dependency artifacts
In-Reply-To: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
References: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
Message-ID: <20170330081745.mlirs6wsf5uh7fo5@nataraja>

Hi Andre,

just a heads-up about my complete absence from jenkins/CI related
discussions: That doesn't mean I'm not interested, it just means that
I'm happy to defer the decisions on this to those who understand
anything about it (which I don't claim to), i.e. currently most likely
Neels and Holger.

-- 
- Harald Welte <laforge at gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)


From holger at freyther.de  Thu Mar 30 15:20:15 2017
From: holger at freyther.de (Holger Freyther)
Date: Thu, 30 Mar 2017 17:20:15 +0200
Subject: RFC: speed up gerrit verifications by using dependency artifacts
In-Reply-To: <20170330081745.mlirs6wsf5uh7fo5@nataraja>
References: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
 <20170330081745.mlirs6wsf5uh7fo5@nataraja>
Message-ID: <5B5BFB5C-3060-4F87-B88C-6844B98BE9A0@freyther.de>


> On 30. Mar 2017, at 10:17, Harald Welte <laforge at gnumonks.org> wrote:
> 
> Hi Andre,

Hey!


> just a heads-up about my complete absence from jenkins/CI related
> discussions: That doesn't mean I'm not interested, it just means that
> I'm happy to defer the decisions on this to those who understand
> anything about it (which I don't claim to), i.e. currently most likely
> Neels and Holger.

I was very happy to see it. One nice aspect of running in a namespace
is that all jobs can install to /somedir without interfering with each
other. This makes caching intermediate results easy and as shown by
Andre a nice speed-up!

I try to look at it over the next couple of days. My uneducated questions
right now would be. Why not use the archive artifact feature of jenkins?
Why do we need to keep more than one? In general we build all software
against master of libraries (e.g. even for the 3G support).

But this feature is something we want, no need to rebuild software that
has not changed. :)

holger


From nhofmeyr at sysmocom.de  Thu Mar 30 16:14:14 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Thu, 30 Mar 2017 18:14:14 +0200
Subject: RFC: speed up gerrit verifications by using dependency artifacts
In-Reply-To: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
References: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
Message-ID: <20170330161414.GB2045@my.box>

On Thu, Mar 30, 2017 at 04:08:33AM +0200, Andr? Boddenberg wrote:
> [2] git clone https://github.com/blobbsen/diy-artifacts.git
> [3] https://jenkins.blobb.me/job/openBSC_multi-configuration_withArtifacts_testRefactoring/buildTimeTrend

Cool, thanks!
I see it, but I can't immediately grasp how it works :)
Let me take a look at it in a quiet moment and come back with comments later...

~N

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170330/ce81cf62/attachment.bin>

From keith at rhizomatica.org  Thu Mar 30 18:47:40 2017
From: keith at rhizomatica.org (Keith Whyte)
Date: Thu, 30 Mar 2017 20:47:40 +0200
Subject: commit python script to openbsc/contrib
Message-ID: <bc2e83b7-0381-4762-2746-1a8616dc3d4f@rhizomatica.org>

Hi all.

I have a python script I'd like to commit to openbsc/contrib.

It reads the SMS table, gives stats and retrieves individual messages.
- should maybe go in openbsc/contrib/sms ?

Anyway, should I submit this via gerrit + code review or is there
another way?
There's really no point in having jenkins look at it, right?

Thanks!

k.




From holger at freyther.de  Thu Mar 30 19:10:26 2017
From: holger at freyther.de (Holger Freyther)
Date: Thu, 30 Mar 2017 21:10:26 +0200
Subject: commit python script to openbsc/contrib
In-Reply-To: <bc2e83b7-0381-4762-2746-1a8616dc3d4f@rhizomatica.org>
References: <bc2e83b7-0381-4762-2746-1a8616dc3d4f@rhizomatica.org>
Message-ID: <32FD27F6-158E-48FA-A1EA-A44DE5993200@freyther.de>


> On 30. Mar 2017, at 20:47, Keith Whyte <keith at rhizomatica.org> wrote:
> 
> Hi all.
> 
> I have a python script I'd like to commit to openbsc/contrib.
> 
> It reads the SMS table, gives stats and retrieves individual messages.
> - should maybe go in openbsc/contrib/sms ?
> 
> Anyway, should I submit this via gerrit + code review or is there
> another way?
> There's really no point in having jenkins look at it, right?

for maintainers gerrit is the easiest. We just press the +2 button
and can submit. If you need help to be able to push into gerrit just
ping me on jabber.

holger


From dr.blobb at gmail.com  Thu Mar 30 23:39:44 2017
From: dr.blobb at gmail.com (=?UTF-8?Q?Andr=C3=A9_Boddenberg?=)
Date: Fri, 31 Mar 2017 01:39:44 +0200
Subject: RFC: speed up gerrit verifications by using dependency artifacts
In-Reply-To: <5B5BFB5C-3060-4F87-B88C-6844B98BE9A0@freyther.de>
References: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
 <20170330081745.mlirs6wsf5uh7fo5@nataraja>
 <5B5BFB5C-3060-4F87-B88C-6844B98BE9A0@freyther.de>
Message-ID: <CALQ9Oh19bqxMtD1aNe=Tkn2gYGQ6nzOvwM4o9XOesQmbnxCFmQ@mail.gmail.com>

>> Why not use the archive artifact feature of jenkins?

True, archiving artifacts comes ootb, but the functionality to copy/fetch
artifacts from another buildjob/project as a build step has to be installed
via plugin [1].

Would you consider installing the copyArtifact plugin?

In case installing such plugin is not an option one can simply use:

"wget $URL_OF_LATEST_SUCCESSFUL_ARTIFACT" [2]

to fetch artifacts. Such command can be put in an additional
"execute shell" build step, so it wouldn't touch any build script...


But a matrixConfigurationJob [3] archives artifacts per axis, so jenkins would
store 8 artifacts [4]. Unfortunately the "archive artifacts" post-build-step
cannot be triggered by a specific axis configuration (afaik).

A quick brainstorming how to change this behavior ended in:

- additional "execute shell" step evaluating its axes configuration
  and delete artifact accordingly to avoid redundant archiving.

- change "archive artifact" "post build step" to not fail build if no artifact
  could be archived. Unfortunately this configuration applies for all axes,
  so when archiving of the "chosen" artifact fails no one will complain.


Furthermore the "latest successful artifact" might already be outdated when
fetching it. This happens every time when a openbsc [5] build is triggered by
an "upstream project" until its artifact has been archived. So a mechanism is
needed to verify whether fetched artifact is still valid and compiling all
dependencies elsewise.


At this point I just got the feeling that writing a custom script and using a
directory as an artifact store may will be much more straight forward. I agrre,
that this directory approach might be to prototyp'ish.

But let's discuss it. We can change everything. :)


>> Why do we need to keep more than one? In general we build all software
>> against master of libraries (e.g. even for the 3G support).

Good question! If one artifact is sufficient, let's archive only one.
(sure thing)

But according to contrib/jenkins.sh in openbsc repo, some non-master branches
(of libosmo-(netif && sccp)) are used in all axes where "IU" is enabled:

   20) if [ "x$IU" = "x--enable-iu" ]; then
   21)      netif_branch="sysmocom/sctp"
   22)      sccp_branch="sysmocom/iu"
   23) fi
   --
   26) osmo-build-dep.sh libosmo-netif $netif_branch
   27) osmo-build-dep.sh libosmo-sccp $sccp_branch

To be honest I'm a bit confused or am I just missing something?

blobb

[1] https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
[2] wget https://jenkins.blobb.me/job/matrixJobArtifactHandlingExample/HELLO=Salut,WORLD=monde,label=masterSlave/lastSuccessfulBuild/artifact/archive.txt
[3] https://jenkins.blobb.me/job/matrixJobArtifactHandlingExample
[4] https://jenkins.blobb.me/job/matrixJobArtifactHandlingExample/HELLO=Salut,WORLD=monde,label=masterSlave/
[5] http://jenkins.osmocom.org/jenkins/view/OpenBSC/job/OpenBSC/

2017-03-30 17:20 GMT+02:00 Holger Freyther <holger at freyther.de>:
>
>> On 30. Mar 2017, at 10:17, Harald Welte <laforge at gnumonks.org> wrote:
>>
>> Hi Andre,
>
> Hey!
>
>
>> just a heads-up about my complete absence from jenkins/CI related
>> discussions: That doesn't mean I'm not interested, it just means that
>> I'm happy to defer the decisions on this to those who understand
>> anything about it (which I don't claim to), i.e. currently most likely
>> Neels and Holger.
>
> I was very happy to see it. One nice aspect of running in a namespace
> is that all jobs can install to /somedir without interfering with each
> other. This makes caching intermediate results easy and as shown by
> Andre a nice speed-up!
>
> I try to look at it over the next couple of days. My uneducated questions
> right now would be. Why not use the archive artifact feature of jenkins?
> Why do we need to keep more than one? In general we build all software
> against master of libraries (e.g. even for the 3G support).
>
> But this feature is something we want, no need to rebuild software that
> has not changed. :)
>
> holger



-- 
Gru?
Blobb


From nhofmeyr at sysmocom.de  Fri Mar 31 11:46:22 2017
From: nhofmeyr at sysmocom.de (Neels Hofmeyr)
Date: Fri, 31 Mar 2017 13:46:22 +0200
Subject: RFC: speed up gerrit verifications by using dependency artifacts
In-Reply-To: <CALQ9Oh19bqxMtD1aNe=Tkn2gYGQ6nzOvwM4o9XOesQmbnxCFmQ@mail.gmail.com>
References: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
 <20170330081745.mlirs6wsf5uh7fo5@nataraja>
 <5B5BFB5C-3060-4F87-B88C-6844B98BE9A0@freyther.de>
 <CALQ9Oh19bqxMtD1aNe=Tkn2gYGQ6nzOvwM4o9XOesQmbnxCFmQ@mail.gmail.com>
Message-ID: <20170331114622.GB2176@my.box>

On Fri, Mar 31, 2017 at 01:39:44AM +0200, Andr? Boddenberg wrote:
> >> Why not use the archive artifact feature of jenkins?
> But a matrixConfigurationJob [3] archives artifacts per axis, so jenkins would
> store 8 artifacts [4]. Unfortunately the "archive artifacts" post-build-step
> cannot be triggered by a specific axis configuration (afaik).
[...]
> At this point I just got the feeling that writing a custom script and using a
> directory as an artifact store may will be much more straight forward. I agrre,
> that this directory approach might be to prototyp'ish.

To me it sounds that an own approach is indeed simpler. If we need to add a
plugin, resolve URLs and go crazy, might as well use the file system that's
already there and working. Maybe one could benefit from distributed builds,
i.e. when one build slave's artifact can be used across other build slaves, but
that doesn't apply to our situation. Would anyway make sense to build
dependencies once per build slave in a project where each build slave is a
different OS or config instead of sharing load.

> But according to contrib/jenkins.sh in openbsc repo, some non-master branches
> (of libosmo-(netif && sccp)) are used in all axes where "IU" is enabled:
> 
>    20) if [ "x$IU" = "x--enable-iu" ]; then
>    21)      netif_branch="sysmocom/sctp"
>    22)      sccp_branch="sysmocom/iu"
>    23) fi
>    --
>    26) osmo-build-dep.sh libosmo-netif $netif_branch
>    27) osmo-build-dep.sh libosmo-sccp $sccp_branch
> 
> To be honest I'm a bit confused or am I just missing something?

The good news is that the dep branches have become unnecessary just two days
ago. I will remove them from the build scripts now.

The IU prepares for the vlr_3G branch of openbsc, which needed these branched
dependencies. The ./configure --with-iu switch has already been merged to
master, so we are building the incomplete IU feature with master as well, to
make sure no new development inadvertently destroys the IU parts. (To really
build 3G, we would need to also use the openbsc vlr_3G branch instead of
master, which is not an option in a gerrit build.).

~N
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.osmocom.org/pipermail/openbsc/attachments/20170331/700424e2/attachment.bin>

From jsteiger at sysmocom.de  Fri Mar 31 14:20:35 2017
From: jsteiger at sysmocom.de (Joachim Steiger)
Date: Fri, 31 Mar 2017 16:20:35 +0200
Subject: weekly test report (w13 2017)
Message-ID: <78e82f5c-c1ff-d9ab-02d9-11a4fc9d4b15@sysmocom.de>

Hi,

this is the BTS test report for week13 2017.

BTS models which have been tested with OsmoNITB are:

 - sysmoBTS1002
 - octasic OCTBTS3500
 - ettus USRP B200
 - nanoBTS model:165G

All test cases have passed.

For additional information related to test cases, timeslots
configuration and voice codecs used, please visit:

https://osmocom.org/projects/cellular-infrastructure/wiki/Weekly_BTS_Tests

following are some git hashes/version strings of tested components:

libosmo-abis:       5e87fdfcabc9cabe0025d7350b7ab31cdc4b6fa3
libosmocore:        d78c973cd89fc7c119573357cfbebb891dbc697a
libosmo-netif:      5fe77a4656f3590c343861ea96bcec18e370e437
libosmo-sccp:       8e708d1f2da1b187f631bf08172a5194a85b1a23
libsmpp34:          cc0bcd6bc051d5ccaf32cdbbc28f073369900857
openbsc:            b1e6b3749389ec5b3f33d5ac0fcc7f43df3f4641
openggsn:           19e19e3609508d121ba46c165e5ed1502a3cf9da
osmo-bts:           e16b59357411ffa4903ac110ac4ce46d343e878d
osmo-bts-octphy:    e16b59357411ffa4903ac110ac4ce46d343e878d
osmo-pcu:           e6d26ec09c2bcd2126416a58cb23af27318ec67e
osmo-trx:           e0c12189d455eb0d17299e4504749ce36629e18b
sysmobts:	OsmoBTS version 0.4.0.414-e16b5
sysmobts:	Osmo-PCU version 0.2.900-e6d2

octbts:		(name='octsdr_gsm', desc='Software Define Radio',
ver='02.07.00-B1039', ver_hdr='02.07.00-B1039')
octbts:		(platform='Opus2',
version='OCTSYS_VERSION=01.02.19.B1;OCTODK_VERSION=01.15.01-B1;OCTADF_VERSION=04.05.01-B2637;')


nanobts:	Equipment_Version='165g029_79'
nanobts:	Software_Version='168d462_v200b202d0'

if you are interested in the details, drop me a mail and i'll forward
the whole report.

kind regards

-- 
- Joachim Steiger <jsteiger at sysmocom.de>      http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Harald Welte




From dr.blobb at gmail.com  Fri Mar 31 14:23:23 2017
From: dr.blobb at gmail.com (=?UTF-8?Q?Andr=C3=A9_Boddenberg?=)
Date: Fri, 31 Mar 2017 16:23:23 +0200
Subject: RFC: speed up gerrit verifications by using dependency artifacts
In-Reply-To: <20170331114622.GB2176@my.box>
References: <CALQ9Oh3WuYci22PKQxG5e662rRTDdpoLNAMpSA7gnrvUN6MN0A@mail.gmail.com>
 <20170330081745.mlirs6wsf5uh7fo5@nataraja>
 <5B5BFB5C-3060-4F87-B88C-6844B98BE9A0@freyther.de>
 <CALQ9Oh19bqxMtD1aNe=Tkn2gYGQ6nzOvwM4o9XOesQmbnxCFmQ@mail.gmail.com>
 <20170331114622.GB2176@my.box>
Message-ID: <CALQ9Oh2SEq8hT9Vg3qjZgcKApaHQqoHe75zB-qsXJ8+3SLarQA@mail.gmail.com>

>> Would anyway make sense to build dependencies once per build slave in a project
>> where each build slave is a different OS or config instead of sharing load.

Actually the current approach works like that. Each build slave simply
has its own directory for holding compiled-dependencies-artifacts. The
osmocom docker image might need an additional volume to mount the
artifact store, in case /build_bin/artifactStore - which is already
mounted - isn't suitable.

One thing probably worth discussing is which job(s) should
expose/archive artifacts. In the current approach every job that
cannot find the latest dependencies will try to archive/expose them
after building from scratch.

Of course we could dedicate this role to a specific job e.g. [1], but
I simply didn't (want to) think about such approach. So currently all
jobs that profit from dependency artifacts, will archive compiled
dependencies in case no artifact holding the latest dependencies was
found (and no other project, which compiled the same dependencies
finished before).

Practically, the mentioned job [1] will almost always update
artifacts, because the chances that a gerrit job finishes before [1]
correlates proportionally with the polling intervals of all upstream
projects of [1].

In theory a patchset could profit by the artifact archived by its
previous patchset, when the duration of both verifications is shorter
than the above mentioned polling intervals and no change to any
dependency has been introduced meanwhile.

What do you think makes most sense one/several/all job(s) to update artifacts?


>> The good news is that the dep branches have become unnecessary just two days
>> ago. I will remove them from the build scripts now.

Nice, I will apply these changes for openbsc jobs on my jenkins as well.

blobb

[1] http://jenkins.osmocom.org/jenkins/view/OpenBSC/job/OpenBSC/