7 Jan
2018
7 Jan
'18
7:48 p.m.
On Wed, Dec 20, 2017 at 03:13:38PM +0100, Harald Welte wrote:
* how can a patch that breaks 'make check' of
osmo-bts/openbsc/osmo-pcu, etc.
be merged to libosmcoore? Isn't that what we have inverse dependencies for
in the gerrit verification jobs?
In libosmocore gerrit, we don't test depending projects. We'd find the master
jobs failing if we merge a breaking change.
* how can it be that even after the patch is merged,
we find the osmo-sgsn issue
only by manual testing and not by automatic testing?
I also wondered about this, and AFAICT the reason is that osmo-gsm-tester
doesn't do more than one GPRS subscriber. SGSN worked fine with one subscriber,
so we need a test that adds a second GPRS user to osmo-gsm-tester...
https://osmocom.org/issues/2820
Maybe a titan test would be more fitting to add a number of subscribers?
b) introduce an intermediate stage, something like a
"staging" branch.
So the existing code review and gerrit-jenkins-build-testing would
make patches end up in 'staging' first, where then the slower
(hourly/daily/...) tests like osmo-gsm-tester are executed. And
only once that's green, we go towards master.
We could cause each and every gerrit patch to trigger osmo-gsm-tester and/or
titan test runs; i.e. perfect coverage means "infinite" jenkins job effort and
time to wait for V+1.
Aiming for perfect coverage before merging patches seems to me to be fighting
windmills; we can get better with every failure, but I guess we'll always need
reassurance from real-world tests.
So far the tradeoff was to rely on master job failures and osmo-gsm-tester
failures to be reported. Which works only as long as we watch those and as long
as they are able to catch the particular issues.
Maybe an IRC bot to report build failures to #osmocom-dev could be nice besides
mail notifications... just dreaming.
I don't think that any given single "event
network" should dictate the patch
merging.
Was of course a selfish, not entirely serious request, with a wink.
~N