osmo-ttcn3-hacks[master]: add compare-results.sh, call from start-testsuite.sh

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.

Neels Hofmeyr gerrit-no-reply at lists.osmocom.org
Thu Apr 5 16:42:22 UTC 2018


Patch Set 1:

> I believe we had this discussion and the conclusion was that we
 > split the tests into two TTCN-3 modules so we have BTS_Tests and
 > BTS_Tests_xfail or something like that?

That was one of the ideas. Even if we do that, I still want a human readable summary (that tells me what changed).
I think having separate suites is not so good, since we constantly need to move code around as tests start passing.
This summary does what I need without so much as touching anything else.

 > Also, I'm quite critical of this in general.  We should focus on
 > fixing all bugs the testsuite produces, rather than trying to hide
 > some of the failures.

To see the failures, I actually need to be able to read them.
And there's nothing more rewarding than updating expected test results to squat one more xfail from the list.
It's not hiding, it's making more visible, IMHO.

-- 
To view, visit https://gerrit.osmocom.org/7646
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
Gerrit-PatchSet: 1
Gerrit-Project: osmo-ttcn3-hacks
Gerrit-Branch: master
Gerrit-Owner: Neels Hofmeyr <nhofmeyr at sysmocom.de>
Gerrit-Reviewer: Harald Welte <laforge at gnumonks.org>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Neels Hofmeyr <nhofmeyr at sysmocom.de>
Gerrit-HasComments: No



More information about the gerrit-log mailing list