Change in osmo-ttcn3-hacks[master]: pcu: Check stats for pcu channel allocation, bytes transferred

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.

fixeria gerrit-no-reply at lists.osmocom.org
Tue Sep 22 19:53:27 UTC 2020


fixeria has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/20226 )

Change subject: pcu: Check stats for pcu channel allocation, bytes transferred
......................................................................


Patch Set 3:

Daniel wrote:
> But why shouldn't we verify the counters for regular tests?

Pau wrote:
> This is an ongoing discussion in lots of places where AFAIU fixeria wants to test only one and one only thing on each test while other people prefer testing all possibles parts are working fine on each scenario. I'm of the opinion we should do the later.

Indeed, I stand for the idea of testing something specific rather than as much as possible in one test case. This way if you see some test case failing, you have more chances to guess what exactly is broken and where the potential regression is. And this is kind of a natural intention when you're designing and implementing a new test case - to make it concrete, short, and simple. Nobody wants to see TC_everything, right?

Daniel wrote:
> One reason would be because a counter failure would fail a test that tests the actual functionality while the functionality still works. I'm not sure how often that will happen, though and if/once we trust our counters they can also help verify assumptions made in a testcase.

A good example is ttcn3-bsc-test-latest, where most of the handover test cases fail because of some counter mismatches. It's clear to us, developers, that it's not a problem in the actual logic, and let's say a missing counter increment. However what would a potential customer see is that some test cases are failing, and he/she would unlikely want to investigate why. Given out release tagging intervals... this approach promises a negative trend for the 'latest' feeds.

Another reason is that changing the test logic may potentially change the counter test coverage. Let's say I found an easier way to trigger sending of some specific message, so I remove half of a test, and some related counters never change after that.

In any case, if the majority of you vote for having counters mixed with the existing test cases, let's accept this as a general policy (so I won't annoy you with 'one specific thing' philosophy) and ensure that we check counters in all test cases, not just in 2/74.


-- 
To view, visit https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/20226
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings

Gerrit-Project: osmo-ttcn3-hacks
Gerrit-Branch: master
Gerrit-Change-Id: I90964b32fa11ed2582afc5fb56bd302b06606f86
Gerrit-Change-Number: 20226
Gerrit-PatchSet: 3
Gerrit-Owner: daniel <dwillmann at sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: daniel <dwillmann at sysmocom.de>
Gerrit-Reviewer: fixeria <vyanitskiy at sysmocom.de>
Gerrit-Reviewer: laforge <laforge at osmocom.org>
Gerrit-Reviewer: pespin <pespin at sysmocom.de>
Gerrit-Comment-Date: Tue, 22 Sep 2020 19:53:27 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/gerrit-log/attachments/20200922/c3ff614b/attachment.htm>


More information about the gerrit-log mailing list