osmo-gsm-tester[master]: Improve test reporting code

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.

Pau Espin Pedrol gerrit-no-reply at lists.osmocom.org
Wed May 17 15:05:27 UTC 2017


Patch Set 1:

(3 comments)

https://gerrit.osmocom.org/#/c/2669/1/selftest/suite_test.ok
File selftest/suite_test.ok:

Line 74:     ERROR: [test_error.py] ([TS_ISO8601], 0 sec) type:'AssertionError' message: AssertionError()
> interesting, "TS_ISO8601"?
JUNit Uses this ISO structure for Timestamp on the XML.


https://gerrit.osmocom.org/#/c/2669/1/src/osmo-gsm-tester.py
File src/osmo-gsm-tester.py:

Line 198:                 trials_run.append((current_trial.name(), report))
> now that I look at it, also applies to my code from before this patch: it m
I don't see a good use case to be able to run it for several trials tbh. I'm fine with removing that feature, it will simplify some stuff.


https://gerrit.osmocom.org/#/c/2669/1/suites/debug/error.py
File suites/debug/error.py:

Line 5: assert False
> in the docs, at least in the example, I propose 'assert' as a way to fail t
I think the intention is the opposite: Failures are typified issues or test-related issues, a small subset. For instance: I check if sms text I received is the same as the one I send. At this point, I really know what's the issue, so I can craft a Failure exception describing with detail the issue + logging all interesting information.

On the other side, there are errors. They can always happen, even if they are not related to the test but more to the infrastructure or other software, etc. which usually need deeper understanding. For instance, trying to start a process and getting some error from the OS because we don't have enough rights or other thousand more possibilities.

It's better to raise our own exceptions for known test failures than using assert, which can be used by any piece of code in any place.


-- 
To view, visit https://gerrit.osmocom.org/2669
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iedf6d912b3cce3333a187a4ac6d5c6b70fe9d5c5
Gerrit-PatchSet: 1
Gerrit-Project: osmo-gsm-tester
Gerrit-Branch: master
Gerrit-Owner: Pau Espin Pedrol <pespin at sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Neels Hofmeyr <nhofmeyr at sysmocom.de>
Gerrit-Reviewer: Pau Espin Pedrol <pespin at sysmocom.de>
Gerrit-Reviewer: neels <nhofmeyr at sysmocom.de>
Gerrit-HasComments: Yes



More information about the gerrit-log mailing list