Change in osmo-gsm-tester[master]: add test.report_fragment()

[neels]

neels has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21516 )

Change subject: add test.report_fragment()
......................................................................


Patch Set 2:

(1 comment)

https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21516/2/selftest/suite_test/suitedirA/test_suite/test_report_fragment.py 
File selftest/suite_test/suitedirA/test_suite/test_report_fragment.py:

https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21516/2/selftest/suite_test/suitedirA/test_suite/test_report_fragment.py@11 
PS2, Line 11:     raise Exception('failure in the third fragment')
> Please see sysmocom/suites/4g/ping.py, last line "test.set_report_stdout(output)". […]
test.set_report_stdout() is replacing the <system-out> contents of a test report item.

This patch is instead *adding new report items*.
The point is that I want to have a good indicator of which part of a handover test failed, and see in the Test Results Analyzer at which time which parts started working. Before this, I basically see a wall of red, and every time I want to find out whether bts1 failed to start or maybe the second handover failed, I need to click through logs and accumulate data manually. This patch gives me the information i need instantly across this and past builds.

Take a look at
https://jenkins.osmocom.org/jenkins/job/osmo-gsm-tester_run-prod_handover/test_results_analyzer/
hit 'expand all': each handover.py run has more detailed reports on partial passes, immediately showing what general area the failure is related to.

When you click on one of those sub-items, you get the stderr log of *only that fragment* shown in jenkins immediately.



-- 
To view, visit https://gerrit.osmocom.org/c/osmo-gsm-tester/+/21516
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings

Gerrit-Project: osmo-gsm-tester
Gerrit-Branch: master
Gerrit-Change-Id: I4ca9100b6f8db24d1f7e0a09b3b7ba88b8ae3b59
Gerrit-Change-Number: 21516
Gerrit-PatchSet: 2
Gerrit-Owner: neels <nhofmeyr at sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-CC: pespin <pespin at sysmocom.de>
Gerrit-Comment-Date: Sun, 06 Dec 2020 22:17:10 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: pespin <pespin at sysmocom.de>
Gerrit-MessageType: comment
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/gerrit-log/attachments/20201206/3d2f8431/attachment.htm>