osmo-gsm-tester[master]: Add JUnit XML reports; refactor test reporting

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.

Pau Espin Pedrol gerrit-no-reply at lists.osmocom.org
Thu May 18 13:22:24 UTC 2017


Patch Set 3:

(13 comments)

https://gerrit.osmocom.org/#/c/2669/1//COMMIT_MSG
Commit Message:

Line 7: Add JUnit XML reports; refactor test reporting
> ah yes, and let's make this 'add JUnit XML reports; refactor test reporting
Done


https://gerrit.osmocom.org/#/c/2669/3/src/osmo_gsm_tester/suite.py
File src/osmo_gsm_tester/suite.py:

Line 124:             if self.status == Test.UNKNOWN:
> Imagine for some obscure reason we have a status != UNKNOWN, but still an e
Hm I initially thought it was more interesting to know the failure rather than the exception, but thinking it makes sense as you say.


Line 133:                 self.set_fail(ftype, fmsg, False)
> I still think using try:..except: with class names would be nicer than isin
It looks clearer for me this way and also you have far less lines, because the logic in all cases is the same, only content passed to self.set_fail changes.


Line 168:         testcase = et.Element('testcase')
> can't we have the junit related stuff separate, in one place? I mean, you h
I'll move it to a separate file.


Line 191:     FAIL = 'FAIL'
> (could use global values and share across Test and SuiteRun)
I prefer to keep it this way as test has more possibilites than suite run (ie. skip)


Line 251:         self.init_test_vars()
> we're not planning to use a suite_run more than once, and init_test_vars() 
Yes, it is.

About running suite_run more than once, we are actually running it several times in selftest.


Line 266:         self.time = round(self.ts_end - self.ts_start)
> again no need to store end timestamp.
Ok I'll remove it.


https://gerrit.osmocom.org/#/c/2669/3/src/osmo_gsm_tester/trial.py
File src/osmo_gsm_tester/trial.py:

Line 67:         self.junitfname = self.get_run_dir().new_file(self.name()+'.xml')
> (junit_path? junit_file_path? junit_output_path?)
I'll use junit_path


Line 190:     def run_suites(self, names=None):
> spotting a stupidity in my design ... created https://osmocom.org/issues/22
Yes, I find it annoying not being able to run a list of specific tests at once.


Line 193:             if suite_run.run_tests(names) == suite.SuiteRun.FAIL:
> (because I was kind of not seeing it for a minute, I'd prefer if the big ca
Agree


Line 195:             elif self.status == suite.SuiteRun.UNKNOWN:
> Trial.UNKNOWN!
This will be fixed when moving  run_tests to its own line above the if-else clause. 

It is correct to compare against suite.SuiteRun.UNKNOWN though.


https://gerrit.osmocom.org/#/c/2669/1/suites/debug/error.py
File suites/debug/error.py:

Line 5: assert False
> I accept your argument, but.
I saw your comment after pushing the new version.

Ok, I think the easiest will be to drop the difference error/failure all together then. The code is gonna be easier then too. I'll prepare that tomorrow in a new version of the patch.


https://gerrit.osmocom.org/#/c/2669/1/suites/debug/pass.py
File suites/debug/pass.py:

Line 5: pass
> heh, a nop. The file might as well be empty.
Done


-- 
To view, visit https://gerrit.osmocom.org/2669
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iedf6d912b3cce3333a187a4ac6d5c6b70fe9d5c5
Gerrit-PatchSet: 3
Gerrit-Project: osmo-gsm-tester
Gerrit-Branch: master
Gerrit-Owner: Pau Espin Pedrol <pespin at sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: Neels Hofmeyr <nhofmeyr at sysmocom.de>
Gerrit-Reviewer: Pau Espin Pedrol <pespin at sysmocom.de>
Gerrit-Reviewer: neels <nhofmeyr at sysmocom.de>
Gerrit-HasComments: Yes



More information about the gerrit-log mailing list