Change in osmo-gsm-tester[master]: doc: manual: Write initial Test API section

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.

pespin gerrit-no-reply at lists.osmocom.org
Fri Jun 5 09:03:08 UTC 2020


pespin has submitted this change. ( https://gerrit.osmocom.org/c/osmo-gsm-tester/+/18663 )

Change subject: doc: manual: Write initial Test API section
......................................................................

doc: manual: Write initial Test API section

Change-Id: I86bc9a8a0b5ff50e72c21a4bd8a857830bd84c4c
---
M doc/manuals/chapters/test_api.adoc
1 file changed, 105 insertions(+), 2 deletions(-)

Approvals:
  Jenkins Builder: Verified
  pespin: Looks good to me, approved



diff --git a/doc/manuals/chapters/test_api.adoc b/doc/manuals/chapters/test_api.adoc
index f541231..f159348 100644
--- a/doc/manuals/chapters/test_api.adoc
+++ b/doc/manuals/chapters/test_api.adoc
@@ -1,4 +1,107 @@
 == Test API
 
-*TODO* (in the meantime, look at src/osmo_gsm_tester/test.py, as well as
-suite.py, which calls the test's setup() function to get an idea)
+All tests run by {app-name} are python script files. On top of usual python
+standard features, {app-name} provides a set of public APIs and tools that
+these tests can use in order to interact with the core of {app-name}, like
+creating object classes, run processes synchronously or asynchronously, wait for
+events, retrieve specific configuration, etc. This section aims at documenting
+the most relevant tools available for tests.
+
+First of all, it is important to avoid blocking out of the core's main event loop in
+the test, since doing that will prevent {app-name} core functionalities to work
+properly, such as control of asynchronous processes.
+
+To get access to those functionalities, a test must import a test environment
+previously prepared by {app-name} before the test was started:
+[source,python]
+----
+#!/usr/bin/env python3
+from osmo_gsm_tester.testenv import *
+----
+
+After the test environment is imported, aome usual functionalities are available
+directly under the global scope. Specially noticeable is the existence of object
+_tenv_, which provides access to most of the functionalities.
+
+The test can simply ask {app-name} to sleep some time, while giving control back
+to {app-name} core's mainloop:
+[source,python]
+----
+sleep(3) # sleep for 3 seconds
+----
+
+One can also wait for events in the background, for instance launch a child
+process locally in the same host and wait for its termination:
+[source,python]
+----
+proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4') <1>
+tenv.remember_to_stop(proc) <2>
+proc.launch() <3>
+proc.wait() <4>
+----
+<1> Create process object. This line doesn't yet runs it.
+<2> Make sure the core will kill the process if this test fails
+<3> Start process asynchronously
+<4> wait until process is done. One could waiting generically here too: _wait(proc.terminated)_
+
+If running asynchronously is not needed, one can run synchronously in an easy
+way:
+[source,python]
+----
+proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4')
+proc.launch_sync()
+----
+
+One can also log output using either the regular _print_ function from python,
+or using {app-name} specific functions available:
+[source,python]
+----
+log('this is a regular log message')
+dbg('this is a dbg message, only printed on outputs where dbg is enabled')
+err('outputs log message for non-expected events')
+print('this is the same as log()')
+----
+
+The test also gains access to suite and/or test specific configuration through
+different APIs:
+[source,python]
+----
+test_config = tenv.config_test_specific()
+threshold = int(test_config.get('threshold', 2))
+suite_config = tenv.config_suite_specific()
+foobar = suite_config['foobar']
+----
+
+A test requiring a really specific config file for an object class it is going
+to run can provide its own template files by overlaying an own directory
+containing them on top of the usual default directory where object class
+templates are (_osmo-gsm-tester.git/src/osmo_gsm_tester/obj/templates/_):
+[source,python]
+----
+tenv.set_overlay_template_dir(os.path.join(os.path.dirname(__file__), 'mytemplatedir'))
+----
+
+Several tests in a suite can also share code by using some APIs provided by
+{app-names}. The shared python code must be placed in files under the 'lib/'
+subdirectory in the suite directory where the test belongs to.
+[source,python]
+----
+# File containing function foobar() available under ${suite_dir}/lib/testlib.py:
+import testlib
+tenv.test_import_modules_register_for_cleanup(testlib)
+from testlib import foobar
+----
+
+For a complete set of features and how to use them, one can have a look at real
+examples present in {app-name} git repository under the _sysmocom/_ directory.
+Besides those, have a look too a _testenv.py_ file, which implements the 'tenv'
+object available to tests.
+
+=== Test verdict
+
+In general, a test reaching the end of the file and returning control to
+{app-name} core will be flagged as a successful test (PASS).
+
+If an exception is thrown from within the test file and propagated to
+{app-name}, the test will be considered as failed and {app-name} will store all
+failure related information from the caught exception.

-- 
To view, visit https://gerrit.osmocom.org/c/osmo-gsm-tester/+/18663
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings

Gerrit-Project: osmo-gsm-tester
Gerrit-Branch: master
Gerrit-Change-Id: I86bc9a8a0b5ff50e72c21a4bd8a857830bd84c4c
Gerrit-Change-Number: 18663
Gerrit-PatchSet: 3
Gerrit-Owner: pespin <pespin at sysmocom.de>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: pespin <pespin at sysmocom.de>
Gerrit-MessageType: merged
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/gerrit-log/attachments/20200605/d6b35c54/attachment.htm>


More information about the gerrit-log mailing list