You are reading the documentation for version 3.0 of ProMod3.
You may also want to read the documentation for:
1.3
2.0
2.1
3.1
3.2
|
1 2 3 4 5 | import sys
# this is needed so there will be no test_actions.pyc created in the source
# directory
sys.dont_write_bytecode = True
|
Line 5 does the trick. This needs to be set by you in every action unit test file since Python only recognises it before the module is imported. Otherwise a module could disable bytecoding for all other modules loaded.
Testing actions, basically those are commands run in a shell, is very similar across various actions. Additionally, there are some things that should be tested for all actions like exit codes. That is why this module exists.
When developing an action, you will try it in the shell during the process. You have to check that its doing what you intend, that it delivers the right output, that it just behaves right on various kinds of input. This module supports you by providing functionality to run scripts out of Python. The goal is to not trigger test runs manually in a shell but have a script that does it for you. From there, you do not need to remember all the calls you punched into the command line a year ago, when you come back to change something, add new functionality, etc..
In the next couple of paragraphs, we will walk through setting up a new unit
test script for an imaginary action. We will continuously extend the file
started above, so keep an eye on line numbers. Lets just assume your action is
called do-awesome
for the rest of this section.
The script to supervise your action needs to be placed in
<SOURCE>/actions/tests
and follow the naming convention
test_action_<NAME>.py
, where <NAME>
is the name for your
action. So here we create a file test_action_do_awesome.py
(recognise
the underscore between do
and awesome
instead of a hyphen, that’s
PEP 8).
$ touch <SOURCE>/actions/tests/test_action_do_awesome.py
$
As a starter, we disable bytecode compilation in the script:
1 2 3 4 5 | import sys
# this is needed so there will be no test_actions.pyc created in the source
# directory
sys.dont_write_bytecode = True
|
As always, when introducing new material to ProMod3, it has to be announced
to the CMake build system. For action unit tests, fire up
<SOURCE>/actions/tests/CMakeLists.txt
in your favourite text editor and
add your new script:
1 2 3 4 5 6 7 | set(ACTION_UNIT_TESTS
test_action_help.py
test_action_do_awesome.py
test_actions.py # leave this as last item so it will be executed first!
)
promod3_unittest(MODULE actions SOURCES "${ACTION_UNIT_TESTS}" TARGET actions)
|
The important thing is to leave test_actions.py
as last item in the
list. This script contains the tests around the
test_actions.ActionTestCase
class, which is the foundation of the
tests for your action. If this class is broken, we are lost. Putting it as the
last element in the list, CMake will execute this script first, before any
other action test script is run.
test_actions.ActionTestCase
is sort of a template class for your
tests. By spawning off from this you inherit a bunch of useful methods for your
testing. To make it work, the childclass needs to be set up properly. But
first, test_actions.py
has to be loaded as a module:
6 | import test_actions
|
To showcase, the test cases, we explain how one would (and does) test the
help
action of pm
.
First, we create the childclass for the action.
Go for <NAME>ActionTests
as a naming scheme:
7 8 9 10 | class HelpActionTests(test_actions.ActionTestCase):
def __init__(self, *args, **kwargs):
test_actions.ActionTestCase.__init__(self, *args, **kwargs)
self.pm_action = 'help'
|
Pay attention that in your own class, you must set pm_action
to make
everything work. Also __init__()
needs certain parameters, as everything
is derived from the unittest.TestCase
class.
What needs testing without exclusion are the exit codes of actions. Those
states will be placed in the userlevel documentation. This topic is already
covered in test_actions.ActionTestCase
by RunExitStatusTest()
.
As an example, testing for $?=0
could work like this:
11 12 | def testExit0(self):
self.RunExitStatusTest(0, list())
|
That will call the action stored in pm_action
with the provided list of
parameters and check that 0
is returned on the command line.
In a more general way, you need to test that your action is working as intended. Do not forget some negative testing, with the idea in mind what happens if a user throws dirty input data in.
In ProMod3, unit tests are run via OST‘s ost.testutils
and Python‘s
unittest.TestCase
. Those are called when the test module is executed
as a script:
13 14 15 | if __name__ == "__main__":
from ost import testutils
testutils.RunTests()
|
These three lines should be the same for all unit tests.
Unit tests are executed via make check
and so are ProMod3 action tests.
But for every test script, we also provide a private make
target, ending
with _run
. To solely run the tests for the awesome action, hit
$ make test_action_do_awesome.py_run
test_actions.ActionTestCase
¶When running the test script you will notice that its not really talkative.
Basically you do not see output to stdout
/ stderr
of your
action, while the test script fires it a couple of times. That is by design.
When running the full unit test suite, usually nobody wants to see the output
of everything tested and working. The interesting bits are where we fail.
But for developing a new application you certainly need all the output you can
get. For this, some functions in test_actions.ActionTestCase
have a
parameter verbose
. That triggers specific functions to flush captured
output onto the command line. The idea is to turn it on for development, but
once done, disable it to keep output of unit tests low.
To get the test for exit code 0
talking to you, just do
11 12 | def testExit0(self):
self.RunExitStatusTest(0, list(), verbose=True)
|
and
11 12 | def testExit0(self):
self.RunExitStatusTest(0, list())
|
keeps it silent (verbose
is set to False
by default). If enabled,
output will be separated into stdout
and stderr
:
$ make test_action_do_awesome.py_run
<Lots of output from the build process>
running checks test_action_do_awesome.py
stdout of '<BUILD>/stage/bin/pm do-awesome'
------
<Output to stdout>
------
stderr of '<BUILD>/stage/bin/pm do-awesome'
------
<Output to stderr>
------
<More output from unit test runner>
test_actions.
ActionTestCase
(*args, **kwargs)¶Class to help developing actions. Comes with a lot of convenience wrappers around what should be tested and serves as a recorder for test calls... just for in two years when you come back to rewrite the whole action...
While inheriting this class, pm_action
needs to be defined.
Otherwise the whole idea does not work.
pm_bin
¶This is the path of the pm
binary. Automatically set by calling
__init__()
inside the initialisation of your class.
Type: | str |
---|
pm_action
¶The action to be tested. Needs to be set by your initialisation routine,
after calling __init__()
from here. Skip the
“pm-” in front of the action name.
Type: | str |
---|
RunAction
(arguments, verbose=False)¶Call an action, return the exit status ($?
shell variable). May be
set to verbose
to print the actions terminal output. The action to
be executed needs to be stored in pm_action
first.
If in verbose mode, output to stdout
of the action will be
printed first followed by stderr
.
Parameters: | |
---|---|
Returns: | The exit code of the action ( |
RunExitStatusTest
(exit_code, arguments, verbose=False)¶Run the action with given arguments and check the exit code.
Parameters: |
---|
Enter search terms or a module, class or function name.
test_actions
- Testing Actions