Before continuing it is worthwhile noting that the examples in this
chapter are bundled within the examples directory of the TESS
distribution (along with a Makefile and baseline test.ref output),
and that they have been written under the assumption that
an operational version of TESS is installed on your system.
It should also be noted that a TESS script is merely a S-Lang script
which has,
somewhere along its execution path, loaded TESS. Normally S-Lang scripts
are named with a .sl suffix, however the author has adopted the
convention of suffixing them with .t instead. Not only does this serve
as a useful mnemonic, it also prevents tests
from being accidently loaded by S-Lang through its default script
loading mechanism (fostering the use of nearly identical names for both
test scripts and the package components which they exercise).
Let's begin our example by augmenting the add() function defined above
with a corresponding subtract() function, and migrating the definition
of both into a S-Lang script named sillymath.sl, as follows:
define add()
{
variable i, j;
if (_NARGS < 2) usage("add(i,j)");
(i,j) = ();
return i+j;
}
define subtract()
{
variable i, j;
if (_NARGS < 2) usage("subtract(i,j)");
(i,j) = ();
return i - j;
}
sillymath.sl, then, are:
add.t:
require("tess");
tess_invoke(1, &add);
tess_invoke(1, &add, "hi there!");
tess_invoke(1, &add, 2);
tess_invoke(0, &add, 2, 3);
tess_invoke(1, &add, "one", 2);
tess_invoke(0, &add, "hello", " there!");
subtract.t:
require("tess");
variable f = &subtract;
tess_invoke(1, f);
tess_invoke(1, f, "hi there!");
tess_invoke(1, f, 2);
tess_invoke(0, f, 2, 3);
tess_invoke(1, f, "one", 2);
tess_invoke(1, f, "hello", " there!");
The first of these should emit feedback resembling that given below.
Since the result of add.t closely resembles that of subtract.t (differing only
in that case 5 signals an exception in one but not the other, since string
subtraction is undefined while string addition is equivalent to concatenation)
we omit output from the latter.
Usage: add(i,j)
Test Case 1: add: PASSED (SHOULD signal error, DID)
Usage: add(i,j)
Test Case 2: add: PASSED (SHOULD signal error, DID)
Stack Contents:
(0)[String_Type]:hi there!
Usage: add(i,j)
Test Case 3: add: PASSED (SHOULD signal error, DID)
Stack Contents:
(0)[Integer_Type]:2
Test Case 4: add: PASSED (SHOULD NOT signal error, DID NOT)
Stack Contents:
(0)[Integer_Type]:5
S-Lang Error: Type Mismatch: String_Type + Integer_Type is not possible
Test Case 5: add: PASSED (SHOULD signal error, DID)
Test Case 6: add: PASSED (SHOULD NOT signal error, DID NOT)
Stack Contents:
(0)[String_Type]:hello there!
=============== add Test Summary ===============
Number of Failures: 0
Number of Passes : 6
In the above tests the absence of an explicit ERROR_BLOCK, and the cleaner
code which results, should be immediately apparent. For example, add.t
contains one fewer non-blank lines than does the example in section
Motivation, while tripling the number of cases tested (6 versus 2).
Also revealed is the fact that tess_invoke() is the single most
important function in the interface.
tess-common.slA more subtle point of
interest is that sillymath.sl does not appear to be loaded by either test
script. How, then, do they function? The answer is that the evalfile()
has been pushed into a file tess-common.sl, which TESS will transparently
load if found within the current directory at startup.
This exploits a common pattern within test suites, namely that prior to
testing any component within a package a test script must first load
the package itself. Furthermore, tess-common.sl can be used to define
variables, data structures, or functions that might be commonly used
amongst all test scripts within a given suite.
Consider the case
tess_invoke(1, &add, "one", 2);
add() (and the similar test defined for subtract()),
which attempts to add an Integer_Type to a String_Type and results
in a type mismatch exception. Even with an ERROR_BLOCK defined S-Lang 1.x would
not, on its own, catch this exception (although S-Lang 2.x will). This
is because S-Lang 1 regards type mismatches as fatal errors, so the
interpreter will not permit execution to continue after they occur.
Because it can be useful to test such conditions, however, TESS relaxes this
constraint by installing an error handler which resets S-Lang 1 internal
state and allows scripts to continue processing. Older versions of TESS
installs this handler by default at startup, but as noted in the function
reference it may be deactivated (or reactivated) by calling
tess_catch_type_errors().
After each test TESS also reports the number and type (or value, for objects which are not aggregates) of any items remaining on the S-Lang stack. Consider the line
Stack Contents:
(0)[String_Type]:hello there!
add.t above. It shows that after test case 6 has
completed the S-Lang stack contains 1 item (as it should), namely the result
of concatenating the strings "hello" and " there!".
This provides an excellent way of validating the return values of exercised functions, without requiring any additional work on the part of the test developer. Conversely, this feature also identifies functions which create unintended side effects by leaving detritus on the stack.
Another point of interest from the add.t output is that the pass/fail
statistics of each test script are automatically summarized, again
with zero work required on the part of the test developer. This happens
because TESS installs an exit handler which, by default, transparently
calls tess_summary when the test application terminates. As noted
in the function reference, this behavior may be customized.
If the test application offers an exit intrinsic then TESS
will invoke it at completion time, passing in the number of failures
observed while running the script. This enables higher-level Makefiles,
or tessrun, to take appropriate action, such as terminating when a non-zero
status is returned.