One of the principles of agile development is that your code should have completely automated unit tests. Your tests should require no interaction on your part, and they should require no interpretation; they should simply run and say pass or fail. If your tests require input, then they are not repeatable. If they require interpretation, then they are not automated.
By building an automated regression suite, you can use the tests as an enabler of change to allow you to test every change that you make to your code-base with little effort. Ideally, you would make running tests as part of your build process. In most agile environments, the automated execution of tests is at least part of the integration process (when you commit your code changes to the repository).
There are many testing tools available for agile developers. The most popular is the xUnit framework. (Java: JUnit, .NET: NUnit, C++: CppUnit). These tools may be too heavyweight to learn just for testing your cs213 assignment, so we're going to see how to write an automated testing framework using some more basic primitives. The C language includes a basic assertion package <assert.h>.
Consider the file:testable.c:
#include <stdbool.h>
#include <assert.h>
int isBiggerThanFive(const int x) {
return x > 5;
}
int main() {
assert(true == isBiggerThanFive(10));
assert(true == isBiggerThanFive(6));
assert(false == isBiggerThanFive(5));
assert(false == isBiggerThanFive(4));
assert(false == isBiggerThanFive(-10));
return 0;
}
Compiling and running this file with gcc -Wall testable.c -o testable; ./testable produces no output since all of the tests pass.
[Note: an assertion is a statement that something is believed to be true, so all of your assertions should pass.]
If we add the incorrect assertion that assert(false == isBiggerThanFive(50)); to the bottom of our method, we get the output:
testable: testable.c:15: main:
Assertion `0 == isBiggerThanFive(50)' failed.
This is telling us that for the executable testable, in the file testable.c, on line 15, in method main, our assertion that false == isBiggerThanFive(50) failed. In this case, it's because our test is invalid, but it could have just as easily been an error in our implementation.
If we add another incorrect assertion to our test method true == isBiggerThanFive(0) and re-build, we still get the same output. There is no output from the second test, even though it should fail. This is the expected behavior.
In unit testing, it's desired that a test harness fails on the first error for each test. There is no sense in continuing to run a test after a failure since it means that the unit under test (or the test itself) has entered an unknown state, and therefore we can no longer make assertions (our belief was wrong).
Failing an assert in C causes the program to abort execution. This has the unfortunate effect that only one unit test can fail per executable (subsequent tests will not run if there was a failure). The xUnit testing frameworks handle testing multiple units better, but assert is good enough for testing small programs.
If you don't want to use the assert library, you can write your test harnesses to use the printf function, but all that you should be printing is pass or fail [details]; don't print things that require you to read them to verify that they are correct, as this will destroy any chance of test automation.
