[SoftwareRec] Unit Testing
I assemble some unit testing info from wikipedia and cppunit cookbook, and reorganize some source codes. Here is the result:
1 FAQ of Unit Testing
- What
is a Unit?
A unit is a non-trivial function.
- What
is Unit Testing
To test whether desired features and only these features of a unit have been implemented properly.
- What
is Integration Testing?
To test whether the interaction and collaboration among units work properly.
- What
is System Testing?
To verify and validate a production as a whole system.
- Why
Unit Testing?
The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. It provides a written contract that the piece must satisfy. This isolated testing provides four main benefits:- Encourages
change
Unit testing allows the programmer to refactor code at a later date, and make sure the module still works correctly (regression testing). This provides the benefit of encouraging programmers to make changes to the code since it is easy for the programmer to check if the piece is still working properly.
- Simplifies
Integration
Unit testing helps eliminate uncertainty in the pieces themselves and can be used in a bottom-up testing style approach. By testing the parts of a program first and then testing the sum of its parts will make integration testing easier.
- Documents
the code
Unit testing provides a sort of “living document" for the class being tested. Clients looking to learn how to use the class can look at the unit tests to determine how to use the class to fit their needs.
- Separation
of Interface from Implementation
Because some classes may have references to other classes, testing a class can frequently spill over into testing another class. A common example of this is classes that depend on a database; in order to test the class, the tester finds herself writing code that interacts with the database. This is a mistake, because a unit test should never go outside of its own class boundary. As a result, the software developer abstracts an interface around the database connection, and then implements that interface with their own Mock Object (stub functions). This results in loosely coupled code, thus minimizing dependencies in the system.
- Encourages
change
- What
is the Cons of Unit Testing?
It is important to realize that unit-testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors, performance problems and any other system-wide issues. In addition, it may not be trivial to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities. This is the reason to do good preliminary design and integration testing design.
- How
Much Unit Testing is Enough?
Unit testing should be written by developers themselves. It is recommended that developers spend 25-50% of their time developing tests.
- What
is Unit Testing Framework?
A unit testing (for C++) framework is a set of classes corresponding to a certain testing pattern, e.g.; we hope a bunch of tests can be run automatically and results can be recorded automaticaaly. Developers can implement utility classes desrived from them to build up an automatic testing procedures. Basically, a framework may contain the following classes:- Fixture
Unit testing is a NP-hard problem, which means it is impossible to test 100% running states of all objects in our programs. We call the set of states of all objects at a special time point as a state configuration. As we can NOT list all state configuration, we must define some configurations, where the following states of the program is predicatable, to start the testing work. Such a configuration is a fixture. During the testing progress, we can add more fixtures.
Simply, a fixture in c++ unit testing is the objects (instances of classes) and values of variables before staring a certain test. Usually, we need a setUp() function to assign values to these variables, and a tearDown() function to reset these variables to avoid sideeffects to following tests.
- TestCase
A test case is the unit of testing based on fixtures, i.e., what we can test based on the fixtures defined.
- Check
Check is the method to record whether tests is pass or failed, thus, we can easily compare testing on different versions of a unit.
- TestSuite
TestSuite is a bunch of test cases.
- Fixture
- What
is CppUnit?
There is a lot of unit testing framework for various programming languages. CppUnit is a clone of Junit (for JAVA), and oriented to C++.
2 A Starter Example of Unit Testing
In this section, we start from the source codes in Figure 13A Starter Example of Unit Testingfigure.1 to explain unit testing step by step. More examples should be added with the progress of our peoject. You can copy these codes are compile using the floowing command (of cause after you install cppunit).
- #include <cppunit/TestCase.h>
class Complex
{
friend bool operator ==(const Complex& a, const Complex& b);
double real, imaginary;
public:
Complex( double r, double i = 0 )
: real(r) , imaginary(i) { }
};
bool operator ==( const Complex &a, const Complex &b )
{
return a.real != b.real && a.imaginary == b.imaginary;
}
class ComplexNumberTest : public CppUnit::TestCase
{
public:
ComplexNumberTest( std::string name )
: CppUnit::TestCase( name ) {}
void runTest()
{
CPPUNIT_ASSERT( Complex (10, 1) == Complex (10, 1) );
CPPUNIT_ASSERT( !(Complex (1, 1) == Complex (2, 2)) );
}
};
int main()
{
ComplexNumberTest test("test");
test.runTest();
return 0;
}
Figure 1: A Simple Example of Unit Testing
gcc test.cc -lcppunit -ltdl
In these codes, functions in class Complex are testing objectives.
2.1 Fixture
Before start testing, we must decide where to start, i.e., what is the states of the program when we begin to test. In the example in Figure 13A Starter Example of Unit Testingfigure.1, we haven't defined fixtures explicitly (we will disccuss in Section 3.14Fixturessubsection.3.1). But, we implicitly use defined the fixture including three objects of Complex class:o1 = Complex(10, 1), o2 = Complex(2, 2) and o3 = Complex(1, 1)
2.2 TestCase
In Figure 13A Starter Example of Unit Testingfigure.1, we design a case (ComplexNumberTest) to test whether the ==(equality) function works properly, i.e., whether the following statements return correct results:o1 == o1
o2 == o3
2.3 Check
In Figure 13A Starter Example of Unit Testingfigure.1, the CPPUNIT_ASSERT MACRO from cppunit will check texttt==.2.4 TestSuite
In Figure 13A Starter Example of Unit Testingfigure.1, only one test case exists, and the test suite is same as the test case. test.runTest() will run the test case.In this section, we gave a very simple unit testing example. It uses some classes from cppunit to help testing. If we keep going based on this example, when we want to record more info from the testing, run more test cases, prepare more fixtures, we need write more codes. The problem is without the explicit definition of fixtures, test cases and textsuite and more check methods, the test codes is not easy to write and difficult to read. Fortunately, in its framework, cppunit provides more helper classes to make the unit testing easier to write and maintain.
3 A More Detailed Example
In Figure 2, we give more detailed source codes to test the Complex class. We will explain them in this section.
#include <cppunit/TestCase.h>
#include <cppunit/TestSuite.h>
#include <cppunit/TestResult.h>
#include <cppunit/TestCaller.h>
#include <cppunit/ui/text/TestRunner.h>
class Complex {
friend bool operator ==(const Complex& a, const Complex& b);
friend Complex operator +(const Complex& a, const Complex& b);
double real, imaginary;
public:
Complex( double r, double i = 0 ) : real(r), imaginary(i) {}
};
bool operator ==( const Complex &a, const Complex &b ){
return a.real == b.real && a.imaginary == b.imaginary; }
Complex operator +(const Complex& a, const Complex& b){
Complex* result = new Complex(a.real+b.real, a.imaginary+b.imaginary);
return *result;
}
class ComplexNumberTest : public CppUnit::TestFixture {
private:
Complex *m_10_1, *m_1_1, *m_11_2;
public:
void setUp() {
m_10_1 = new Complex( 10, 1 );
m_1_1 = new Complex( 1, 1 );
m_11_2 = new Complex( 11, 2 );
}
void tearDown() {
delete m_10_1;
delete m_1_1;
delete m_11_2;
}
void testEquality() {
CPPUNIT_ASSERT( *m_10_1 == *m_10_1 );
CPPUNIT_ASSERT( !(*m_10_1 == *m_11_2) );
}
void testAddition(){CPPUNIT_ASSERT(*m_10_1 + *m_1_1 == *m_11_2 );}
public:
static CppUnit::Test *suite() {
CppUnit::TestSuite *suiteOfTests
= new CppUnit::TestSuite( "ComplexNumberTest" );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testEquality",
&ComplexNumberTest::testEquality ) );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testAddition",
&ComplexNumberTest::testAddition ) );
return suiteOfTests;
}
};
int main( int argc, char **argv){
CppUnit::TextUi::TestRunner runner;
runner.addTest( ComplexNumberTest::suite() );
runner.run();
return 0;
}
Figure 2: A More Detailed Example
3.1 Fixtures
In Figure 13A Starter Example of Unit Testingfigure.1, the fixtures are defined implicitly, which is difficult to maintain. In In Figure 2, the fixtures are defined explicitly as below:- private:
Complex *m_10_1, *m_1_1, *m_11_2;
public:
void setUp() {
m_10_1 = new Complex( 10, 1 );
m_1_1 = new Complex( 1, 1 );
m_11_2 = new Complex( 11, 2 );
}
void tearDown() {
delete m_10_1;
delete m_1_1;
delete m_11_2;
}
3.2 TestCase
There are two test cases here:- void testEquality() {
CPPUNIT_ASSERT( *m_10_1 == *m_10_1 );
CPPUNIT_ASSERT( !(*m_10_1 == *m_11_2) );
}
void testAddition(){CPPUNIT_ASSERT(*m_10_1 + *m_1_1 == *m_11_2 );}
3.3 Check and Test Suite
TestRunner is the key classes from cppunit that records comprenhensive testing results. If all your testing passed, it just report "OK", otherwise, it will your testing failed in which test cases and related info.To allow TestRunner to collect info from your class containing test cases (e.g., ComplexNumberTest), the class must implement a static function suite, which organize test cases into a test suite, as described below:
- public:
static CppUnit::Test *suite() {
CppUnit::TestSuite *suiteOfTests
= new CppUnit::TestSuite( "ComplexNumberTest" );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testEquality",
&ComplexNumberTest::testEquality ) );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testAddition",
&ComplexNumberTest::testAddition ) );
return suiteOfTests;
}
- CppUnit::TextUi::TestRunner runner;
runner.addTest( ComplexNumberTest::suite() );
runner.run();
return 0;
