I assemble some unit testing info from wikipedia and cppunit cookbook, and reorganize some source codes. Here is the result:
1 FAQ
of Unit Testing
- What
is a Unit?
A unit is a non-trivial function.
- What
is Unit Testing
To test whether desired features and only these features of a unit have
been implemented properly.
- What
is Integration Testing?
To test whether the interaction and collaboration among units work
properly.
- What
is System Testing?
To verify and validate a production as a whole system.
- Why
Unit Testing?
The goal of unit testing is to isolate each part of the program and
show that the individual parts are correct. It provides a written
contract that the piece must satisfy. This isolated testing provides
four main benefits:
- Encourages
change
Unit testing allows the programmer to refactor code at a later date,
and make sure the module still works correctly (regression testing).
This provides the benefit of encouraging programmers to make changes to
the code since it is easy for the programmer to check if the piece is
still working properly.
- Simplifies
Integration
Unit testing helps eliminate uncertainty in the pieces themselves and
can be used in a bottom-up testing style approach. By testing the parts
of a program first and then testing the sum of its parts will make
integration testing easier.
- Documents
the code
Unit testing provides a sort of “living document" for the
class being tested. Clients looking to learn how to use the class can
look at the unit tests to determine how to use the class to fit their
needs.
- Separation
of Interface from Implementation
Because some classes may have references to other classes, testing a
class can frequently spill over into testing another class. A common
example of this is classes that depend on a database; in order to test
the class, the tester finds herself writing code that interacts with
the database. This is a mistake, because a unit test should never go
outside of its own class boundary. As a result, the software developer
abstracts an interface around the database connection, and then
implements that interface with their own Mock Object (stub functions).
This results in loosely coupled code, thus minimizing dependencies in
the system.
- What
is the Cons of Unit Testing?
It is important to realize that unit-testing will not catch every error
in the program. By definition, it only tests the functionality of the
units themselves. Therefore, it will not catch integration errors,
performance problems and any other system-wide issues. In addition, it
may not be trivial to anticipate all special cases of input the program
unit under study may receive in reality. Unit testing is only effective
if it is used in conjunction with other software testing activities. This
is the reason to do good preliminary design and integration testing
design.
- How
Much Unit Testing is Enough?
Unit testing should be written by developers themselves. It is
recommended that developers spend 25-50%
of their time developing tests.
- What
is Unit Testing Framework?
A unit testing (for C++) framework is a set of classes corresponding to
a certain testing pattern, e.g.; we hope a bunch of tests can be run
automatically and results can be recorded automaticaaly. Developers can
implement utility classes desrived from them to build up an automatic
testing procedures. Basically, a framework may contain the following
classes:
- Fixture
Unit testing is a NP-hard problem, which means it is impossible to test
100% running states of all objects in our programs. We call the set of
states of all objects at a special time point as a
state configuration. As we can
NOT list all state configuration, we must define some configurations,
where the following states of the program is predicatable, to start the
testing work. Such a configuration is a fixture. During the testing
progress, we can add more fixtures.
Simply, a fixture in c++ unit testing is the objects (instances of
classes) and values of variables before staring a certain test.
Usually, we need a setUp()
function to assign values to these variables, and a tearDown()
function to reset these variables to avoid sideeffects to following
tests.
- TestCase
A test case is the unit of testing based on fixtures, i.e., what we can
test based on the fixtures defined.
- Check
Check is the method to record whether tests is pass or failed, thus, we
can easily compare testing on different versions of a unit.
- TestSuite
TestSuite is a bunch of test cases.
- What
is CppUnit?
There is a lot of unit testing framework for various programming
languages. CppUnit is a clone of Junit (for JAVA), and oriented to C++.
2 A
Starter Example of Unit Testing
- #include <cppunit/TestCase.h>
class Complex
{
friend bool operator ==(const Complex& a, const Complex& b);
double real, imaginary;
public:
Complex( double r, double i = 0 )
: real(r) , imaginary(i) { }
};
bool operator ==( const Complex &a, const Complex &b )
{
return a.real != b.real && a.imaginary == b.imaginary;
}
class ComplexNumberTest : public CppUnit::TestCase
{
public:
ComplexNumberTest( std::string name )
: CppUnit::TestCase( name ) {}
void runTest()
{
CPPUNIT_ASSERT( Complex (10, 1) == Complex (10, 1) );
CPPUNIT_ASSERT( !(Complex (1, 1) == Complex (2, 2)) );
}
};
int main()
{
ComplexNumberTest test("test");
test.runTest();
return 0;
}
Figure
1: A Simple Example of Unit Testing
In this section, we start from the source codes in Figure
13A Starter Example of Unit
Testingfigure.1 to explain unit
testing step by step. More examples should be added with the progress
of our peoject. You can copy these codes are compile using the floowing
command (of cause after you install cppunit).
gcc test.cc -lcppunit -ltdl
In these codes, functions in class
Complex
are testing objectives.
2.1 Fixture
Before start testing, we must decide where
to start, i.e., what is the states of the program when we begin to
test. In the example in Figure
13A Starter Example of Unit
Testingfigure.1, we haven't
defined fixtures explicitly (we will disccuss in Section
3.14Fixturessubsection.3.1).
But, we implicitly use defined the fixture including three objects of
Complex
class:
o1 = Complex(10, 1),
o2 = Complex(2, 2)
and
o3 = Complex(1, 1)
2.2 TestCase
In Figure
13A Starter Example of Unit
Testingfigure.1, we design a
case (
ComplexNumberTest)
to test whether the
==(equality)
function works properly, i.e., whether the following statements return
correct results:
o1 == o1
o2 == o3
In Figure
13A Starter Example of Unit
Testingfigure.1, the
CPPUNIT_ASSERT MACRO from cppunit will check texttt==.
2.4 TestSuite
In Figure
13A Starter Example of Unit
Testingfigure.1, only one test
case exists, and the test suite is same as the test case.
test.runTest()
will run the test case.
In this section, we gave a very simple unit testing example. It uses
some classes from cppunit to help testing. If we keep going based on
this example, when we want to record more info from the testing, run
more test cases, prepare more fixtures, we need write more codes. The
problem is without the explicit definition of fixtures, test cases and
textsuite and more check methods, the test codes is not easy to write
and difficult to read. Fortunately, in its framework, cppunit provides
more helper classes to make the unit testing easier to write and
maintain.
3 A
More Detailed Example
#include <cppunit/TestCase.h>
#include <cppunit/TestSuite.h>
#include <cppunit/TestResult.h>
#include <cppunit/TestCaller.h>
#include <cppunit/ui/text/TestRunner.h>
class Complex {
friend bool operator ==(const Complex& a, const Complex& b);
friend Complex operator +(const Complex& a, const Complex& b);
double real, imaginary;
public:
Complex( double r, double i = 0 ) : real(r), imaginary(i) {}
};
bool operator ==( const Complex &a, const Complex &b ){
return a.real == b.real && a.imaginary == b.imaginary; }
Complex operator +(const Complex& a, const Complex& b){
Complex* result = new Complex(a.real+b.real, a.imaginary+b.imaginary);
return *result;
}
class ComplexNumberTest : public CppUnit::TestFixture {
private:
Complex *m_10_1, *m_1_1, *m_11_2;
public:
void setUp() {
m_10_1 = new Complex( 10, 1 );
m_1_1 = new Complex( 1, 1 );
m_11_2 = new Complex( 11, 2 );
}
void tearDown() {
delete m_10_1;
delete m_1_1;
delete m_11_2;
}
void testEquality() {
CPPUNIT_ASSERT( *m_10_1 == *m_10_1 );
CPPUNIT_ASSERT( !(*m_10_1 == *m_11_2) );
}
void testAddition(){CPPUNIT_ASSERT(*m_10_1 + *m_1_1 == *m_11_2 );}
public:
static CppUnit::Test *suite() {
CppUnit::TestSuite *suiteOfTests
= new CppUnit::TestSuite( "ComplexNumberTest" );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testEquality",
&ComplexNumberTest::testEquality ) );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testAddition",
&ComplexNumberTest::testAddition ) );
return suiteOfTests;
}
};
int main( int argc, char **argv){
CppUnit::TextUi::TestRunner runner;
runner.addTest( ComplexNumberTest::suite() );
runner.run();
return 0;
}
Figure
2: A More Detailed Example
In Figure
2,
we give more detailed source codes to test the Complex class. We will
explain them in this section.
3.1 Fixtures
In Figure
13A
Starter Example of Unit Testingfigure.1,
the fixtures are defined implicitly, which is difficult to maintain. In
In Figure
2,
the fixtures are defined explicitly as below:
- private:
Complex *m_10_1, *m_1_1, *m_11_2;
public:
void setUp() {
m_10_1 = new Complex( 10, 1 );
m_1_1 = new Complex( 1, 1 );
m_11_2 = new Complex( 11, 2 );
}
void tearDown() {
delete m_10_1;
delete m_1_1;
delete m_11_2;
}
Here,
m_10_1 *m_1_1 *m_11_2
are those objects that the testing running on;
setUp
function initialize these objects to a certain configuration and
tearDown()
is responsible for resetting the configuration.
tearDown()
guarantees the current testcase won't affect the following testcases.
3.2 TestCase
There are two test cases here:
- void testEquality() {
CPPUNIT_ASSERT( *m_10_1 == *m_10_1 );
CPPUNIT_ASSERT( !(*m_10_1 == *m_11_2) );
}
void testAddition(){CPPUNIT_ASSERT(*m_10_1 + *m_1_1 == *m_11_2 );}
Cppunit provides prolific classes to run these test cases in different
way. In the following Section
3.34Check and Test
Suitesubsection.3.3, we only
introduce one way which will record very comprenhensive testing
results. For mroe info, readers can refer to:
here
3.3 Check
and Test Suite
TestRunner
is the key classes from cppunit that records comprenhensive testing
results. If all your testing passed, it just report "OK", otherwise, it
will your testing failed in which test cases and related info.
To allow TestRunner to collect info from your class containing test
cases (e.g., ComplexNumberTest), the class must implement a static
function
suite,
which organize test cases into a test suite, as described below:
- public:
static CppUnit::Test *suite() {
CppUnit::TestSuite *suiteOfTests
= new CppUnit::TestSuite( "ComplexNumberTest" );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testEquality",
&ComplexNumberTest::testEquality ) );
suiteOfTests->addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testAddition",
&ComplexNumberTest::testAddition ) );
return suiteOfTests;
}
Then, we can ask TestRunner to run the test suite and record results,
as below:
- CppUnit::TextUi::TestRunner runner;
runner.addTest( ComplexNumberTest::suite() );
runner.run();
return 0;
