Reusing existing test suites for new features

Redundancy is one of the biggest sources of problems on any project, especially in tests. Having an unmaintainable test suite is in nobody’s interest. Eliminating redundant code is one of those things that can trump other design considerations. In the case I’ll describe here I hijacked my an existing test suite to test a new feature.

The existing tests

Type conversion is a major part of Leaf and I have very good test coverage for it. These tests are extremely compact, expressing just the minimal amount needed for a test. They start with a specialized testing class, just called tester, with an overloaded function operator to perform the test.

1
2

tester t;
t( "integer", "integer shared lvalue", "c_drop_lvalue->c_deref" );

This test checks the process of converting integer shared lvalue to integer. It is expecting the conversion engine will return c_drop_lvalue->c_deref. This is made possible by a good separation of concerns in my conversion code. The first step, tested here, simply identifies the conversion that needs to be done, but can’t actually do the conversion.

t can be used multiple times. It’s instantiated only once for a logical group of tests. It also has a few properties on it to control how the testing works.

Good coverage and a new test

I have several hundred of these tests covering, hopefully, every conditional, branch, and logical output from the conversion engine. As it covers all conversion it makes sense it also covers all types. If I need to test something new on types, which I do now, this would be a good starting point.

Type serialization is the new feature I’m adding. All of the types in Leaf must be written to an object file to be read back later. This is part of shared library creation and loading. Instead of writing a new test suite I decided it’d be better to overload this existing one. It’s not just to save coding, it also ensures I have proper coverage. If I were to write a new set of tests I’d most likely miss types.

I injected a new serial test into the function that tests conversion:

1
2
3
4
5
6

void operator()( char const * sto, char const * sfrom, char const * expect ) {
    ...
    test_serial( sto );
    test_serial( sfrom );
    ...
}

The serial test only requires a single type so I just call it twice with the two types used in the original test. The expect parameter is ignored here as it isn’t used in serialization.

This ends up creating a lot of redundant tests for serialization: tests that don’t test anything different from the previous tests. I don’t care much about this type of redundancy since it’s just a minor runtime cost (this entire suite still executes in under a second).

Multi-purposed tests

I know this violates some guidelines about singular purpose tests, but redundancy elimination to this degree is a good overriding goal. Plus, the test still is quite focused. I’m not actually mixing the two tests, just reusing the outer level call t( "integer", "integer shared lvalue", "c_drop_lvalue->c_deref" );. That code just happens to invoke two unit tests now instead of one.