Test::Class Hierarchy Is an Antipattern

Test::Class Hierarchy Is an Antipattern

Test::Class is particularly good at testing object-oriented code, or so it is said. You can create a hierarchy of test classes that mirrors the hierarchy of classes under test. But this pattern, common in Perl projects, is conspicuously missing from the rest of the xUnit world, and with good reason.

We’ve all heard of it.

Our project has a class Animal that implements method move() (because all animals can move). This class has a test class AnimalTest that derives from Test::Class and has a test for $animal->move().

So far so good.

Our project also has a class Bat that derives from Mammal, which derives from Animal, and implements method fly(). So we create a test class BatTest that derives from MammalTest, which in turn derives from AnimalTest, and has a test for $bat->fly(). That means that BatTest not only exercises all the behavior of Bat but also of Mammal and Animal, because it inherits all the tests in its ancestors.

Wow! What a cool feature! We get all that testing functionality essentially for free by inheritance!

And if the foregoing description sounded confusing, just imagine how good it’s going to get as we extend the object hierarchy.

Repeat for umpteen different classes.

This arrangement of test classes is what I mean by Test::Class Hierarchy, or more generally, Test Hierarchy.

Multiple prominent sources in the Perl community recommend Test Hierarchy.

And most notably, when I ask a roomful of Perl developers about their experiences with Test::Class, I’m sure to hear at least one person complain about “the rabbit hole of inheritance,” as jnap once put it in a conversation. Of course, not every project misuses Test::Class and its support for inheritance, but as he noted, “I’ve just seen it so wildly abused.” (And to be fair, this is not the only abuse of Test::Class, but it’s the pattern I’m examining at the moment.)

  • This makes fragile, overlapping tests. When we inherit test methods in this way, we end up with test action at a distance. That is, each test class includes tests that are defined in its superclasses, which are completely different modules. A change to any of the superclasses can produce failures in any and all of the subclass tests.
  • These tests are obscure. We can’t know by looking at the test module what functionality it’s testing, at least not without following the inheritance hierarchy all the way to the top.
  • And they’re slow to boot. The test suite takes exponentially longer to run than it needs to, because the same tests are being run over and over again in each subclass.

In the words of Alyssa Mastromonaco (or maybe her publisher): Who thought this was a good idea?

Enough Perl projects use Test Hierarchy that it pops up in criticisms of Test::Class itself, and the negative effects are a significant point when they do.

It bears noting, however, that this practice is strongly discouraged in the rest of the programming world. It’s so rare, in fact, that Gerard Meszaros doesn’t even mention it in his book xUnit Test Patterns.

Rather, the recommended practice is to inherit our test classes directly from Test::Class (or possibly from a project-or subsystem-specific test base class—but that’s a different post). In general, we use as little hierarchy as possible, and whatever hierarchy we do use is organized according to the needs of the tests, not the needs of the system under test. And we never inherit test methods (although we may inherit setup and teardown code).

In summary, we write independent test classes, and we never inherit test methods.

That qualifies Test Hierarchy as a Perl antipattern:

  1. It’s a commonly used structure that despite initially appearing to be appropriate and effective, has more bad consequences than good; and
  2. Another solution exists that is documented, repeatable, and proven to be effective.

In the next post, I’ll talk about why Test Hierarchy does not even make good unit tests.

Peace, love, and may all your TAP output turn green…

Tim King is Lead Developer at The Perl Shop. Tim got his start writing real-time embedded software for high-speed centrifuges the 1980’s and went on to do embedded software for Kurzweil Music Systems and Avid Technology. He has been developing for the web since the web existed, and brings discipline and skills honed from embedded systems to enterprise software. His expertise is in designing for software quality, achieved through automated code testing, test-first development, and risk managed refactoring, all through an agile process. This approach naturally lends itself to working with legacy code, such as successfully and safely refactoring a 465-line legacy function used in a video streaming application into a structurally sound design. Or designing for maintainability, through cleanly layered architectures, like a web service that can handle multiple RPC protocols using a common controller and a thin view layer, that can easily be supplemented to handle additional protocols. Tim is skilled in Perl, JavaScript, and other programming languages, in Internet protocols, in SQL, and is familiar with the internals of a variety of open source applications. Tim also writes and performs music, and has authored and published a number of inspirational books.

Leave a Reply

Your email address will not be published. Required fields are marked *