Why Programmers Use the Test Hierarchy Antipattern

Why Programmers Use the Test Hierarchy Antipattern

The first part of this series described Test Hierarchy, a hierarchy of test classes that mirrors the classes under test, and explained why it’s an antipattern. Part two explored what makes a good unit test and why Test Hierarchy does not. This third and final post reflects on why programmers use Test Hierarchy and why these reasons aren’t persuasive.


I think there are a couple reasons why programmers use Test Hierarchy.

Test Hierarchy may appear to “just make sense” at first blush. After all, you have a hierarchy of classes under test—superclasses and subclasses—and you have a collection of test classes. It seems very symmetrical to have the test classes mirror the classes under test.

However, there’s no design justification for the test classes to be arranged in a parallel hierarchy. The easiest way to see this is to consider what happens when developers get tired of Test Hierarchy. What do they do? They drop back to procedural tests, with no inheritance at all. If you don’t need Test Hierarchy to test object-oriented code using procedural tests, why do you need it when using Test::Class? Answer: You don’t.

Test inheritance should only be used to meet the needs of the tests, not the needs of the code under test.

This usually means that if we have test superclasses, they specifically contain shared setup and teardown code or test utility functions. See the Testcase Superclass pattern, by which a test class can inherit common functionality from an abstract test superclass. Using this pattern, the test hierarchy is organized in order to share common code across the entire project’s tests or an entire subsystem’s tests.

(But use SharedTestModule qw(shared_function) is still preferred over inheritance, because it more explicitly states what is being shared and where.)

I’ve also seen programmers appeal to the Liskov Substitution Principle. This is the idea that if Bat is a subclass of Mammal, then any code that requires a Mammal can be handed a Bat without ill effects. Barbara Liskov and Jeanette Wing formally defined it like this:

Let ϕ(x) be a property provable about objects x of type T. Then ϕ(y) should be true for objects y of type S where S is a subtype of T. (“Behavioural Subtyping Using Invariants and Constraints.” Barbara Liskov, Jeanette Wing. CMU-CS-99-156. MIT Lab, July 1999.)

In other words, a subclass adheres to the same interface contract as its superclasses.

Some programmers will say that if Bat is a subclass of Mammal, then a Bat can do anything that a Mammal can. In other words, a Bat “is a” Mammal. Therefore, it directly follows that BatTest should test all the Mammal behaviors that Bat inherits.

No, it doesn’t, and no, it shouldn’t.

To understand why, consider a couple simple cases.

  • If a Mammal has a care_for_young() method, then every Bat must also be able to care_for_young(). This does not mean that the way bats care for their young is exactly the same as every other mammal. In fact, it’s distinctly different, because baby bats have needs that are distinct from the needs of other baby mammals. In fact, Mammal::care_for_young() may even be an abstract method that dies with a “must be implemented in subclass” error.

  • Similarly, every Animal can move(). That means every Mammal also can move(), because Mammal is a subclass of Animal. By extension every Bat can also move(), because Bat is a subclass of Mammal. Now explain to me how the way a bat moves is identical to the way a sloth moves or the way a tarantula moves. It isn’t.

A subclass adheres to the same interface contract as its superclasses. It does not necessarily implement identical behaviors.

Therefore, just because a Bat “is a” Mammal, that doesn’t mean that a BatTest “is a” MammalTest. Actually, no, a BatTest is not a MammalTest. Not even close. Both BatTest and MammalTest are just tests. Or they might, at most, be derived from OrganismTest abstract class which contains helper methods to set up and manage test fixtures common to all organisms.

Test::Class can test anything straight Test::More can, and vice-versa. The power in Test::Class is not as legend says in testing object-oriented code. The power Test::Class brings is its ability to collect related test methods together, run them independently, and inherit setup and teardown. (I’ll explore more of these details in Testing Strategies for Modern Perl.) Each test class should be derived directly from Test::Class or from an abstract subclass thereof. Never inherit test methods. Just don’t do it.

Peace, love, and may all your TAP output turn green…

Tim King is Lead Developer at The Perl Shop. Tim got his start writing real-time embedded software for high-speed centrifuges the 1980’s and went on to do embedded software for Kurzweil Music Systems and Avid Technology. He has been developing for the web since the web existed, and brings discipline and skills honed from embedded systems to enterprise software. His expertise is in designing for software quality, achieved through automated code testing, test-first development, and risk managed refactoring, all through an agile process. This approach naturally lends itself to working with legacy code, such as successfully and safely refactoring a 465-line legacy function used in a video streaming application into a structurally sound design. Or designing for maintainability, through cleanly layered architectures, like a web service that can handle multiple RPC protocols using a common controller and a thin view layer, that can easily be supplemented to handle additional protocols. Tim is skilled in Perl, JavaScript, and other programming languages, in Internet protocols, in SQL, and is familiar with the internals of a variety of open source applications. Tim also writes and performs music, and has authored and published a number of inspirational books.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.