What's the value of a passing unit test?

A unit test that always passes has a certain value - but one that fails can be priceless. Think about the reasons why you might write unit tests for your code.

Sure, they have immediate value in helping you to write sound code the first time around. After all, there’s a big difference between code that has been extensively exercised by unit tests and code that’s never been run at all.

However, if that was the limit of their value, we’d all be deleting the tests as soon as we finished working on a module. We don’t keep them in our codebase due to inertia - there’s a better reason.

We keep the tests in our codebase as a safety net against future errors - that if we inadvertently change the behavior of our system, a test failure (or, many test failures) will tell us that we’ve done so.

It’s therefore as important that our tests fail well as that they run well.

When a test fails, it should clearly indicate what’s gone wrong. Having a test that fails with a blunt NullReferenceException (or worse) isn’t very useful at all.

Consider this simple test:

[Theory]
[MemberData(nameof(FindLoaderTypesForTheory))]
public void EveryLoaderShouldHaveOnePublicConstructor(Type loaderType)
{
    var constructors = loaderType.GetConstructors();
    Assert.Equal(1, constructors.Count());
}

When this test fails, the error message is … charitably … less than helpful:

Message: Assert.Equal() Failure
Expected: 1
Actual: 2

This doesn’t actually tell the developer anything more than the test failed. It forces the developer to read - and understand - the intent of the test before trying to fix the problem. Given that test failures always occur at inconvenient times, the effect is to make a frustrating situation worse.

What if we reduced the amount of thinking required? Let’s rewrite the assertion using a third party assertion framework like FluentAssertions?

[Theory]
[MemberData(nameof(FindLoaderTypesForTheory))]
public void EveryLoaderShouldHaveOnePublicConstructor(Type loaderType)
{
    var constructors = loaderType.GetConstructors();
    constructors.HaveCount(
        1, $"{loaderType.Name} should have one public constructor");
}

The error message generated here is much more useful because it tells the developer what went wrong.

Message: Expected collection to contain 1 item(s) because ExportFileStepLoader should have one public constructor, but found 2.

The key here is that the failure is actionable - any developer reading the message is explicitly told what the problem is and what needs to be done to address the issue.

This can be generalised - any developer seeing a test fail should be explicitly advised what went wrong (but be careful to ensure the guidance is not misleading).

When you write your next unit test, consider the way that it fails and take an extra moment to ensure that any failure is as informative as possible. The time (and sanity) you save might just be your own.