Assembly binding redirects for the fail

Adding a new feature to a personal project, I ran into an odd case where the unit tests weren’t running correctly. It took me hours to find the source of the problem, so I figured I’d share the result.

The Symptoms

Things took an odd turn when I noticed that test runs within Visual Studio weren’t completing. This isn’t to say that they went into an infinite loop, but rather that each test run aborted partway through:

As a double check, I tried the Resharper test runner to see if the tests ran there:

While this superficially looks better, it’s actually much, much worse - I knew that some of those tests were failing, so the results were misleading.

Tracing the problem

Using a debugger, I was quickly able to verify that the expected tests were properly failing by throwing an exception.

Hacking around with some of the existing tests to make them artificially fail showed that they too were showing as green (passed) when they shouldn’t.

Somewhere along the line, something had changed with the project that masked the test outcomes, making it look like they all passed even when they failed.

The Solution

I created a parallel project, imaginatively called Document.Factory.Liquid.Tests2 and started copying tests across, one at a time, hoping to isolate the cause of the problem.

When I managed to copy all of the tests across and everything was working, I was understandably perplexed.

Using Beyond Compare, I compared the two project directories, looking for changes - and then I spotted the culprit: app.config

The app.config for the dodgy project contained these assembly redirection bindings:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="System.Reflection" 
          publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.1.1.0" newVersion="4.1.1.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Runtime.Extensions" 
          publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.1.1.0" newVersion="4.1.1.0"/>
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.2"/>
  </startup>
</configuration>

None of these redirects are required; removing them eliminated the problem and now the formerly dodgy project works perfectly. Blogged under the good Samaritan principle in the hope it saves someone some stress.