Adding a new feature to a personal project, I ran into an odd case where the unit tests weren’t running correctly. It took me hours to find the source of the problem, so I figured I’d share the result.

The Symptoms

Things took an odd turn when I noticed that test runs within Visual Studio weren’t completing. This isn’t to say that they went into an infinite loop, but rather that each test run aborted partway through:

As a double check, I tried the Resharper test runner to see if the tests ran there:

While this superficially looks better, it’s actually much, much worse - I knew that some of those tests were failing, so the results were misleading.

Tracing the problem

Using a debugger, I was quickly able to verify that the expected tests were properly failing by throwing an exception.

Hacking around with some of the existing tests to make them artificially fail showed that they too were showing as green (passed) when they shouldn’t.

Somewhere along the line, something had changed with the project that masked the test outcomes, making it look like they all passed even when they failed.

The Solution

I created a parallel project, imaginatively called Document.Factory.Liquid.Tests2 and started copying tests across, one at a time, hoping to isolate the cause of the problem.

When I managed to copy all of the tests across and everything was working, I was understandably perplexed.

Using Beyond Compare, I compared the two project directories, looking for changes - and then I spotted the culprit: app.config

The app.config for the dodgy project contained these assembly redirection bindings:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="System.Reflection" 
          publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.1.1.0" newVersion="4.1.1.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Runtime.Extensions" 
          publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.1.1.0" newVersion="4.1.1.0"/>
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.2"/>
  </startup>
</configuration>

None of these redirects are required; removing them eliminated the problem and now the formerly dodgy project works perfectly. Blogged under the good Samaritan principle in the hope it saves someone some stress.

Comments

blog comments powered by Disqus
Next Post
Sharpen The Saw #22  04 Dec 2017
Prior Post
Sharpen The Saw #21  27 Nov 2017
Related Posts
Using Constructors  27 Feb 2023
An Inconvenient API  18 Feb 2023
Method Archetypes  11 Sep 2022
A bash puzzle, solved  02 Jul 2022
A bash puzzle  25 Jun 2022
Improve your troubleshooting by aggregating errors  11 Jun 2022
Improve your troubleshooting by wrapping errors  28 May 2022
Keep your promises  14 May 2022
When are you done?  18 Apr 2022
Fixing GitHub Authentication  28 Nov 2021
Archives
December 2017
2017