.NET Core Unit tests failing with exit code -100 when dependency DLL's fail to load. with Resharper

[Iain-Stevenson]

I am using Resharper 2017.1 as a test runner and
I was using the nuget package of "3.4.0-beta-2".
On a fairly large solution (80+ DLL's and climbing)

I was suddenly finding many tests were refusing to work and getting -100 codes back from the call out to dotnet to run them. This I learned from the source code was UNEXPECTED_ERROR

I plugged the 3.3.0 source code into my solution directly and debugged what was happening.

Same result!

The following xml is what I saw when I trapped the case "test-suite": switch statement within TestExecutionListener.OnTestEvent

<test-case id="1034" name="myTest" fullname="myTest.myTests" methodname="myTest" classname="myTest.myTests" runstate="Runnable" seed="1362515035" result="Failed" label="Error" site="Parent" start-time="0001-01-01 00:00:00Z" end-time="0001-01-01 00:00:00Z" duration="0.000000" asserts="0" parentId="1033"> <failure> <message>**<![CDATA[OneTimeSetUp: System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.Logging.Abstractions, Version=1.1.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) ----> System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.Logging.Abstractions, Version=1.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)]]**></message> </failure> </test-case>

Microsoft.Extensions.Logging.Abstractions had failed to load (incorrect version) but I didn't ever see that at the R# end, all I go was

....dotenet-test-nunit.exe exited with code '-100'

My fix was to consolidate the nuget packages in my solution (this was the cause)

The test output was getting dumped and the underlying exception lost because by the time it got to

_testDiscoverySink.SendTestCompleted();

The sink had already exited and terminated as a network connection and there was an IO error.

Which is why we saw the -100 UNEXPECTED_ERROR code come back.

So: two exceptions and the wrong (second) one being reported as output in my use case;

• .NET Core solution
• Unconsolidated DLL versions (it all compiled by the way) cause the problem and terminated the sink
• Using Resharper 2017.1 as the test runner the final output resulted in another error IO Exception, and renders -100 as a result code.

Happy to provide more info if needed. I hope this helps the effort.

[rprouse]

Thanks for the info. Visual Studio 2017 and the new csproj format for .NET Core changed the test adapter API, so the dotnet-test-nunit adapter is now deprecated, so we won't be updating this project. I'll leave this issue open in case other people run into the same problems.

See #108

[Iain-Stevenson]

I'll be upgrading the solution to VS2017 soon so that's good to know. Thanks for the feedback.

[halex2005]

I just found that NUnit3TestAdapter v3.8.0-alpha1 do the job: it is possible now to run the tests using dotnet test command. We just need to have these dependencies in your test project:

    <PackageReference Include="nunit" Version="3.7.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.8.0-*" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />

Good job! Thanks!
Hope this helps anyone.

[juan-cs]

If the project is deprecated, wouldn't it be better to update the README with @halex2005 instructions?

[CharliePoole]

@rprouse He has a point there. I can't find anything to tell me not to use this project. Probably we should go through each of the archived projects and make sure we are telling people clearly what the status is.

BTW, I would call it "deprecated" if you are advising people not to use it, "unsupported" or "not maintained" if you are merely saying that bugs won't be fixed. Probably a sentence is needed rather than a word. 😄

In the extreme, we could move all these projects to a new nunit-archive "organization" to make up for the fact that GitHub doesn't yet support a folder structure for projects.