Test Discovery Error with ASPNET Core 1.1

[Robula]

There seems to be an issue that causes test-nunit to fail to discover tests when "Microsoft.AspNetCore.Mvc": "1.1.0" is added to the test project either directly or via another project reference.

Interestingly, removing the dependency to "Microsoft.AspNetCore.Mvc": "1.1.0" or simply lowering the version ("Microsoft.AspNetCore.Mvc": "1.0.2") allows my tests to be discovered.

Here is my project.json:

{
  "version": "1.0.0-*",

  "dependencies": {
    "NUnit": "3.6.0",
    "NUnit3TestAdapter": "3.6.0",
    "dotnet-test-nunit": "3.4.0-beta-3",
    "Moq": "4.6.38-alpha",
    "Microsoft.AspNetCore.Mvc": "1.1.0"
  },

  "testRunner": "nunit",

  "frameworks": {
    "netcoreapp1.1": {
      "dependencies": {
        "Microsoft.NETCore.App": {
          "version": "1.1.0",
          "type": "platform"
        }
      }
    }
  }
}

The error I get when I build and "Run All" tests is this:

------ Discover test started ------
NUnit Adapter 3.6.0.0: Test discovery starting
Error: Unable to get runner for this assembly. Check installation, including any extensions.
FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.
Dependent Assembly nunit.framework of D:\project\bin\Debug\netcoreapp1.1\Project.Tests.dll not found. Can be ignored if not a NUnit project.
NUnit Adapter 3.6.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.1140183) ==========
------ Discover test started ------
Discovering tests in 'D:\project\Project.Tests\project.json' ["C:\Program Files\dotnet\dotnet.exe" test "D:\project\Project.Tests\project.json" --output "D:\project\Project.Tests\bin\Debug\netcoreapp1.1" --port 15591 --parentProcessId 206184 --no-build]
'test-nunit' returned '-2'.
========== Discover test finished: 0 found (0:00:00.823869) ==========

'test-nunit' returned '-2'. resolves to INVALID_ASSEMBLY.

[rprouse]

FYI, this likely won't be fixed because Microsoft is dropping support for the dotnet-test-* style adapters, so this project will be deprecated soon.

As a workaround, does the dotnet test command work?

[Robula]

Hi @rprouse, thanks for your help.
I didn't realise they were dropping support for the adapters, does this mean that NUnit will no longer support/work with .NET Core?

dotnet test does appear to discover the tests but fails elsewhere

NUnit .NET Core Runner 3.4.0
Copyright (C) 2016 Charlie Poole
Could not load file or assembly 'Microsoft.DotNet.InternalAbstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
SUMMARY: Total: 1 targets, Passed: 0, Failed: 1.

[rprouse]

@Robula NUnit will support .NET Core again, it will just take us a bit of time to play catch-up and add in support in our other adapter 😄

As for your error, I need to look at it more closely, but my guess is that you are directly or indirectly using a different version of Microsoft.DotNet.InternalAbstractions than the adapter is. Unfortunately, .NET Core does not have support for AppDomains, so we cannot isolate your tests from the adapter code in a separate AppDomain like we do in desktop.

[Robula]

Thanks @rprouse, you've been most helpful! I'll look forward to the full .NET Core support, in the meantime I'll fall back to MSTest for any new .NET Core projects.

[rprouse]

It stumbled across a blog post where xUnit is having the same problem with Microsoft.DotNet.InternalAbstractions. MSTest probably has the same issue. I haven't tested the fix yet, but I would appreciate feedback if it works.

The blog post is http://ardalis.com/xunit-test-discovery-error-with-aspnet-core-1-1

The fix is to explicitly add Microsoft.DotNet.InternalAbstractions version 1.0.0 as a dependency to your project.json

[MrMartinRoy]

We had the same problem here. And yes, MSTest has the same issue.

Above fix worked for us.

[rprouse]

Thanks for confirming the fix.