Skip to content
This repository was archived by the owner on Nov 27, 2017. It is now read-only.
This repository was archived by the owner on Nov 27, 2017. It is now read-only.

Playlists wrongly group tests #114

@nepdev

Description

@nepdev

dotnet-test-nunit v3.4.0-beta-3, with NUnit 3 Test Adapter (on Visual Studio 2015 Update 3 Community Edition)

The project under test is ASP.Net Core targeting .Net 4.5.2 (i.e. not a "pure" dotnet project).

Creating a playlist with 20 tests (out of 100) by selecting in Test Explorer, right-click, Add To Playlist > New Playlist. Only these 20 tests show in Test Explorer.
Now run "All Tests".

Result:
The first 13 tests show in green, the next 7 tests are dimmed out (normally this indicates they were not run).
At the bottom of the screen it says "18 tests passed".
See screenshot
nunit-error1

What should have happened:
All 20 tests are highlighted green. At the bottom it says "20 tests passed".

Workaround:
Do not use playlists. The default view with all tests, grouped by "outcome" works fine.
See also other issue I submitted - it seems that the only grouping which works properly is the default "by outcome" view with "all tests".

Update Edit:

  1. I found that in addition to above the playlist file contains OTHER tests than what I had selected in the UI.

  2. And, reversely, the UI shows other tests than what the playlist contains.
    Example - Making a short playlist with two tests. File then, for example contains:
    <Playlist Version="1.0"><Add Test="StatsServicesTest.AreaTests.AreaRetrieve_MustRetrieve" /><Add Test="StatsServicesTest.AreaTests.AreaRetrieve_ProperData" /></Playlist>
    When filtering the tests to that playlist, it shows 2 tests - but only one is on the playlist.
    In above example in VS it is shown:

AreaCreateDuplicateRoot
AreaRetrieve_MustRetrieve

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions