[FIXED] Azure pipeline not finding Selenium tests

Issue

I am trying to set up an Azure pipeline which will run a Selenium test suite. The suite was build in Visual Studio 2022 with C# and Selenium. The suite runs fine from Visual Studio, command line and also Jenkins but we are looking to build it into a release pipeline and this is a test to see how this will work.

The test pipeline appears to work until we reach task - VSTest@2. At this point the process finds the build dll but does not find the tests which are in this file, below is the current code

trigger:
- master

pool:
  name: Azure Pipelines
  vmImage: 'windows-latest'
  demands: vstest

variables:
  solution: '**/Testapp.sln'

steps:
- task: NuGetToolInstaller@1

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"'

- task: VSTest@2
  displayName: 'Regression tests - warmup'
  inputs:
    testSelector: testAssemblies
    testAssemblyVer2: |
      D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll
    testRunTitle: Warmup
    testFiltercriteria: 'TestCategory=TestappE2EWarmup'

I have also tried a few variations, thus

testAssemblyVer2: |
  D:\a\1\s\Testapp\Testapp.csproj

and

testFiltercriteria: 'Name=TestWarmup'

What we are getting at the moment is this

Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220626-01 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.
vstest.console.exe "D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll"
/TestCaseFilter:"Category=TestappE2EWarmup"
/Settings:"D:\a\_temp\qhskojhglda.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"D:\a\1\s"
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
NUnit Adapter 4.0.0.0: Test execution started
Running selected tests in D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll
   NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
NUnit Adapter 4.0.0.0: Test execution complete
No test matches the given testcase filter `Category=TestappE2EWarmup` in D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll

Now from what I understand, the VSTest@2 plugin should be able to find Selenium tests and test adapters such as NUnit as per the screenshot below:

This is a .NET6.0 project which is being built to replace a .NET Core 3.1 project. The old core code used to find the tests without any problem which is the code below.

- task: DotNetCoreCLI@2
  displayName: 'Regression tests - public'
  inputs:
    command: test
    projects: '$(regressionTests)'
    arguments: '--configuration $(BuildConfiguration) --filter "Category=TestappE2EWarmup"'

Where have I have gone wrong?

Solution

Thanks for the replies, I now having this working by switching from nunit to mstest in the project

<PackageReference Include="MSTest.TestAdapter" Version="2.2.10" />
<PackageReference Include="MSTest.TestFramework" Version="2.2.10" />
<PackageReference Include="SpecFlow.MsTest" Version="3.9.74" />

Now the tests are being found and they run

Answered By - Kev