Wiki

Clone wiki

SCons / MsTest

Integrating mstest and coverage reporting in 3rd party Build system with scons

(Tested with VS 2005 only)

I have added mstest.py as new patch, there is a plan to add it to 1.0. But I also added generation of coverage report from mstest run (if coverage was enabled). It does involve running special converter utility and use of external XML python module. So it can not be added in base scons package. I decided to describe steps here.

What will it do?

  • Runs mstest
  • If you configured coverage in your xxx.testrunconfig scons will convert result to .xml file and then generate short .html report by using .xsl. Why?

  • You don't want to use TFS to build your .Net projects, you are using some other build system like CruiseControl, Parabuildetc.

  • You want to have Unit Tests and Coverage result presented in your build system How to do it?

  • Install: libxml2 and libxslt. It developed by: http://xmlsoft.org/python.html you can download Windows installer from: http://users.skynet.be/sbi/libxml-python/ You will need both modules libxml2 and libxslt

  • Download Microsoft.VisualStudio.Coverage.Analysis.dll, coverage.xsl, mstest2xml.exe from this page attachments and put it to your system. Also download , mstest.py. Note: you might need to modify lines: 55, 56, 58,59 to make scons find these files you just downloaded on your system
  • Add mstest.py as tool
  • After you run mstest builder, something like: MsTest("TestResults/unittest.trx","mytests.vsmdi"), and your mytests.vcmdi has coverage enabled, it will generate data.coverage binary file somewhere in In directory TestResults/username_host 2008-04-10 09_01_43/In/host/.
  • Then scons will look for that file in TestResults/username_host 2008-04-10 09_01_43/In/host/data.coverage You might need to adjust it
  • mstest.py will run mstest2xml.exe out covReport xml, where: out location of Out directory, like TestResults/username_host 2008-04-10 09_01_43/Out, covReport full path to data.coverage file and xml location of xml file
  • Then mstest builder will convert generated XML file to html by using coverage.xsl. So from 10 Mb xml file you get small html report like this: This is the coverage report

Assembly coverage

Assembly Blocks not covered % blocks not covered
my1.dll 2419 of 4008 60.3542914171657
my2.dll 125 of 146 85.6164383561644
my3.dll 56 of 74 75.6756756756757
my4.dll 105 of 384 27.34375
my5.dll 401 of 463 86.6090712742981
my6.dll 1605 of 3934 40.7981698017285
my7.dll 287 of 458 62.6637554585153

If you want to modify mstest2xml.exe, there is .Net code for it, also it depends on Microsoft.VisualStudio.Coverage.Analysis.dll, so add it as reference:

using System;
using System.Collections.Generic;
using Microsoft.VisualStudio.CodeCoverage;

namespace mstest2xml
{
    class Program
    {
        static void Main(string[] args)
        {
            String CoveragePath = args[0];
            // Create a coverage info object from the file
            CoverageInfoManager.SymPath = CoveragePath;

            CoverageInfoManager.ExePath = CoveragePath;

            CoverageInfo ci = CoverageInfoManager.CreateInfoFromFile(args[1]);
            // Ask for the DataSet
            // The parameter must be null
            CoverageDS data = ci.BuildDataSet(null);

            // Write to XML
            data.WriteXml(args[2]);
        }
    }
}

Note: for reference I used next article as source for ideas and instructions: joc'c bLog

Updated