Run unit tests in a batch loop

We had a case where a tests was randomly crashing the MSTEST agent process, leave no trace of the root cause. So to reproduce, we ran the test in a loop in both MSTEST and xUnit frameworks using the following scripts, posted here for future reference:

rem Run unit tests in an assembly in a loop using xUnit (build 1705).
FOR /L %%i IN (1,1,5000) DO (
xunit.console.clr4 UnitTests.dll /xml C:\xunit-results\output%%i.xml
)

rem Run unit tests in an assembly in a loop using MSTEST.
FOR /L %%i IN (1,1,5000) DO (
mstest /testcontainer:UnitTests.dll /resultsfile:c:\mstest-results\output%%i.xml
)

Unknown Hard Error

This stupid error plagued me for almost a year before a colleague of mine found a fix. If you see this:

unknown-hardware-error

You can make the following registry change to avoid this. This was happening on machines that were running UI tests, so this top-most dialog was causing tests to fail, pretty horrible considering there is no explanation as to the cause. Apply this reg change and move on:

HKLM/SYSTEM/CurrentControlSet/Control/Windows/ErrorMode=2

For reference see:

http://support.microsoft.com/kb/124873/en

On-screen performance monitoring for acceptance tests

If you record videos of your automated acceptance tests (we are using CamStudio), an excellent addition is to install the free Prism HUD utility. This will give you on-screen (but transparent to mouse actions) CPU / disk / network and memory statistics which can be very useful when reviewing these videos. It appears to be the only application that allows reporting of metrics of any process that exceeds a certain (configurable) delta, or is over a certain (configurable) threshold. This means you get much more useful information than just a summary metric (e.g. total CPU utilization), but you don’t clutter your screen with the metrics for all processes, just those ‘that matter’ at any point in time, automatically!!

Go get it here.