Devlico.Us
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @devlicious

Derik Whittaker

Thoughts on Software Development, .Net, OOP, Design Patterns and all things cool



MSTest, why I hate you.... You cause me too much friction

Yesterday I spent 4+ hours trying to tackle a bug that should not have been a bug in the first place, and the culprit was MSTest.  Please keep in mind that I have never really used MSTest in any real form.  I am a NUnit guy through and through.

Here is some background on what I was trying to do (keep in mind, I have done this numerous times in the past).  I was setting up an application that was going to use the StructureMap Ioc/DI (Version 2.4.9 -- aka 2.5) container.  Because I do not mind using the attributres and StructureMap.config files this was the way I was going to do all the wiring for my container. 

During the testing (via MSTest) of this code, I immediatly ran into an issue with the .config file not being found at runtime.  Now this was odd to me as I could see that the .config file was being copied to the output directly like it should be.  I spent the next 20-30 minutes ensuring that all my code was correct and that the config file was indeed in the right place.

It was not till I was stepping though the current directory was not my /bin folder, but rather \TestResults\whitt010_CM4NRG1 2008-07-22 16_18_25\out (if you have ever used MSTest you will understand the folder naming convetion).  I found this to be very, very odd.  I did a little digging and realized that for each test run MSTest will create a new folder to run all the tests in (why, I have now figgen clue).  But because my .config file was not a 'normal' file in the eye of MSTest it was not being copied.  Well, my good buddy Steven Andrews was nice enought to help me figure out how to copy the .config file to the correct output folder (post on that forthcoming).  Well, I thought this would solve my issue, but it did not.

Now that I have the .config in the correct test folder (again, not in the bin like ALL the other testing frameworks..why does MS always have to be different) I thought I would be golden.  However, I was still not able to get this working.  After another hour or two of beating my head against the wall, and of bothering both Chad Myers and Jeremy Miller (father of StructureMap).  I finally realized what was going on.  When MSTest runs, it sets the current working directory to c:\<InstallBase>\Microsoft Visual Studio 9.0\Common7\IDE because that is where MSTest runs.  Because of this StructureMap was having issues finding the .config file (before you jump to SM as being the issue, think again.  I have done this before both with NUnit and MBUnit and both work fine).

Alas after 4+ hours into my journey I threw in the towel.  I decided that using the StructureMap.config file method with MSTest was just too much friction.  So again with the help of Jeremy and Chad (do not know who was actually typing in the IM window as they were both there) I decided to solve the problem another way.

Here is what I did to get everything working:

  1. Download and use StructureMap 2.4.9 (aka 2.5)
  2. No-longer used the StructureMap.config file.... bye bye my friend
  3. Setup StructureMap to read my config info from my app.config
    1. This involves moving my config info to the app.config
    2. Telling StructureMap how to read from the app.config (blog post on this soon)
  4. Swore over and over again at the pain that MSTest caused me.

In intend to create a few more detaild posts on this over the next few days and will links to them here.

Hope this helps someone.

Till next time,

[----- Remember to check out DimeCasts.Net -----]



Comments

Craig Stuntz said:

How to do this with MSTest.

1.  In Solution Explorer, right-click the Solution (not the Project).

2.  Click Add, New Item

3.  In Categories, select Test Run Configuration

4.  Now choose the Test Run Configuration item and add it to your project

5.  In Solution Explorer, double-click the Test Run Configuration you just created

6.  Click the Deployment item

7.  Add your config file as a deployed file (or deploy the entire folder which contains it, if appropriate)

This took me a little while to figure out, but I'm in a similar situation and it does work for me.

# July 23, 2008 11:03 AM

Derik Whittaker said:

@Craig,

That kinda worked.  but the major underlying issue is that by default MSTest's default pathing is not where you would expect it.

# July 23, 2008 11:10 AM

jdn said:

It copies things to the oddly named folders because on that instance of running the test, those are exactly the files being tested, plus that is where it puts all of the logs, reports, and whatnot.

# July 23, 2008 12:41 PM

Simon said:

I have to use MSTest as it is a org standard but it suxs

so with this code

#if NUNIT

using TestClass = NUnit.Framework.TestFixtureAttribute;

using TestMethod = NUnit.Framework.TestAttribute;

using TestCleanup = NUnit.Framework.TearDownAttribute;

using TestInitialize = NUnit.Framework.SetUpAttribute;

using ClassCleanup = NUnit.Framework.TestFixtureTearDownAttribute;

using ClassInitialize = NUnit.Framework.TestFixtureSetUpAttribute;

using Assert = NUnit.Framework.Assert;

#else

using Microsoft.VisualStudio.TestTools.UnitTesting;

#endif

and a little project/solution config and a compilation constant i can use nunit on my machine but other people can still use MSTest on their machines.

# July 23, 2008 8:48 PM

Derik Whittaker said:

@Simon,

I am going to give this a spin tomorrow.  If it works, you rock.  But you have to be careful to use the 'classic' syntax with NUnit an not the new fluent syntax.

# July 23, 2008 9:52 PM

Simon said:

@Derik

Are talking about the “Assert.That” syntax?

If so since these eventually throw an exception (NUnit.Framework.AssertionException in this case) you can us nunits assert from mstest (or the other way around).

So perhaps the usings should be

#if NUNIT

using TestClass = NUnit.Framework.TestFixtureAttribute;

using TestMethod = NUnit.Framework.TestAttribute;

using TestCleanup = NUnit.Framework.TearDownAttribute;

using TestInitialize = NUnit.Framework.SetUpAttribute;

using ClassCleanup = NUnit.Framework.TestFixtureTearDownAttribute;

using ClassInitialize = NUnit.Framework.TestFixtureSetUpAttribute;

#else

using Microsoft.VisualStudio.TestTools.UnitTesting;

#endif

using NUnitAssert = NUnit.Framework.Assert;

using MsAssert = Microsoft.VisualStudio.TestTools.UnitTesting.Assert;

# July 23, 2008 10:32 PM

Jarle Nygård said:

I don't see the big problem... I'd never used a test framework before MSTest and I spent maybe 10 minutes googling for the answer (read first comment from Craig). Not a big deal, IMO.

There are other far worse issues that I'm having; specifically when I debug a test... Now that is bad. This is just peanuts IMO.

# July 24, 2008 4:00 AM

Alex McMahon said:

I had a similar problem using MSTest for integration tests of a CAB application. CAB looked in the BaseDirectory which MSTest set as "C:\\Program Files\\Microsoft Visual Studio 9.0\\Common7\\IDE\\" Whereas in reality it should look in the current Directory.

to get around this I put a nasty hack in the test:

AppDomain.CurrentDomain.SetData("APPBASE", Environment.CurrentDirectory);

Be careful to set it back afterwards otherwise your other tests may be affected

# July 24, 2008 4:20 AM

Derik Whittaker said:

@Jarle,

MSTest in terms of syntax is not an issue.  It is how it goes about executing the tests.  If you take a look at how other frameworks do it such as NUnit, MBUnit, xUnit they all do it the same.  MSTest is the one that differs

# July 24, 2008 6:11 AM

Dew Drop - July 24, 2008 | Alvin Ashcraft's Morning Dew said:

Pingback from  Dew Drop - July 24, 2008 | Alvin Ashcraft's Morning Dew

# July 24, 2008 1:28 PM

Brian Johnston said:

The default directory is a bug in VS2008.  In VS2005 it worked correctly...set this in you class initialize (MyClassInitiaize or whatever the code generate method is)...

AppDomain.CurrentDomain.SetData("APPBASE", Environment.CurrentDirectory)

The folders are created so that at anytime you can pull up a previous test run to view the DLL's as they were at the time of the test and what tests passed or failed.   Very handy if you work on a legacy app like mine and need to go back a few days to see what some junior programmer changed in order to figure out why a test failed today that you can now show passed 2 days ago.

# July 24, 2008 8:14 PM

Daniel said:

Yes, I had a lot of problems with MsTest the first times I was testing custom configuration sections and the alike. But once you figure out what it going on (and it's a lot, that's why it is soooo slow) you are fine.

@Brian

I think they use that strategy to have some sort of "isolation" between test run. In order to find out why code is broken I think you are better off using SCM...

# July 25, 2008 8:30 AM

BlogCoward said:

Developer Friction is Relative

# July 26, 2008 7:33 PM

Dave Schinkel said:

Just use NUnit and TestDriven.NET, that's ALL YOU NEED.

# July 28, 2008 8:56 PM

Constantijn Enders said:

Goto to the testconfig and select user-defined scheme.

At prefix text: type TestDir

Uncheck Append date-time stamp.

In the pre-build event command line of the test project

use the following line

RD "..\..\..\TestResults"  /q /S

Now the test will always build to testresults\testdir

Have fun

Constantijn

# August 4, 2008 11:31 AM

Dave said:

FYI...I hate MSTest too, but the issue of the base directory being "C:\\Program Files\\Microsoft Visual Studio 9.0\\Common7\\IDE\\" was introduced in VS2008 but fixed in VS2008 SP1.

It took me hours today before I'd discovered it was a bug and SP1 fixed it...

# August 21, 2008 8:05 PM

Leave a Comment

(required)  
(optional)
(required)  

Enter the numbers above:
Add

About Derik Whittaker

Derik is a .Net Developer/Architect specializing in WinForms working out the northern suburbs of Chicago. He is also believer and advocate for Agile development including SCRUM, TDD, CI, etc.

When Derik is not writing code he can be found spending time with his wife and young son, climbing on his bouldering wall, watching sports (mostly baseball), and generally vegging out. Check out Devlicio.us!

Our Sponsors

Red-Gate!