Derik Whittaker

Let me just say this slowly.... Simply because your tests 'pass' does not mean your code has been proven correct or that it works.

I know from time to time the notion that 'because I test, my code is solid' floats around the blog-a-sphere.  Well, I am here to say, that is a 100% lie.

Here is a classic example.  Today I was writing a few tests to ensure that our data transaction management was working as expected.  In my test sweet I was going to have 2 tests:

1. Transaction_EnsureRollbackWorks()
2. Transaction_EnsureCommitWorks()

When I setout to write the test for the rollback all was going good.  My test passed and I was happy (kid in a candy store kinda happy :) ).  It was not till I was working on my test for the commit that I was a sad (like a kid who was told he could not eat candy in that candy store :( ).

When I started to test my commit logic I was not able to pull the newly committed data from the database.  I was perplexed.... what was the issue.  I decided to take a look at the rollback code (I had hijacked most of the logic from that test).  When I started to step through the code I noticed that something was missing.  My call to SubmitChanges() (we are using Linq). 

Here is the code for the test:

[Test]
public void Transaction_EnsureRollbackWorks( )
{
    RepositoryOverride repository = new RepositoryOverride();
    Domain.Entities.AccessKey accessKey1 = new Domain.Entities.AccessKey { AccessKeyActionTypeID = 1, Key = "123456789", PIN = "ertyui", ProgramID = 1, RecruitingID = 2, UserID = 3 };
    Domain.Entities.AccessKey accessKey2 = new Domain.Entities.AccessKey { AccessKeyActionTypeID = 1, Key = "56789", PIN = "ertyui", ProgramID = 1, RecruitingID = 2, UserID = 3 };
    
    repository.BeginTransaction();

    repository.DBContextInstance().AccessKey.InsertOnSubmit( accessKey1 );
    repository.DBContextInstance().AccessKey.InsertOnSubmit( accessKey2 );
    // code was not in the original test repository.DBContextInstance().SubmitChanges();

    repository.RollbackTransaction();

    // Check to make sure the data is NOT in the DB
    var results = from ak in repository.DBContextInstance().AccessKey
                  where ak.Key == "56789"
                  select ak;

    Assert.That( results.Count(), Is.EqualTo( 0 ) );
}

If you notice before the repository.RollbackTransaction(); line there is NO call to SubmitChanges().  Because of this my roll back appear to work and then when I attempted to re-query the data it came up empty, which was the expectation.

The goal to any test is to clearly put your code though its paces, to prove it correct.  However, if you test is bad, or has logic issues you are not proving your code.

You have been forewarned Green Test != Proven Code

Till next time,