How not to write unit tests

In doing some major refactoring of code recently I was extremely glad we had unit test coverage in place.  Unfortunately there were some fairly problematic issues in the implementation of the unit tests that ended up costing a lot more time than it should.

The problem was two-fold.  Firstly there was a mismatch of custom hand-crafted stub classes and automatically mocked interfaces.  Secondly there was no consistency over the level of dependencies that were stubbed.  In some cases "true" (i.e. non-stubbed) implementations of dependencies were used three or four dependencies deep, and others only the direct dependency was mocked.

The combination of this meant it was really difficult to update the unit tests as part of the refactoring.  Each time a test failed I had to check whether it was due to a manually crafted stub that needed to be revised, a top level dependency that needed to be updated, or a dependency deep in the dependency tree that needed to be updated.

If the unit tests were configured appropriately, the only time I would need to update the unit tests would have been when a direct dependency changed.

What should have been done?

One of the more important features of Unit Tests is isolation and repeatable consistency of testing.  This ensures your unit test is appropriately testing the code you are targeting, and not the multitude of dependencies that the code may have.

If a direct dependent class has a nested dependency that is modified but the direct dependency retains the same expected behaviour, then you don't need to change your unit test.  Without this separation, major refactoring of your code becomes far more labourious, as you need to both refactor your code as well as all of the unit tests that have a dependency on the altered code at any level.

For this reason mocking or stubbing the direct dependencies of a method has become standard practice when working with unit tests.  Back in the old days this used to mean creating custom stub classes for all your dependencies.  This is one reason why Unit Testing was so often implemented as full end-to-end tests.
These days there is no excuse, Tools like Moq and RhinoMocks provide the ability to automatically create mocked versions of your dependencies, return appropriate objects based on the the calling parameters, and confirm that the services were called with the expected parameters.  Using these tools allows you to focus on testing the "unit of code" without worrying about what dependencies of dependencies might be doing.

Given an arbitrary method that calculates the cart total cost with a discount based on the total cost and number of items purchased.

        public decimal CalculateCart(Item[] items, IDiscountCalculator discountCalculator)
        {
            decimal total = 0;

            foreach (var item in items)
            {
                total += item.Cost;
            }

            var discountedAmount = discountCalculator.calculate(total, items.Length);

            return total - discountedAmount;
        }

IDiscountCalculator has its own unit tests to verify its behaviour, so we don't want to be testing this as part of the cart calculation.  We also don't want our CalculateCart to fail if our discountCalculator algorithm changes - as long as we trust the discountCalculator is going to work we don't need it to do the actual calculation and can return an arbitrary value.

public void TestCalculateCart()
{
    var cart = new Cart();
    Mock<IDiscountCalculator> discountCalculator = new Mock<IDiscountCalculator>();
           
    //arrange
    var expectedDiscount = (decimal)2;
    var expectedOriginalTotal = 10 + 10;
    var expectedDiscountedTotal = 10 + 10 - (decimal)2;

    var items = new Item[] {new Item() {Cost = 10}, new Item() {Cost = 10}};
    discountCalculator
        .Setup(x =>
            x.calculate(It.IsAny<decimal>(), It.IsAny<int>())
        )
        .Returns(expectedDiscount);

    //act
    var total = cart.CalculateCart(items, discountCalculator.Object);

    //assert
    //confirm the expected total is returned
    Assert.AreEqual(expectedDiscountedTotal, total );
           
    //confirm the discount calculator was called with the appropriate input
    discountCalculator.Verify(mock =>
        mock.calculate(It.Is<decimal>(x => x.Equals(expectedOriginalTotal)), It.Is<int>(x => x.Equals(2)))
        , Times.Once
        );

}

Now no matter what the implementation of DiscountCalculator, as long as we trust our DiscountCalculator is behaving (i.e through valid unit tests) then we don't need to change our unit test for the Cart calculation.

If our cart calculator was extended to throw an InvalidDiscountException in the event the discount is great than the total, we can arrange the discountCalculator to return 12, and assert that the exception was thrown without changing our 'happy path test' and with the same level of confidence.

Proper isolation and repeatability means refactoring is far simpler and your tests become less brittle.

The difference between bad code and good code

A colleague asked for some advice today on a project that he inherited (which I am extending with a separate module incidentally).

The issue was related to the usage of Entity Framework in the code that he had to maintain, and he needed some advice on how to proceed.  The problem was that the service layer was calling the repository multiple times, but each repository method was wrapped in a separate unit of work.
e.g.

public void DeleteEntity(int entityID)
{
    using (var context = EntityContext())
    {
        var entity = context.Postings.SingleOrDefault(p => p.entityID == entityID);
        context.Entities.DeleteObject(entity);
        context.SaveChanges();
    }
}
and
public Entity GetPosting(int entityID)

{
    using (var context = EntityContext())
    {
        return context.Entities.FirstOrDefault(p => p.entityID == entityID);
    }
}

This caused two problems for the developer, who needed to perform a complex action in his service that referenced multiple repository calls.

1. He had no control over the transactional scope for the repository methods
2. Each operation was on a separate EF context, so the service could not load and entity, edit it, and then save the changes (unless the repository was designed for disconnected entities, which it wasn't).

From a maintainability and testability point of view this was also a very poor design, as the repository methods created instances of dependency object (the service method also created instances of the repositories, making the services inherently untestable).


The version of this design that I implemented for my component follows a similar service/repository/entity pattern, but is implemented in a far more testable and robust manner.

The first improvement over the legacy design is in the dependency management
My service accepts a context and all required repositories in the constructor, and my repositories accepts a context, which allows for improved maintainability (all dependencies are described) and testability (all dependencies can be mocked).  This also allows us to use dependency injection/IoC to create our object instances.

The second improvement was in the Unit of Work design.
Rather than have each repository method as a single unit of work, the service methods are the units of work, so any action within the service uses the same context (as it is passed as a dependency to the repositories that the service uses), and each service call acts as a Unit of Work, calling SaveChanges at the end of the service to ensure that the changes act under a single transaction.
There are limitations to this design (your public service methods become an atomic transaction and you should not call other public methods from within another method) but for simplicity and maintainability it is a pretty good solution.

Below is a simple example of the design I am using, preserving maintainability, testability, and predictability.  I'm not saying it is necessarily the best code around, but it solves a number of issues that I often see in other developers code.

public class HydrantService
{
  public HydrantService(HydrantsSqlServer context, EFRepository<Hydrant> hydrantRepository, EFRepository<WorkOrder> workOrderRepository, EFRepository<HydrantStatus> hydrantStatusRepository)
  {
    _context = context;
    _hydrantRepository = hydrantRepository;
    _workOrderRepository = workOrderRepository;
    _hydrantStatusRepository = hydrantStatusRepository;
  }
  public void createFaultRecord(WorkOrder order)
  {
    HydrantStatus status = _hydrantStatusRepository.GetSingle<HydrantStatus>(x => x.StatusCode == "Fault"); //_context.HydrantStatuses.Where(x => x.StatusCode == "Fault").FirstOrDefault();
    order.Hydrant.HydrantStatus = status;
    _workOrderRepository.Add(order);
    _context.SaveChanges();
  }
}

  public class EFRepository<T>
 {
 public EFRepository(IDbContext context)
 {
    _context = context;
  }   public virtual ICollection GetAll()

  {
    IQueryable query = _context.Set();
    return query.ToList();
  }
}

Mocked Repository and Generic Constraints

So, a productive couple of days - three issues resolved.

Reinstated a Repository - Moq cannot mock EF IDbSet so I decided that bringing the repository back would be a good idea. Example unit test below:

[TestMethod]

public void ListInventoryTest()

{

Mock<IDbContext> context = new Mock<IDbContext>();

Mock<AssetRepository> assetRepository = new Mock<AssetRepository>(context.Object);

Mock<StockpileRepository> stockpileRepository = new Mock<StockpileRepository>(context.Object);

List<Asset> assets = new List<Asset>();

assets.Add(new Asset() { AssetId = 1 });

assets.Add(new Asset() { AssetId = 2 });

Stockpile stockpile = new Stockpile() { StockPileId = 1, Assets = assets };

//assetRepository.Setup(x=>x.GetSingle(It.IsAny<expression<func<asset, bool="">>>())).Returns(asset);<expression<func<asset,>

stockpileRepository.Setup(x => x.GetSingle(It.IsAny<Expression<Func<Stockpile, bool>>>(), It.IsAny<IEnumerable<string>>(), It.IsAny<bool>())).Returns(stockpile);

var inventoryService = new InventoryService( stockpileRepository.Object, assetRepository.Object );

List<Asset> rv = inventoryService.ListInventory("Hailes", "Jita01");

Assert.IsNotNull(rv);

Assert.IsTrue(rv.Count == 2);

}

Then, now that I had a repository I could add a Null Object pattern solution to my repository, which was a bit tricker than I expected. In order to create a new T in the generic method, I needed to include a generic constraint to ensure T had a blank constructor.

public virtual T GetSingle<T>(Expression<Func<T, bool>><func whereClause, IEnumerable<string> customIncludes = null, bool overrideDefaultIncludes = false) where T : new()<string><func

{

IQueryable query = (IQueryable)this.ApplyIncludesToSet(customIncludes, overrideDefaultIncludes);

T val = query.SingleOrDefault(whereClause);

if (val == null)

{

val = new T();

}

return val;

}

The final issue I resolved was picking an appropriate lifetime manager for the EF DbContext when running in an 'application' context. Using the PerResolveLifetimeManager ensures that when resolving a class, any common Dependency in the entire resolve pat are shared - this means that a service with two repository dependencies, which both depend on a dbContext, will both use the same dbContext when the service is resolved - yay. This does exactly what I want it to, each operation should use a new service instance, which will use a single dbContext across all repository actions within that method.

So yeah, productive (and thanks to @wolfbyte for the generic constraint tip).

Next job to flesh out my unit tests, and continue on the functionality, as this covered the majority of my architecture issues.

Unit Testing, Moq, EF, and Repositories

　
Well, I have just started a small (8-12 week / 1 resource) project using an unfinished version of our in-house framework for some parts of it. In the process I want to ensure that I integrate some key design patterns (null object, repository, and unit of work) and full unit testing on the service implementation. This will hopefully help alleviate the pain of working with DotNetNuke, cross-application dependencies, and webforms.

So my first step was to property expose services from the dependent application as this is a major point of failure in other systems that use this application, which was pretty straight forward as the application design is not too bad. As this is a shared dependency on the DotNetNuke instance, I did not need to expose this as a WCF service, but could easily change it in the future if necessary. The new service interface will help prevent changes in the core application from breaking the dependent application, as any changes will be reflected as build failures in the service class, highlighting this to the developers and ensuring they either make the change to not break the interface, or let all consumers of this service know there is a breaking update and plan appropriate changes. This is a key issue encountered when services and application references are not well defined, and has caused a number of deployment issues at my current client.

The guts of this post however is to discuss my plan for unit testing, and how I had to rethink my previous statement of going ‘repository-less’. I previously discussed the removal of the repository from the framework and using the DbSet functionality in the EF context as the repository pattern. This worked really well, until I decided to do some unit tests.
I decided to use a mocking library in my unit tests specifically to ensure I was performing appropriately isolated tests, and to reduce the impact of managing test data. I had previously looked at Moles (Microsoft stubbing tool), but it always seemed so cumbersome and confusing, so I picked up Moq instead. I really like the Moq usage pattern, and so I thought it would be a good fit.

So, the plan was to use Moq to create mocks of the repository functions that act in predictable and repeatable ways, which means we can run the service and test that the service behaves as we expect.

An example is given below – in this example I created a service to get a list of ‘stations’ from the dependent application. Since I am testing my service, I want to Mock the dependent application service to act predictably, so I can ensure that my service acts the way I want it to (we are not performing end-to-end integration testing, so we don’t want to rely on the dependent application succeeding or failing at this point)

//when we call ‘GetStations’ with a parameter of 0, our mocked service throws an exception – I know the dependent service reacts in this way, so I can ensure this is integrated in my test
_samsServiceMoq.Setup(x => x.GetStations(0)).Throws();
//when we call ‘GetStations’ with a parameter of -1, our mocked service returns no results
_samsServiceMoq.Setup(x => x.GetStations(-1)).Returns(new List());
//when we call ‘GetStations’ with a parameter of 1, our mocked service returns a list with one item in it
_samsServiceMoq.Setup(x => x.GetStations(1)).Returns(new List() { new Unit(){ UnitID = "100" } });
　
UserService target = new
UserService(_samsServiceMoq.Object); //create an instance of my service, and pass in the mocked dependent service
List actual1;
List actual2;
List actual3;
actual1 = target.GetStations(-1); //execute the service method with the specified parameter
actual2 = target.GetStations(1); //execute the service method with the specified parameter
actual3 = target.GetStations(0); //execute the service method with the specified parameter
_samsServiceMoq.VerifyAll();
//check whether the mocked service methods were called in the execution of our tests – this is useful to ensure that your service method is calling the expected mocked method with the expected parameters.
//check the results from the service to ensure they match what you expect (based on the response from the mocked service)
Assert.IsNotNull(actual1);
Assert.IsNotNull(actual2);
Assert.IsNotNull(actual3);
Assert.AreEqual(0, actual1.Count);
Assert.AreEqual(1, actual2.Count);
Assert.AreEqual("100", actual2[0].UnitID);
Assert.AreEqual(0, actual3.Count);

The above example shows how you can configure a test without worrying about the dependent services, so you can test only the functionality in your service. You will also note that the service itself needs to be designed so that all dependencies are passed to the service, instead of created in the service (this is a key point in ensuring testability of components, all dependencies must be passed to the object). If we did not do this, we could never mock the dependent service, which means we would need to set up the test to ensure the dependent service responds appropriately (configure the dependency, and know/configure sample data that the dependency will respond to).
This works really well, I can test my (admittedly very simple) service without caring about configuring the dependent service. However doing the same thing on an EF repository instead of the dependent service does not work so well. The code below should work, but doesn’t due to limitations in EF/C#/Moq.

_hydrantContextMoq.Setup(x=>x.Hydrants).Returns(_hydrantDbSetMoq.Object);
_hydrantDbSetMoq.Setup(x => x.ToList()).Returns(new List() { new Hydrant() });
HydrantService service = new HydrantService(_hydrantContextMoq.Object);
List actual;
actual = service.GetHydrantList();
_hydrantContextMoq.VerifyAll();
_hydrantDbSetMoq.VerifyAll();
Assert.IsTrue(actual.Count == 1);

Here I am mocking my DbContext to return a mocked IDbSet, and mocking the IDbSet.ToList() to return a list of Hydrants with 1 item. This way I can test my service so that calling getHydrantList on my service returns the single length list. Unfortunately, IDbSet.ToList() is not a mockable method (it is actually an extension method) which means it is not possible to set up a mock for this method. Since my service is using this method, I cannot test my service in isolation of the database.

This is where the Repository comes in. Instead of using the IDbSet.ToList() directly, I would use a Repository GetAll() method which abstracts the call to the underlying DbSet method. As the repository is just another dependency on the service, we can mock this instead of the EF IDbSet, and hence have an appropriately testable service. We will also then have the ability to ensure that the repository supports the null object pattern, so a call to the IDbSet that may return null (such as a find() with an invalid key) can return an appropriate null object to the service, so the service, and all clients, know it will never receive a null as the result of a service operation.

So, big backtrack on the framework repository, and big kudos to Moq for making testing easier (at least for my simple examples so far).