The example code is
int orderId = order.OrderID;
Order newOrder = _orderRepository.getAll(x=>x.orderID == orderId);
The above example is a bit contrived, there's a fair bit more that goes on but this code highlights the issue.
Now that I know what is going on this is realtively straight forward, but it is a bit counterintuitive when starting out.
The problem was in my repository, where I was using the DbContext DbSet property for each entity directly, instead of the DbSet.Local property. The difference between the two is that the root DbSet property contains all elements in their modified state (e.g. it contains the deleted order, with an updated state of Deleted), while the Local DbSet property (which is an IObservable of the root DbSet) has the entities in their 'current state' so if you delete an entity from the context it is removed from the Local DbSet.
I say this is counterintuitive because the only way to identify whether an item is deleted or not is through the root context Entry() method, you cannot base a query on the DbSet to exclude deleted items.
The solution is however fairly simple. Since I am using a Unit of Work pattern on the context, and my service methods are a single unit of work, I can use the Local DbSet for my repository actions without any issues down the line with disconnected or orphan entities, and I can do this without any modifications to my service.
So where all my repository queries used to use the code below as the base for all repository queries
I now simply base all my queries off
Now deleted items should not show up in my list of queries. I hope - I haven't had a chance to actually test it just yet.
Well, that was short lived - it seems as though using the Local context only works on previously loaded data, for instance you do a load, then delete an entity, then a load from the local context will not show the deleted item.
This is incredible frustrating as it means I need to know under what scenario I am 'loading' data in order to choose the right context to load from, and means I need to front-load all the entities I will be working with, then use the local context from that point on.
I am seriously thinking of switching to nHibernate over this one.
I have identified a possible solution, but I am concerned by performance implications
When performing a query on my context, I can use Linq to query the state of each entity in the resulting query and further filter the results.
query.Where(whereClause).ToList().Where(x=> ((DbContext)_context).Entry(x).State != System.Data.EntityState.Deleted ).ToList();
The two performance issues with this are
a) I need to 'ToList()' the query and then apply the state filter (otherwise EF will attempt to apply the filter to the SQL query, which it can't do). This is not ideal, but not critical, and I may be able to force the first Where() to resolve the EF part in another way to avoid the extra list creation.
and b) queries that return a large number of results will be impacted (potentially severely) since each entity will be inspected individually against the context change manager.
So perhaps an nHibernate implementation could wait if this does what I want it to without critical performance implications.