Wow, it's been over a week since my last blog post, and despite being excited about starting some Expanz work, I only got as far as downloading the tools and creating a sample model.
I was struck with two immediate things, the first being that creating a model called System breaks everything (well duh), and second that I had no idea how to 'regenerate' the server project after changing the model. I think this was caused by enabling the Linq to Sql model when creating my server which seemed to make things harder for no appreciable gain. I really did not have more than a few minutes to get familiar with the tools though, so I am not saying these are actually problems.
The last week has been crazy which is why I haven't really looked at the expanz platform, with a 7th birthday party, multiple bbqs, a sick kitty, and a weekend only SW:TOR beta invite on the weekend, followed by a Foo Fighters concert, a non-sleeping baby, and work 'heightened operation period' during the week. Hopefully the next weekend and week will be a bit more relaxing.
I haven't heard from Mr R&D (@csharpzealot) whether I have a project to work on with Expanz yet, but if not I'll definitely try and get my market game up and running on the Expanz platform over the weekend.
Thursday, December 1, 2011
Wednesday, November 23, 2011
Expanz
I was invited to a presentation by Expanz today to go through their development platform and how it can improve our company's development strategies.
From a technical standpoint it is quite an impressive showing, matching rich model driven development with a robust and scalable application platform, and cross-platform rich thin client support.
The three key features that I think will make it a game changing experience for application development workshops are the:
From a technical standpoint it is quite an impressive showing, matching rich model driven development with a robust and scalable application platform, and cross-platform rich thin client support.
The three key features that I think will make it a game changing experience for application development workshops are the:
- Rich modeling experience and customised data types: being able to define an 'email' data type with built in validation for example. This improves consistency between applications and provides a robust library of types that can be used for any application. This is also designed to flow through the business layer and UI, so an email field will always behave consistently throughout your applicaiton.
- Excellent designer and UI generation templates: the coding is left for the business rules, while the model and UI can be completed by the designers / analysts.
- Stable and scalable application server: the developer does not ned to worry about the plumbing of authentication, session management, etc. This is probably the largest differentiation factor between expanz and most application development frameworks. The design allows for load-balancing, sticky session management and multi-tier application hosting, locally or in the cloud, without the developer having to worry about how this all fits together.
Very impressive in all, and I am definitely looking forward to actually putting the theory into practice. I don't know how this will fit into our company due to client expectations, licensing, etc, but if we can get a foothold it could be a gamechanger for us. It would allow us to build up on successive projects to improve future productivity, and at some point allow for internal projects to be delivered quickly and easily enough to position ourselves as solution vendors, not just consultants.
This is the sort of 'dream' that I have been aiming for with the framework development I have been doing on the side for the last 9 months, so getting hold of this will exciting (as well as disappointing that I won't really be continuing with that).
For more info on Expanz, visit http://www.expanz.com/
Tuesday, November 22, 2011
Ads Redux
According to my statistics for the last couple of weeks I will be earning an estimated 8c per 1000 requests. Given that my traffic is <500 views per month I'm looking at over 2 years to make my first dollar.
This highlights the volume of trafic needed to actually make any money from blogging, and while I'm sure there are ways to tweak this, and potentially using other ad services, it still takes a hell of a lot of ad views to make money.
So bye bye ads, it was fun while it lasted.
This highlights the volume of trafic needed to actually make any money from blogging, and while I'm sure there are ways to tweak this, and potentially using other ad services, it still takes a hell of a lot of ad views to make money.
So bye bye ads, it was fun while it lasted.
Moq Testing and Lambda Equivalence
I have been steadily improving my unit tests as my experience with Moq improves, but I have now encountered an issue that has thrown me a little.
To recap the app design, I am using EF Code First for the ORM, accessed via Repository classes that expose (mockable) operations on the repository. I then have a service layer that performs the business operations across multiple repositories.
My unit testing is focusing on the Service Layer methods, with (partially) mocked repositories. My initial tests were pretty simple setup the mocks, perform the action, assert the response. However as I became more familiar with the Moq Verify function I was able to improve the way I tested by actually verifying that my repositories were being accessed with the expected parameters.
An example is that when my service method GetItem(itemId) is called, I expect my repository method
to be called with a value of itemId.
I can then test using the following
The test above ensures that calling GetItem on my service calls the GetSingle on my repository with the expected parameter, and calls it only once. It is a very basic test for a simple method, but is a good example.
The issue is that my repository is a bit more complex than I have shown, where the Get methods actually accept Lambda expressions, so we use
instead of just
Now this actually works IF we are using static/const variables in the collection, for example
Item class has a static const ITEM_TYPE_BOX = "Box"
and my service class calls
If we are passing variables to the service method, and that variable is used to create the repository expression, the test fails as the Verify method cannot find a matching execution of the method.
My service method accepts a string itemType
and my unit test uses the following verify method
The test code above fails because while the two lamdba expressions are functionallity identical, they are not expressively equal, so the Moq verify function comparison fails. Drilling down into the issues I have found that the expression specifically includes the namespace of the method that creates the expression when using local types, which means creating the lambda expression in one class, and comparing it to an identical expression created in another class will always fail (you can see this if you create an expression and call ToString() on the expression). The reason it works for constant comparisons is that there are no local variables to compare to.
I cannot remove the dependency on lambdas in my repository as this forms the core of the repository flexibility, but I have identified one way of keeping my unit tests robust while overcoming this issue.
It is possible to expose methods in the service that create an expression object which can be used by both the unit test verify, and the service method. The issue with this is that it is somewhat cumbersome since you need to expose a method for each of the expression combinations your service is using. In many cases this will be relatively straight forward, but it is still a fair bit of effort.
This is pretty disappointing as the unit testing was going quite well up to this point.
To recap the app design, I am using EF Code First for the ORM, accessed via Repository classes that expose (mockable) operations on the repository. I then have a service layer that performs the business operations across multiple repositories.
My unit testing is focusing on the Service Layer methods, with (partially) mocked repositories. My initial tests were pretty simple setup the mocks, perform the action, assert the response. However as I became more familiar with the Moq Verify function I was able to improve the way I tested by actually verifying that my repositories were being accessed with the expected parameters.
An example is that when my service method GetItem(itemId) is called, I expect my repository method
to be called with a value of itemId.
I can then test using the following
The test above ensures that calling GetItem on my service calls the GetSingle on my repository with the expected parameter, and calls it only once. It is a very basic test for a simple method, but is a good example.
The issue is that my repository is a bit more complex than I have shown, where the Get methods actually accept Lambda expressions, so we use
instead of just
Now this actually works IF we are using static/const variables in the collection, for example
Item class has a static const ITEM_TYPE_BOX = "Box"
and my service class calls
If we are passing variables to the service method, and that variable is used to create the repository expression, the test fails as the Verify method cannot find a matching execution of the method.
My service method accepts a string itemType
and my unit test uses the following verify method
The test code above fails because while the two lamdba expressions are functionallity identical, they are not expressively equal, so the Moq verify function comparison fails. Drilling down into the issues I have found that the expression specifically includes the namespace of the method that creates the expression when using local types, which means creating the lambda expression in one class, and comparing it to an identical expression created in another class will always fail (you can see this if you create an expression and call ToString() on the expression). The reason it works for constant comparisons is that there are no local variables to compare to.
I cannot remove the dependency on lambdas in my repository as this forms the core of the repository flexibility, but I have identified one way of keeping my unit tests robust while overcoming this issue.
It is possible to expose methods in the service that create an expression object which can be used by both the unit test verify, and the service method. The issue with this is that it is somewhat cumbersome since you need to expose a method for each of the expression combinations your service is using. In many cases this will be relatively straight forward, but it is still a fair bit of effort.
This is pretty disappointing as the unit testing was going quite well up to this point.
Thursday, November 17, 2011
SOA and how much information to disseminate
I was involved in a meeting today that made me think a little bit about how much detail is, and should be, shared when defining services and consumers.
To put this into context, there is Company A that has created application X (used by a number of different organisations) which needs to access common functionality from Company A, B and C. As Company A owns application X, as well as being one of the service providers, they are designing the service interface to be exposed by Company A, B and C.
Company A will create the Service Definition (preferably with input from all parties) as the primary owner of the system, but a burden of responsiblity has been placed on the other companies who will be implementing the service to provide details of their implementation in a supporting definition. This 'Consumer Definition' is intended to document the processes and flows that are followed in the implementation of the service interface, focusing on non-logical functionality such as error handling, logging, issue escalation and monitoring.
The two companies raised some concerns about this additional requirement, centered around two facts: Why should we provide this information; and as long as we conform to the interface why does it matter?
Both these concerns are valid, but I propose that providing such information is invaluable to good SOA architecture. While the service definition is the only thing that the services require, providing the additional information simply increases the ability for the users of the service to understand what is happening in each implementation. A contrived example would be if Company B's service is down, the administrators of the consuming application will know that an issue ticket should have been created in Company B's application fault log, and can contact Company B to verify the issue and obtain an ETA.
In a previous post I discussed some of the SOA points made by Steve Yegge from his time at Amazon, and this to me is clearly the sort of thing that provides massive value in SOA designs. In defining clear operational contexts for the services as well as the services themselves you can provide a much more meaningful and robust SOA environment.
To put this into context, there is Company A that has created application X (used by a number of different organisations) which needs to access common functionality from Company A, B and C. As Company A owns application X, as well as being one of the service providers, they are designing the service interface to be exposed by Company A, B and C.
Company A will create the Service Definition (preferably with input from all parties) as the primary owner of the system, but a burden of responsiblity has been placed on the other companies who will be implementing the service to provide details of their implementation in a supporting definition. This 'Consumer Definition' is intended to document the processes and flows that are followed in the implementation of the service interface, focusing on non-logical functionality such as error handling, logging, issue escalation and monitoring.
The two companies raised some concerns about this additional requirement, centered around two facts: Why should we provide this information; and as long as we conform to the interface why does it matter?
Both these concerns are valid, but I propose that providing such information is invaluable to good SOA architecture. While the service definition is the only thing that the services require, providing the additional information simply increases the ability for the users of the service to understand what is happening in each implementation. A contrived example would be if Company B's service is down, the administrators of the consuming application will know that an issue ticket should have been created in Company B's application fault log, and can contact Company B to verify the issue and obtain an ETA.
In a previous post I discussed some of the SOA points made by Steve Yegge from his time at Amazon, and this to me is clearly the sort of thing that provides massive value in SOA designs. In defining clear operational contexts for the services as well as the services themselves you can provide a much more meaningful and robust SOA environment.
Tuesday, November 15, 2011
EF Deleted Items Issue
I noticed an issue in one of my service methods whereby a record I deleted showed up in a subsequent query within a single unit of work.
The example code is
int orderId = order.OrderID;
_orderRepository.delete(order);
Order newOrder = _orderRepository.getAll(x=>x.orderID == orderId);
The above example is a bit contrived, there's a fair bit more that goes on but this code highlights the issue.
Now that I know what is going on this is realtively straight forward, but it is a bit counterintuitive when starting out.
The problem was in my repository, where I was using the DbContext DbSet property for each entity directly, instead of the DbSet.Local property. The difference between the two is that the root DbSet property contains all elements in their modified state (e.g. it contains the deleted order, with an updated state of Deleted), while the Local DbSet property (which is an IObservable of the root DbSet) has the entities in their 'current state' so if you delete an entity from the context it is removed from the Local DbSet.
I say this is counterintuitive because the only way to identify whether an item is deleted or not is through the root context Entry() method, you cannot base a query on the DbSet to exclude deleted items.
The solution is however fairly simple. Since I am using a Unit of Work pattern on the context, and my service methods are a single unit of work, I can use the Local DbSet for my repository actions without any issues down the line with disconnected or orphan entities, and I can do this without any modifications to my service.
So where all my repository queries used to use the code below as the base for all repository queries
IQueryable query = _set.AsQueryable(); //_set is the appropriate DbSet for the entity T in the context
I now simply base all my queries off
IQueryable query = _set.Local.AsQueryable();
Now deleted items should not show up in my list of queries. I hope - I haven't had a chance to actually test it just yet.
*edit*
query.Where(whereClause).ToList().Where(x=> ((DbContext)_context).Entry(x).State != System.Data.EntityState.Deleted ).ToList();
The example code is
int orderId = order.OrderID;
_orderRepository.delete(order);
Order newOrder = _orderRepository.getAll(x=>x.orderID == orderId);
The above example is a bit contrived, there's a fair bit more that goes on but this code highlights the issue.
Now that I know what is going on this is realtively straight forward, but it is a bit counterintuitive when starting out.
The problem was in my repository, where I was using the DbContext DbSet property for each entity directly, instead of the DbSet.Local property. The difference between the two is that the root DbSet property contains all elements in their modified state (e.g. it contains the deleted order, with an updated state of Deleted), while the Local DbSet property (which is an IObservable of the root DbSet) has the entities in their 'current state' so if you delete an entity from the context it is removed from the Local DbSet.
I say this is counterintuitive because the only way to identify whether an item is deleted or not is through the root context Entry() method, you cannot base a query on the DbSet to exclude deleted items.
The solution is however fairly simple. Since I am using a Unit of Work pattern on the context, and my service methods are a single unit of work, I can use the Local DbSet for my repository actions without any issues down the line with disconnected or orphan entities, and I can do this without any modifications to my service.
So where all my repository queries used to use the code below as the base for all repository queries
IQueryable
I now simply base all my queries off
IQueryable
Now deleted items should not show up in my list of queries. I hope - I haven't had a chance to actually test it just yet.
Well, that was short lived - it seems as though using the Local context only works on previously loaded data, for instance you do a load, then delete an entity, then a load from the local context will not show the deleted item.
This is incredible frustrating as it means I need to know under what scenario I am 'loading' data in order to choose the right context to load from, and means I need to front-load all the entities I will be working with, then use the local context from that point on.
I am seriously thinking of switching to nHibernate over this one.
*edit 2*
I have identified a possible solution, but I am concerned by performance implications
When performing a query on my context, I can use Linq to query the state of each entity in the resulting query and further filter the results.
query.Where(whereClause).ToList().Where(x=> ((DbContext)_context).Entry(x).State != System.Data.EntityState.Deleted ).ToList();
The two performance issues with this are
a) I need to 'ToList()' the query and then apply the state filter (otherwise EF will attempt to apply the filter to the SQL query, which it can't do). This is not ideal, but not critical, and I may be able to force the first Where() to resolve the EF part in another way to avoid the extra list creation.
and b) queries that return a large number of results will be impacted (potentially severely) since each entity will be inspected individually against the context change manager.
So perhaps an nHibernate implementation could wait if this does what I want it to without critical performance implications.
Monday, November 14, 2011
Javascript standards
Html5 and css3 are well on their way to becoming standardised and you can be fairly sure that when using these technologies your knowledge can be reused time and time again. Why then are there so many different bloody javascript libraries, all with their own syntax and base functionality. Every time I start looking to re-learn Javascript for web app development I feel like I am starting all over again.
Granted jquery seems to be leading the pack, but even libraries built on jquery decide to do their own thing with data sources and other core features more often than not.
Of course other languages are not immune to this, with a plethora of frameworks and tools in .net alone, but at least the basic syntax its the same and with .net you get the benefit of an excellent dev environment to help manage the differences. Each tool and library can usually work regardless of the other tools and libraries you are using as well, whereas in javascript if you find a nice calendar control for jquery and you are using yui you are SOL.
Ok rant over, I should go check out some javascript libraries to see if I like them enough to learn since I am lagging a bit in my skills. Twitterverse is all over #kendoui at the moment, and the demos of #knockoutJS I ran through a month or so ago were nice, so maybe I should start there.
Granted jquery seems to be leading the pack, but even libraries built on jquery decide to do their own thing with data sources and other core features more often than not.
Of course other languages are not immune to this, with a plethora of frameworks and tools in .net alone, but at least the basic syntax its the same and with .net you get the benefit of an excellent dev environment to help manage the differences. Each tool and library can usually work regardless of the other tools and libraries you are using as well, whereas in javascript if you find a nice calendar control for jquery and you are using yui you are SOL.
Ok rant over, I should go check out some javascript libraries to see if I like them enough to learn since I am lagging a bit in my skills. Twitterverse is all over #kendoui at the moment, and the demos of #knockoutJS I ran through a month or so ago were nice, so maybe I should start there.
Subscribe to:
Posts (Atom)