Thursday, February 27, 2014

NHibernate Cross-database Hacks

Cross-Database Queries


Ok, so this is a bit of a hack, but it does work.  Thanks to this which set me down the "right" path.

I work with a few legacy systems that have central databases for common information, and individual databases for application specific information.  The data is queried using joins across the databases to retrieve the data required.
A separate service model was introduced for the common data, but when performing filtered queries across both data sets the service model was not efficient (this is a greater issue of context boundaries that I won’t go into here).  To perform the queries that were previously performed using stored procedures using cross-database joins in nHibernate required a bit of a cheat.

nHibernate mappings have a “schema” property as well as the more commonly used “table” property.  By manipulating this schema property you can convince nHibernate to perform cross-database queries.  Setting the schema to “{database}.{schema}” any join query to that element will effectively use the full cross-database query syntax when converted to a SQL query.

Neat (but ultimately not very satisfying because it is not a very nice design).


Bigger Hack, run away


If the target database name is not known until runtime, you can even hack it more to support this.

During the configuration of the nHibernate session factory, you can make a few modifications that will allow you to update the schema property of an entity.  This is useful if you have a different ‘other’ database name for each environment (e.g. OtherDatabase-dev, OtherDatabase-prd).

First we appropriately generate the fluent configuration, and build it.
We then iterate through the class mappings.  Each persistentClass.MappedClass is the underlying POCO model object of the entity.
We check if this is one that we want to override the schema property for (IOtherDatabaseEntity is a simple blank interface, it could be done via naming convention or whatever)
And then update the schema property on the mapping
Finally we create the session factory from the modified config

var fluentConfiguration = config.Mappings(m =>
        m.FluentMappings
        .AddFromAssemblyOf()
        );
var builtConfig = fluentConfiguration.BuildConfiguration();

foreach (PersistentClass persistentClass in builtConfig.ClassMappings)
{
        if (
            typeof(IOtherDatabaseEntity)
                .IsAssignableFrom(persistentClass.MappedClass)
            )
        {
            persistentClass.Table.Schema = cdrName;
        }
}
                  
ISessionFactory sessionFactory = builtConfig
        .BuildSessionFactory();


Hacks away!

NHibernate + LINQPad

While looking at ways to assess and improve performance some nHibernate queries, I was frustrated with the tools at my disposal.

I will start with the point: I don't have access to nhprof.  It seems to be the gold standard for any nHibernate shop, but them's the breaks.

What I was doing:


It was painful to execute the entire codebase to run one or two queries and view the output sql, so I started writing integration-unit tests for query optimisation.  This was slightly better, but still required a change-compile-run-review process which was annoying.

What was I thinking:

I then remembered I have a personal LINQPad license that I hadn't used in a while and wondered if I could get it working.  I saw this which helped me on my way, but we don't use nHibernate.Linq, so the steps were a bit different.

The outcome was extremely useful however, and now I am free to tinker with my queries a lot more freely.

How I did it:

To start you need to add the references to nHibernate that you require (in my case NHibernate, Iesi.Collections, and FluentNHibernate).  You then add the references to your Domain / Models and mapping assemblies.

The next step is to create a hibernate.cfg.xml file with the appropriate configuration for your database.  Make sure you set show_sql=true in the configuration so LINQPad can display the generated SQL.

Then you can call the following code

var cfg = new Configuration().Configure(@"D:\linqpad\ahs\hibernate.cfg.xml");
var factory = Fluently.Configure(cfg)
                .Mappings(m =>
                {
                    m.FluentMappings.AddFromAssemblyOf();
                })
                .BuildSessionFactory();


using (var session = factory.OpenSession()){
    var site = session.QueryOver()
    .List();
}
and viola, you can now tinker with queries as you will, with immediate feedback on the generated SQL and execution time.


You can then save this as a query, or save the assembly references/using statements as a snippet to get you up and running quickly for new queries. 

Caveats:

This method only works with pre-compiled entity mappings, so if you intend to improve performance at the entity mapping layer you still need to do this through your application and export the assemblies for LINQPad to use.

Extensions:

LINQPad allows you to create an 'application config' file that is used when your inner assemblies require web/app.config sections.  Run the code:
AppDomain.CurrentDomain.SetupInformation.ConfigurationFile
to find the location of the file, and if it does not exist create it.  Note that unlike most .NET apps, this is not the LINQPad.exe.config, but LINQPad.config.  Enter any configuration you need into this file.  This can include the nHibernate config  instead of the separate file (but limits configuration flexibility).

This allows you to configure things like nHibernate 2nd level cache instances, such as memcache.  As long as you include the necessary libraries in the query references, and the configuration in the linqpad.config file this will work and provide even greater flexibility for performance analysis and testing.


Conclusion:

So there you go, a "poor man's" guide to nHibernate performance analysis, thanks to the ever awesome LINQPad.

Thursday, August 1, 2013

WinJS and RequireJS

I had been looking at improving my JavaScript skills and learning WinJS at the same time.  One of the key things I wanted to do was work better with larger projects and larger numbers of JavaScript files. 

RequireJS is great for that, but I had considerable trouble getting it to work with WinJS.  When I did get it working I then had trouble getting it to work properly. 

RequireJS is expected to be initialised with an include reference in the page html with a data-main tag, but this doesn't really work with the WinJS app lifecycle, so it took a bit of work to get it behaving the way I expected.

Eventually I got it working, and working with async initialisation code which was a key part of the initialisation process in my sample app.

So the basics of getting RequireJS working in WinJS is the following - I have probably made a million big JavaScript no-no's here, but as I said, still learning.

default.html - add
    <script src="/js/require.js" ></script>
before default.js

default.js - initialise requireJS and include whatever dependencies you want to use in your startup code.

app.addEventListener("activated", function (args) {
        if (args.detail.kind === activation.ActivationKind.launch) {
            if (args.detail.previousExecutionState !== activation.ApplicationExecutionState.terminated) {
                // TODO: This application has been newly launched. Initialize
                // your application here.
            } else if (args.detail.previousExecutionState !== activation.ApplicationExecutionState.closedByUser) {
                // TODO: This application has been launched after a requested shutdown. Initialize
                // your application here.
            } else {
                // TODO: This application has been reactivated from suspension.
                // Restore application state here.
            }
 
            if (app.sessionState.history) {
                nav.history = app.sessionState.history;
            }
 
            //RequireJS Initialisation here
            require.config({
                baseUrl: '/js'
            });
 
            //this creates a promise that calls complete() when the initAsync promises finish
            //all this code happens within the require context so we are guaranteed that the dependencies are resolved before the promise is resolved
            //the initAsync methods return promises that load data from internal json data files
            //persistence.loadState is synchronous code so doesn't return a promise
            var loadPromise = new WinJS.Promise(function (complete) {
                require([
                    'persistence',
                    'game/staticdata/itemClassStore',
                    'game/staticdata/itemStore',
                    'game/gamestate/world',
                    'game/gamestate/characters',
                    'game/classes/character'
                ], function (persistence, itemClassStore, itemStore, world, characters, character) {
 
                    itemClassStore.initAsync()
                        .then(function () { return itemStore.initAsync(); })
                        .then(
                            function () {
                                var data = persistence.loadState();
                                world.fromJson(data);
                                complete();
                            }
                        );
 
 
                });
            });
 
            //args.setPromise is a function on the activate event which allows you to wait for promises to complete before continuing
            args.setPromise(
                loadPromise
                    .then(function () { return WinJS.UI.processAll(); })
                    .then(
                        function () {
                            if (nav.location) {
                                nav.history.current.initialPlaceholder = true;
                                return nav.navigate(nav.location, nav.state);
                            } else {
                                return nav.navigate(Application.navigator.home);
                            }
                        }
                    )
            );
        }
    });

xPage.js - resolve your dependencies in the page ready function with a call to require(), and perform your page initialisation as per normal. 

///  
/// 
/// 
/// 
/// 
 
(function () {
    "use strict";
 
    WinJS.Namespace.define("CharacterSelect", {
        CharacterSelectViewModel: WinJS.Class.define(
            //the viewmodel constructor takes the injected requireJS dependency from the page ready code.
            function (world) {
                this._charactersModule = world.characters;
                this._itemsDataSource = new WinJS.Binding.List(this._charactersModule.characters);
                this._inventoryDataSource = new WinJS.Binding.List(null);
            },
            {
                _charactersModule: null,
                _itemsDataSource: null,
                _inventoryDataSource: null,
                _selectedCharacterId: 0,
                selectedCharacterId: {
                    get: function () {
                        return this._selectedCharacterId;
                    },
                    set: function (value) {
                        this._selectedCharacterId = value;
                        if (value == 0) {
                            this._inventoryDataSource = new WinJS.Binding.List(null);
 
                        } else {                            
                            this._inventoryDataSource = new WinJS.Binding.List(this._charactersModule.get(value).inventory.items);
                        }
                        this._getObservable().notify("selectedCharacterId", value);
                        this._getObservable().notify("inventoryDataSource", this._inventoryDataSource);
                    }
                },
 
                listDataSource: {
                    get: function () {
                        return this._itemsDataSource;
                    }
                },
                
                inventoryDataSource: {
                    get: function () {
                        return this._inventoryDataSource;
                    }
                },
 
                addNew: function () {
                    WinJS.Navigation.navigate("/pages/newCharacter/newCharacter.html");
                },
 
                deleteCharacter: function (that) {
                    that._charactersModule.deleteCharacter(this.selectedCharacterId);
                    that._itemsDataSource = new WinJS.Binding.List(this._charactersModule.characters);
 
                    that._getObservable().notify("listDataSource", that._itemsDataSource);
                }
 
            },
            {}),
 
    });
 
    WinJS.UI.Pages.define("/pages/characterSelect/characterSelect.html", {
 
        // This function is called whenever a user navigates to this page. It
        // populates the page elements with the app's data.
        ready: function (element, options) {
            require(
                [
                    'persistence',
                    "appbar",
                    "game/gamestate/world"
                ],
                function (persistence, appbar, world) {
                    //normal page initialisation code - all dependency modules are initialised at this point
                    //create view model (passing in resolved dependencies) and bind to relevant events
                    //(WinJS only has 1-way bindings and can't declaratively bind to button events that i can see)
                    var section = element.querySelector("section");
 
                    var viewModel = new CharacterSelect.CharacterSelectViewModel(world);
                    var observableviewModel = WinJS.Binding.as(viewModel);
                    WinJS.Binding.processAll(section, observableviewModel);
 
                    document.getElementById("cmdCreate").addEventListener("click", observableviewModel.addNew, false);
                    document.getElementById("cmdDeleteCharacter").addEventListener(
                        "click",
                        function(){
                            observableviewModel.deleteCharacter(observableviewModel);
                            persistence.saveState();
                        }
                        ,
                        false
                        );
 
                    document.getElementById("characters").winControl.onselectionchanged = function (ev) {
                        var selection = document.getElementById("characters").winControl.selection;
                        if (selection.getItems()._value.length > 0) {
                            viewModel.selectedCharacterId = selection.getItems()._value[0].data.id;
                            appbar.characterSelectSelected();
                        } else {
                            viewModel.selectedCharacterId = 0;
                            appbar.characterSelectDeSelected();
                        }
 
                    };
                    appbar.characterSelectInit();
 
 
                }
            );
 
 
        },
 
        unload: function () {
            // TODO: Respond to navigations away from this page.
        },
 
        updateLayout: function (element, viewState, lastViewState) {
            /// 
 
            // TODO: Respond to changes in viewState.
        }
    });
})();

Thursday, March 14, 2013

New job update

Well... This post is probably a few months late, as I have been in my new job for 9 months now.

I was getting frustrated in my old job due to a change in business direction away from my core career goals so when a .net community member advised me to hand in a resume I thought I didn't have much to lose.

So within a week of handing in a resume I was given an offer I could not refuse: a substantial pay rise, a company focusing on my strengths,  and working with a few people I had a healthy respect for.

9 months on and I am now at a new client with a substantial responsibility increase, learning a lot, and generally doing what I had expected would take me another few years to reasonably obtain at my old job if any opportunities were even to arrive.  The client even seems happy with my performance too, which is a bonus.

So far the risk of moving to a new job has paid off, and I hope to keep pushing my boundaries now that I have the opportunity to do so.

Thursday, June 21, 2012

Team Learning

I have been fortunate enough at work to have participated in a couple of trials for online learning/training proposals for our teams, and I thought I would post some opinions.

The two trials were Pluralsight, and Safari Books Online, both of which which focus on self-learning rather than formal training, which is my personal preference.  I would rather determine what I want to focus on than be led down a particular path.

However both of these options have provided a very different experience, and I am not sure which one I prefer.

Pluralsight
Pros:
  • Good quality videos
  • Good range of topics
  • Concise and straight forward guides
  • Offline support
  • Mobile device support

Cons:
  • Does not always go to the 'next level' of complexity
  • Videos arguably require more 'attention' than books.

Safari Books Online
Pros:
  • Massive Library
  • Ok mobile device support (no offline access)
Cons:
  • A lot of crap to wade through to find good resources
  • Books can generally be overly verbose / take more time to digest

Usage Patterns
As an avid reader the poor kindle support of the safari books online is a bit of a letdown, but it does work relatively on modern mobile browsers for reading on the go.  The lack of an offline mode is not too much of an issue as data usage is quite low, though it does mean you need a tethered connection if you are not reading on your phone. 
Pluralsight has very good mobile support (with most major phone platforms supported), as well as offline access for both web and mobile devices.  My key concern is that 'video' requires more investment in attention than reading, as you require both audio and video (arguably you can just listen, making it less of an investment than reading, but i feel you lose too much by doing this).  This makes it more difficult to, for example, sit in the living room while your wife is watching terrible TV.

Content
The sheer breadth and depth of knowledge available on safari books online is outstanding, IF you have the patience to find the right resource, and the patience to actually read a book on a particular topic.  Quite often technical books build on the knowledge of previous chapters to present more complex topics, which can make it difficult to try an pick up the complex topics without having read the rest of the book. 
Pluralsight is definitely a more accessible learning tool to come up to speed with new concepts and tools, but the depth of knowledge cannot compare with that of the safari books online.  Where it excells is providing a very concise information with clear samples that can easily be picked up without having to wade through verbose text or code to understand what is being portrayed.  One area that Pluralsight probably falls down is as a reference resource, as it can be difficult to use to look up a particular detail / syntax, while there are literally hundreds of books you could bookmark and index for this very purpose.
 
Conclusion

For a small dedicated team of expert developers, Safari Books Online is an excellent resource.
For a larger team with a wide range of expertise and commitment levels, Pluralsight is an excellent way to introduce new technologies without a heavy investment of time for the developers.

Personally I like having access to the entire Safari Books Online library, however the content available on Pluralsight is far more accessible and immediately useful than trying to find and wade through entire books on the same topics.

Tuesday, May 8, 2012

TFS - Creating a Branch from Local Source

Well that was frustrating.

The Problem:
Work has been performed on the source Trunk that now needs to be in a separate branch, and not checked into the Trunk.

The Solution:
Create a branch selecting 'current workspace' as the source version
*Ba Bow* - nope, this doesn't do what you think it does, your shiny new branch does not have your local workspace changes.

Create a branch and merge the changes across
*Ba Bow* merges only work on checked in code.

Create a branch, shelve your changes, unshelve to the branch
*ding ding* we are on a winner, only it is a PITA to get it working.

Firstly you need the TFS power tools - here: http://visualstudiogallery.msdn.microsoft.com/c255a1e4-04ba-4f68-8f4e-cd473d6b971f
Next you need to, for some reason, ensure you have no other pending changes in your workspace.  I have no idea why, but even if you have pending changes competely unrelated to the shelve files, you will get warnings and may get errors.  Strange but true.
Next you need to call the "tfpt unshelve" power tool command. But you need to call this from a folder in the workspace you are working with as there is no way to set the workspace/server in the tfpt command.

tfpt unshelve MyShelfsetName /migrate "/source:$/MyPath/My Path with spaces/MyBranch" "/target:$/MyPath/My Path with spaces/MyOtherBranch"
Finally, when you run the command you will need to merge all the shelved files into the new branch, one by one.  You can perform an auto-merge, but this will actually perform a merge on the destination/shelf, when in reality you very likely want to take the shelf version rather than do a merge, which requires you to go through file by file.

The link below is a good source of information on this process.
http://codereferences.blogspot.com.au/2012/02/migrating-shelveset-from-one-branch-to.html

Tuesday, May 1, 2012

Solving Race Conditions



Example

We have 'Tags' that can be applied to a 'Post'. When creating a post, we want to only create new tag entities when the tag does not already exists. A race condition can exist when two posts with the same new tag are created at the same time as the check if the tag exists can be false for both posts.

Solutions

1) Always create a new tag - we don't really care that we have duplicate tags if we always perform operations on the tagName, not the actual instance of a tag - this in ddd would be a 'Value Object'.

Side effects are; a potential drop in performance, and an increased database size

2) Use Database constraints to mark the field as unique, so the second post will attempt to create the tag and fail.

Side effects are; it is difficult to impose unique constraints in ORM, not all underlying providers (object store for example) will support this, and requires 'retry' logic.

3) Use a Transaction to check the existance of the entry before the insert.

Side effects are; a drop in performance from transactional locking and increased solution complexity.

4) Use a messaging service model for processing each post creation as a separate operation so the race conditions wont exist.

Side effects are; an increase in application complexity due to the asynchronous nature of the queue, and reduced performance due to the overhead (albeit with an increased scalability)

Conclusions

Option 1 in this scenario is entirely valid but this is not always the case, often guaranteed uniqueness is important, so using this as a general solution to race conditions is not appropriate.

Options 2 and 3 are very reiliant on the functionality of the underlying data provider.  In most LOB solutions is probably OK, but it is not as flexible and scalable as I would like.

Option 4 seems like a fairly drastic change in application design, but in reality this should not be as large an impact as you would think.