Tuna Toksoz : nhibernate

My Thoughts on ORM comparisons

Tuna Toksoz — Fri, 21 Aug 2009 16:27:00 GMT

UPDATE 1: I made clear that NH actually performs better in those posts, when a simple optimization such as Enabled Batching.
UPDATE 2: Please check Ayende's latest post for how those benchmarks can be made fair.

I am a bit late to the party but there we go.

Last several days, there have been a hot debate on the performance and capability of NH vs some other commercial/noncommercial ORMs. This issue has been discussed both in NHibernate Developer List and in Ayende’s blog. I have recently noticed another benchmark between NH and EF, which I hope to address, too.

Fallacies of benchmark

What I have seen in those benchmarks is the wrong use of ORMs. This statement has two aspects: Wrong use of ORM and wrong use of NH itself.

In ORM, you don’t use it to do bulk operations. And 0.001 second difference will not differ anything in your application. If it does, you don’t also use RDBMS, you use something else.

There is no point on making a loop of thousands of iterations which saves an entity. Davy has well pointed out that this has a lot of implications in NHibernate, unlike some other ORMs. First of all, NHibernate first-level cache will be in action and it will fill the cache with each item saved. Secondly, as NH makes use of POCO it has no mechanism of things like INotifyPropertyChanged, it has to compare the old to the new value. As you may guess, the more entity you have, the longer it will take. No exception. If you want to leave first-level cache out of the game, you use StatelessSession feature of NHibernate, no exception. Actually when you check Fabio Maulo’s post you’ll see it can even become faster without Stateless Session.

Another point I have noticed in both benchmarks is that they are/were not fair. If I compare it to EF, lets say, ctx.SaveChanges() will batch the chages and send the query to the database in one go. NH has this feature, as Ayende already pointed out, since 2006. However, NH by default doesn’t make use of it, and this is not a bug, it is a feature. You have to use setting to enable it.

Gergely Orosz has made his own benchmark and didn't use batching support and NH fell well behind of EF. I contacted him and he was kind enough to discuss the issues. I showed him how to enable batching and why not to flush everytime, as it wouldn't be fair. Now NH performs much better and he said he will update his post.

Benchmark the community, too

NHibernate has a big community, and great minds as committers. It really does matter. Let me give you some statistics about NHibernate development and its community

Here is the commit statistics.

It is one of the most active projects in .net environment. The image above doesn’t include the contrib projects. A great kiss goes to Fabio Maulo for his massive efforts.

Let’s look at discussion group, shall we?

Do you know any other group with that activity? I don’t know.

In addition, NHibernate gives you more than _ANY_ ORM can give. It gives you a lot of extensibility points which can be used in a lot of scenario. Validator, Search, Spatial, Shards to name a few.

In conclusion

NHibernate is the obvious winner.

NHibernate Linq Released

Tuna Toksoz — Sun, 26 Jul 2009 20:27:00 GMT

Just as a reminder, even though you might have seen it before on twitter or nhforge or Ayende’s blog

As of today, we released the first version of NHibernate.Linq. It is currently based on Criteria API, and can handle many situations that it can handle.

NHibernate.Linq was the first project that I contribute as a committer (it was in Rhino Tools), it is a strange feeling to see its release.

I would like to thank Oren Eini(Ayende), Chad Lee and Will Shaver for making this project come true.

You can read the official announcement here on NHForge

Querying on Child Count With NHibernate

Tuna Toksoz — Thu, 09 Jul 2009 23:23:00 GMT

This is a recent question raised in NHibernate Users Group. The user wanted to realize the following query with Criteria api.

var result = db.Person.Where(x => x.Pets.Count > 0 && x.Alive).OrderBy(x => x.Name);

This is not a simple query, but it has a solution

  
DetachedCriteria crit = DetachedCriteria.For(typeof (Person), "p2")
    .CreateCriteria("p2.Pets","Pets")
    .Add(Restrictions.EqProperty("p.Id", "p2.Id"))
    .SetProjection(Projections.Count("Pets.Id"));

ICriteria c = s.CreateCriteria(typeof (Person), "p")
    .Add(Restrictions.Gt(Projections.SubQuery(crit), 0))
    .Add(Restrictions.Eq("p.Alive",true))
    .AddOrder(Order.Asc("p.Name"));

What we had to do is to create a DetachedCriteria and on that execute CreateCriteria so that we can do querying on our collection.

The other way is simpler, but requires you to use HQL (below query is provided by Fabio Maulo)

session.CreateQuery("from Person p where  size(p.Pets) > 0 and p.Visible = true order by p.Name")

session.CreateQuery("from Person p where  p.Pets.size > 0 and p.Visible = true order by p.Name")

I hope this helps.

Fluent Interface for NH Facility – Take 1

Tuna Toksoz — Sat, 27 Jun 2009 11:39:00 GMT

We’ve been getting many requests on having fluent configuration for NHibernate Integration Facility, and as I like programmatic configuration more than XML configuration (did I mention that I hate XML?), I decided to work on it. After 2-3 hours, I got the below more or less working

container.Register(
Fluently.ConfigureFacility()
    .Id("nhibernateFacility")
    .DefaultConfigurationBuilder<DefaultConfigurationBuilder>()
    .DefaultConfigurationPersister<DefaultConfigurationPersister>()
    .AddFactory(
                    Fluently.ConfigureFactory()
                        .Alias("myAlias")
                        .Id("myId")
                        .UsingConfiguration(
                        FactoryConfigurator.DefaultBuilder()
                            .ConnectionProvider("…………………………")
                            .ConnectionDriver("…………………………")
                            .ConnectionString("…………………………")
                            .Dialect("…………………………")
                            .ProxyFactory("…………………………")
                            .Assemblies("…………………………")))
    .AddFactory(
                    Fluently.ConfigureFactory()
                        .Alias("myAlias")
                        .Id("myId")
                        .UsingConfiguration(
                        FactoryConfigurator.XmlBuilder().File("myFile.xml"))));

The pieces in Italic are not necessary to write as they have their defaults, also we have some generic overloads for things like Dialect, ProxyFactory and ConnectionProvider.

What the above interface does is that it actually converts all the above configuration to IConfiguration and add them to the container as Facility Configuration, this is actually what is done behind the scenes when you use XML configuration.

I asked for a review over the syntax from several tweeps (@kkozmic, @dagda1, @mikehadlow, @chriscanal) and from one other NH Facility user, German Schuager, and got great feedback.

One of the issues that they pointed out with the above interface is that it is less discoverable. You have to find Fluently class, and then FactoryConfigurator class. Another issue is that it feels less natural to configure the facility like this. Instead, they prefer the configuration take place right on the facility.

This one is what German Schuager has suggested

container.AddFacility<NHibernateFacility>("nhibernateFacility", cfg => cfg
        .DefaultConfigurationBuilder<DefaultConfigurationBuilder>()
        .DefaultConfigurationPersister<DefaultConfigurationPersister>()
        .AddFactory("id1", f => f
                .Alias("myAlias")
                .UsingConfiguration<DefaultBuilder>(c => c
                        .ConnectionProvider("………")
                        .ConnectionDriver("………")
                        .ConnectionString("………")
                        .Dialect("………")
                        .ProxyFactory("………")
                        .Assemblies("………")
                ))
        .AddFactory("id2", f => f
                .Alias("alias")
                .UsingConfiguration<XmlBuilder>(c => c
                        .ReadFrom("myfile.xml")
                ))
 });

and similar one from Krzysztof Kozmic.

Looks like i was the only one that is fan of Castle Microkernel style Fluent Interface.

Let’s see what I’ll come up for the second take, If I ever do it.

Id Generation for db4o

Tuna Toksoz — Mon, 18 May 2009 16:55:00 GMT

Disclaimer: This will be another post on events and extensibility points of a framework, but I can’t resist!

Recently, on one of my pet projects, I wanted to use db4o, and the problem was I should have explicit POIDs on my objects. Db4o has two mechanisms for Id stuff, one is GetId function, which is the physical address of the object on the disk and the second was the UUID that is used for replication purposes. The first one is fast but you can’t rely on it because the physical address may change after defragmentation, and the other is not very readable. This is the time I started to think of Id Generators for db4o. My friend Dario Quintana from NHibernate definitely had same needs(who doesn’t?) and as one having the right spirit, he implemented similar feature for db4o but this time with touching the codebase. My feature will use db4o Extensibility Point and will allow you to have several different Id Generators. I will use ideas from NHibernate POID generators specifically HiLo algorithm.

What is HiLo algorithm?

It is a nice algorithm that allows you to have ids without putting locks, thus slowing down the things. The algorithm works as follows.

Each Generator requests a Hi value from database and with every request, this value is incremented. The generator then uses a magic formula (Hi-1)*Capacity+(++Lo) value. Lo values are generated by the generator and incremented with every new object saved into database. Hi value is stored in database while Lo value is stored in a variable in the client. When Lo becomes larger than Capacity (that is Lo values are exhausted), the generator requests a new Hi value from the database. As you see, we don’t issue many requests for Hi values thus there is practically no overhead on the database side. The larger the capacity, the less there is need to request for new id thus the less overhead. However, Hi Capacity means there will be ranges that will never ever be used, and people dislike it (why does it bother them is a question for me, too). Practically Int32 can handle values up to 2,147,483,647 while Int64 can handle up to 9,223,372,036,854,775,807 (no I could not say this number in spoken English but learnt that is is nine quantillion). Having Capacity=32767 will allow you to have 281483566907400 Hi values and assuming you have 100 servers… wait it is obvious that you cannot consume all those in several million years?

How can we integrate it into db4o?

Well, this is easy. Db4o provides us an event mechanism that can be used for such stuff. It has several events namely Creating, Created, Updating, Updated etc. All we have to do is to catch this event on the server side, and increment Hi values there. There is one thing to be careful about: Db4o events may fool you. In case of Client/Server mode you should call the below code on ServerContainer, otherwise you may have to use another ObjectClient as each object container runs in its own transaction. If you are using it as embedded database, then you won’t have any problem with it.

Now, I define a contract called IIdGenerator

public interface IIdGenerator
{
    object Generate();
}
public interface IIdGenerator<T>:IIdGenerator
{
    new T Generate();
}

Pretty self explanatory, isnt’ it? Now it comes to persistent id generator, which is for now the increment generator

public class IncrementGenerator:BaseIdGenerator<long>
{
    private readonly Type type;
    private readonly IObjectContainer container;
    private readonly string semaphoreName;
    public IncrementGenerator(Type type,IObjectContainer container)
    {
        this.type = type;
        this.container = container;
        this.semaphoreName = string.Format("id_gen_{0}", this.type.Name);
    }

    public override long Generate()
    {
        long valueToBeReturned;
        while (!container.Ext().SetSemaphore(semaphoreName, 1000)) ;//Do some busy wait
        IObjectSet set=this.container.QueryByExample(new IncrementTypeValuePair{Type=type});
        IncrementTypeValuePair pair;

        if (set.Count == 0)
            pair = new IncrementTypeValuePair {Type = type};
        else
            pair = set[0] as IncrementTypeValuePair;
        valueToBeReturned = ++pair.Value;
        container.Store(pair);
        container.Ext().ReleaseSemaphore(semaphoreName);
        return valueToBeReturned;
    }
}

There we used Semaphore in order not to have concurrency issue, otherwise we could give very same id to different objects. This id generator will go to db everytime an id is requested, and it will create some bottleneck in case several hundreds of entities are inserted in a second. As an enhancement to this generator, we’ll inherit from this generator and create HiLoGenerator.

[MethodImpl(MethodImplOptions.Synchronized)]
public override long Generate()
{
    if(currentLo>=capacity)
    {
        currentHi=base.Generate();
        currentLo = 0;
    }

    return (currentHi - 1)*capacity + (++currentLo);
}

Exactly what I told in oral.

As the last step, I designed an ugly fluent interface for this, which looks like the following

var serverContainer = this.server.Ext().ObjectContainer();
serverContainer.IdMap(Map<Person>.On(x => x.Id)
         .SetGenerator(new HiLoGenerator(3, typeof(Person),serverContainer)));
var person = new Person();
container.Store(person);

The event wiring stuff is done a bit ugly too.

void registry_Creating(object sender, CancellableObjectEventArgs args)
{
    object item = args.Object;
    foreach (var map in idMaps)
    {
        if(map.EntityType.IsAssignableFrom(item.GetType()))
        {
            MemberExpression member = ((LambdaExpression)map.Expression).Body as MemberExpression;
            ((PropertyInfo) member.Member).SetValue(item, map.Generator.Generate(),null);
            break;
        }
    }
}

Whole code can be found if you follow this link.

If you liked this post, please kick it.

Castle NHibernate Facility – Configuration

Tuna Toksoz — Mon, 06 Apr 2009 12:56:47 GMT

Castle’s extensibility points provide you many ways to make your life easier. One and most widely used extensibility point is the facilities. By using facilities, you can integrate various other frameworks and technologies easily.

I am going to talk about NHibernate Integration Facility, which is currently lead by me. It was originally written by Hamilton Verissimo, and its current shape is more or less the same as what he wrote.

The purpose of this post is to get feedback and shape the documentation accordingly.

Now, after an historical introduction, lets talk about the facility’s purpose, and in as the first part of the documentation, the configuration.

The purpose
The purpose of this facility to provide an easy way to integrate NHibernate into MicroKernel backed applications. The facility provides a nice way to manage multiple session factories, a good way to manage sessions and it also provides some other structures that can be used to further integrate other frameworks such as Fluent NHibernate. It also plays nicely with other Castle services such as Transaction Management.

The Configuration
Currently, the facility can only be configured via XML, but it provides some extension points that can be used to configure it programmatically. The traditional way of configuring it is as given below.

<configuration>
    <facilities>
        <facility
          id="nhibernatefacility"
          type="Castle.Facilities.NHibernateIntegration.NHibernateFacility, Castle.Facilities.NHibernateIntegration"
          [optional: configurationBuilder="Your custom configuration builder"] [optional: isWeb="Your custom configuration builder"]>
            <factory id="sessionFactory1">
                <settings>
                    <item key="connection.provider">NHibernate.Connection.DriverConnectionProvider</item>
                    <item key="connection.driver_class">NHibernate.Driver.SqlClientDriver</item>
                    <item key="connection.connection_string">Data Source=.;Initial Catalog=test;Integrated Security=SSPI</item>
                    <item key="dialect">NHibernate.Dialect.MsSql2000Dialect</item>
                    <item key="proxyfactory.factory_class">NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle</item>
                </settings>
                <assemblies>
                    <assembly>YourAssembly.Name.Here</assembly>
                </assemblies>
            </factory>
        </facility>
    </facilities>
</configuration>

isWeb: In case of web application, you should set this to true, implement IContainerAccessor in your HttpApplication (the global.asax) also add following lines to your <httpModules> section.

<httpModules>
    <add name="NHibernateSessionWebModule"
             type="Castle.Facilities.NHibernateIntegration.Components.SessionWebModule, Castle.Facilities.NHibernateIntegration"/>
</httpModules>

As it can bee seen here, it is not very much different from the way we configure NHibernate. What happens behind the scenes are that: Facility creates an NHibernate Configuration, registers the SessionManager, and make those ready to be resolved.

The facility provides you some way to modify the Configuration that is created behind the scenes. The point is called IConfigurationContributor. By implementing classes that derive from that interface and registering them to container, you’ll be able to modify the NHibernate Configuration, just before the SessionFactory is resolved. The interface is simple

public interface IConfigurationContributor
{
    void Process(string name,Configuration config);
}

The name corresponds to the id in the configuration.

You can also Configure NHibernate Configuration via the IConfigurationBuilder interface. Currently, there are 2 built in ConfigurationBuilders, namely DefaultConfigurationBuilder and XmlConfigurationBuilder.
The default one is the one that parses the thing you see above while XmlConfigurationBuilder uses NH’s native xmls.

In the next post, I will hopefully show to integrate FluentNHibernate using custom ConfigurationBuilder

An improvement on SessionFactory Initialization

Tuna Toksoz — Fri, 13 Mar 2009 22:40:00 GMT

UPDATE: I have just committed the PersistentConfigurationBuilder for Castle NHibernate Facility. Thank you Jonathon Rossi for informing me!

We have received several complaints about slowness of SessionFactory initialization when there’s hundreds of entities, and Ayende has replied one of them here. It even gets worse if you’re using it in a web environment. You may think that it is not a problem since SessionFactory is initialized once in a web environment, but the major impact is not on production but development. Think how many times you start your application a day.

The problem is not really with NHibernate but with xml validation against the schema. Here are some profiler results for SessionFactory initialization with one thousand entities:

As you see, the adding XML resources takes the most time and the reason behind this is the schema validation. There is also an I/O cost involved (1040 resources should be read by NHibernate). There are several ways to get rid of it, one being the serialization of configuration. I spend 3 days (statics prevented me from spotting some bugs in the code) on this and I believe it pretty much works for every configuration. Another way of doing this is the merging of HBM files, which I believe faster than Serialization as Deserialization also takes some amount.

Now the results for the one using the Deserialized Configuration.

A nice feature of dotTrace allows us to compare the performance improvements over the old way.

We got 10 seconds rescued! Yay!

Now I am going to show how I used this feature in Castle NHibernate Facility. We have IConfigurationBuilder that is used to integrate various Configuration sources (such as FluentNHibernate).

First of all I must ensure that if any of the files that are used to create the Configuration change, we shouldn’t use the serialized configuration, instead the Configuration should be re-created.

public override Configuration GetConfiguration(IConfiguration config)
{
    log.Debug("Building the Configuration");

    string fileName = config.Attributes["fileName"];

    IConfiguration dependsOn = config.Children["dependsOn"];
    IList<string> list = new List<string>();

    foreach (var on in dependsOn.Children)
        list.Add(on.Value);

    Configuration cfg;
    if (IsNewConfigurationRequired(fileName, list))
    {
        log.Debug("Configuration is either old or some of the dependencies have changed");
        using(var fileStream = new FileStream(fileName, FileMode.OpenOrCreate))
        {
            cfg = base.GetConfiguration(config);
            this.WriteConfigurationToStream(fileStream, cfg);
        }
    }
    else
    {
        using (var fileStream = new FileStream(fileName, FileMode.OpenOrCreate))
        {
            cfg = this.GetConfigurationFromStream(fileStream);
        }
    }
    return cfg;
}



protected virtual bool IsNewConfigurationRequired(string fileName,IList<string> dependencies)
{
    if (!File.Exists(fileName))
        return true;
    FileInfo fi = new FileInfo(fileName);
    DateTime lastModified = fi.LastWriteTime;
    bool requiresNew=false;
    for (int i = 0; i < dependencies.Count && !requiresNew; i++)
    {
        FileInfo dependency = new FileInfo(dependencies);
        DateTime dependencyLastModified = dependency.LastWriteTime;
        requiresNew |= dependencyLastModified > lastModified;
    }
    return requiresNew;
}

Code doesn’t look really good, I guess, so I am open to any suggestions on improvement. The code is not yet in Castle Codebase, as our NH dependency on trunk is not the latest (and i am too lazy to update it). When I find time, I may update the dependency if others agree.

There is one thing that you have to be careful about. You must be aware that if you’re using IUserType, IInterceptor, ISqlFunction etc, all of those should be Serializable too!