ORMBattle.NET

It finally happened! Almost 9 months has passed since last scorecard update, and now the baby is born :) you can see the newest results.

Most important changes:

• Test suite was migrated to .NET 4.0.
• All tested ORM tools are updated to their most recent versions. New versions are mentioned on their description pages.
• There is a new player, Devart LinqConnect ORM, showing perfect results right on the first round.
• LightSpeed ORM is removed from this test suite by decision of Mindscape.
• Finally, all the tests were performed on 64-bit Windows 7.

Brief overview of new results:

• Most of players show nearly perfect LINQ test results, so I suspect they're quite comparable from this point now. All the typical cases are fully covered. I hope ORMBattle LINQ test played a noticeable role in this process.
• There are no changes in LINQ test pass results for Subsonic (that's predictable - the project frozen for at least one year) and NHibernate. The case with NHibernate is a bit more complex: currently we test its latest stable version (v2.1.2 GA), although NHibernate 3.0.0 alpha1 is already available. But we were unable to update our tests for it (see the commit comment here). Thus any help in resolving this issue is appreciated.
• BLToolkit is still definitive leader in performance tests; all the other commercial players are quite comparable there now. As I hope, that's partly because of this test. There are no changes in NHibernate and Subsonic results here.
• Lots of people were interested in EF 4.0 results: EF team has improved LINQ support (they got additional 5% here), but its performance results are noticeable worse - from 10 to 20% in most of the cases, and nearly twice - on materialization test. So if earlier it was one of leaders on this test, now it's closer to losers. If you know any possible reasons of this, please describe it in comments to this post.
• New LINQ to SQL shows much worse performance on CRUD operations. Please let know us, if you know the reason \ possible fix - the difference looks quite suspicious; probably, there is some mistake in test (this could appear e.g. because of change in default behavior).
• Our own ORM, DataObjects.Net, also shows some decrease in materialization performance. The only reason I can imagine here is lack of almost any profiling of this part during last year. So hopefully, we'll improve our own results soon.

That's all for now. Since there are few suspicious results and upcoming upgrade to NHibernate 3.0.0 alpha1, the next update is near.

Enjoy the new scorecard.

Kind regards,
Alex Yakunin

In March I started to write a series of posts called "Can ORM make your application faster?". 6 posts are already there, although the whole sequence isn't finished yet (planning to do this in two weeks). The topic is perfectly fitting to this website, so I sharethe link here.

I'd appreciate if you could give me more links to similar articles (~ optimization techniques used by ORM tools). Please leave them in comments - I'll share the most interesting ones (or all) in this blog.

We decided to do this before making further changes. So to checkout the code now, you should run the following command:

hg clone https://ormbattle.googlecode.com/hg/ OrmBattle

Alternatively, you can use TortoiseHg GUI.

Hi everyone!I

I didn't post here for pretty long time - we worked really hard on update of our own product. Now this part is finished, and we're going to spend some time on improving ORMeter (yes, we renamed it - you might notice it even by new header).

What's planned:

• Add collection performance tests (CRUD & simple bulk load tests)
• Add few more LINQ tests - see http://groups.google.ru/group/ormbattle-development/t/6dbeb2497f3544a9
• Implement HTML output mode for TestRunner. Currently it prints out text tables, and actually I hate converting them to Excel, and then - to nicely looking HTML with pictures. But if it would export HTML, relatively simple jQuery script + Google Chart API could make this manual conversion process unnecessary.
• Update the scorecard there using the most recent versions of all products.
• Likely, we'll do some minor changes related to design and scorecard layout.

I hope we'll be able to finish at least testing-related part of this plan in March 2010.

As usual, any opinions and ideas are welcome.