ORMBattle.NET

The goal of this test sequence is to measure peak performance of basic operations for different ORM tools. We've chosen several basic operations common to all object-relational mappers and measure their operation per second rate for each ORM. The following operations were selected:

Data modification:

• Creating new entity. One operation = creation of a single entity.
  
• Updating persistent entity - changing a single non-key field value. One operation = updating a single entity.
• Deleting entity. One operation = removing a single entity.

Data access:

• Reading (fetching) entity by its key (identifier). One operation = fetching a single entity.
• Materializing entities: enumerating entities from large query result. One operation = getting a single entity from query result.
• Query execution. One operation = executing a single LINQ query returning entity by its key. I.e. like fetching, but using LINQ.
• Compiled query execution. The same, but using compiled query execution API, if it exists. If it does not exist, or framework ensures any LINQ query is executed via internal compiled query API, the results of this test are the same as of query test.
  

All these operations are utilizing Simplest entity in order to measure overhead added by a particular ORM better:

public class Simplest
{
..public long Id { get; set; }
..public long Value { get; set; }
}

There is one test per each tested operation per each ORM. Each of such tests executes nearly the following sequence:

1. Open a session
2. Start time measurement
3. Open a transaction
4. Perform N tested operations, currently N = 50, 100, 500, 1000 (published in scorecard), 5000, 10000 and 30000
   
5. Commit transaction
6. Finish time measurement
7. Close session

For example, entity creation test for NHibernate is:

protected override void InsertTest(int count)
{
..using (var transaction = session.BeginTransaction()) {
....for (int i = 0; i < count; i++) {
......var s = new Simplest(i, i);
......session.Save(s);
....}
....transaction.Commit();
..}
}

So we measure the time an ORM spends on this procedure and calculate how many tested operations it can run per each second.

You can download the whole test suite source code or browse it online. See testing precautions section for other important details related to performance testing.