I have a project where I need to process every night large amounts of data. Basically I have to fetch a lot of objects and related collections from the DB using NHibernate, operate on them, and then do some updates. This process usually takes about 2-3 hours.

Today I discovered a way to brutally increase the application speed. I noticed that the application started fast, but would gradually become slower. I didn't know why, so I tried to free already used objects and splitting the process into several separate instances. This helped only a little. I'm doing something like this (we use a facade on top of NHibernate, so: different types, but the same concepts/names):

C#:
  1. public static void Operate()
  2. {
  3.     using( IPersistanceSession
  4.         session = GetNHibernateSession(MasterSession) ) {
  5.        
  6.         IList list = GetALotOfObjects();
  7.         foreach( object obj in list ) {
  8.             Operate(session, obj);
  9.         }
  10.        
  11.     }
  12. }

This code is very slow. Some time ago, I had to go into NHibernate source code to understand how I could use sessions with ASP.NET, and I noticed that NHibernate uses a lot of internal objects. Well, on this case, I am fetching a lot of objects that have no relation with each other, so, I guessed I could clear the session from time to time... And adding a clear, boosted the speed considerably:

C#:
  1. public static void Operate()
  2. {
  3.     using( IPersistanceSession
  4.         session = GetNHibernateSession(MasterSession) ) {
  5.        
  6.         session.Clear();
  7.        
  8.         IList list = GetALotOfObjects();
  9.         foreach( object obj in list ) {
  10.             session.Clear();
  11.             Operate(session, obj);
  12.         }
  13.        
  14.     }
  15. }

I really can't explain this as my knowledge of NHibernate internals is minimal, but believe me when I say that Clear made a difference... a 2/3h to 10 minutos difference!

Related Posts

One Response to “Batch Processing with NHibernate”

  1. Mário Lopes Says:

    I wonder whether something is seriously broken within NHibernate internals.