Smooth sailing with Mongoid

We’re working on a project that uses MongoDB as a legacy database. Thanks to Mongoid, the paradigm shift from standard SQL to NoSQL when interacting with the database from Rails is quite affordable, once you get used to some things like having your fields defined in your models (I like this, though!), or special/different syntax for your queries. Smooth sailing, you could say.

Anyhow, the project database has grown a lot in the last few months, until the point we started noticing longer than usual response times, and even the database going away.

Enter the indexes

We analysed the way we queried our models in the database and recognized the critical fields in our model, those that were queried most of the time, like the ‘status’ field on a model that had a states machine, the date fields that marked another model’s period of validity, or flags we used on other models to discriminate if an instance should be part of a collection or another.

There are many options for indexes in Mongoid, but we kept things simple and wanted to try an easy approach, so kept to just using background and unique, on a couple of indexes defined on some of our models. Anyway, we indexed all our belongs_to relationships, to stay safe. Even more, we updated Mongoid to the latest version on the 3.1 branch, to make sure we could get all the optimizations done in the gem.

The results

After deploying these small changes, we got the database to behave correctly and stopped having the problems explained above. But in any case, we are aware now that we must take extra care when writing queries, as their complexity increases along with size of the database.

We’ve since then identified and amended several queries that where slower than they should, as well as using newly introduced (back then) methods as pluck, which is way faster than map when collecting object ids for example.

But more importantly, we’ve started applying different approaches when building new queries, so we hit the database as less as possible. It takes a bit more of times in some cases, but just with keeping an eye open and giving it some thought, you can save a lot of time, easing the database usage and keeping the application usable.

Picture Index Cards’ ‘DNA’ by Aya Otake from Flickr. Used without changes under license