While this may sound like an obvious metric to check, I’m often seeing that developers don’t verify the cache hit rate on their code under realistic conditions. The end result is a server which performs worse than if it had no cache at all.
We all know the benefits of keeping a local cache.. relatively cheap to keep and it saves having to make more expensive calls to obtain the data from wherever it ultimately resides. Just don’t forget that keeping that cache, while cheap, takes non-zero CPU and memory resources. The code must get more benefit from it than the cost of maintaining the cache, otherwise it is a net loss.
I was recently reviewing a RESTful service which kept a cache of everything it processed. The origin retrieval was relatively expensive so this seemed like a good idea. However, given the size of the objects being processed vs. the amount of RAM allocated to this service in production, my first question was what’s the cache hit rate?
Developers didn’t know, but felt that as long as it saves any back-end hit it must help, right?
A good rule of thumb is that anything that isn’t being measured is probably misbehaving… and this turned out to be no exception.
Testing under some load (I like faban for this) showed the server was allocating and quickly garbage collecting tens of thousands of buffer objects per second for the cache. Hit rate you ask? Zero!
Merely commenting out the cache gave a quick 10% boost in overall throughput.
So that’s my performance tip of the day.. be aware of you cache hit rates!
Are these the same devs who caution against premature optimization?
I was once asked to add caching for HTTP digest authentication responses. I might have gotten it done too, if I hadn’t remembered what a nonce was.
While there is truth to the Knuth quote, it has certainly been abused to justify sloppy code and myopic designs far more often that it has been used in its proper context!