Redis Optimization

If you're using Redis as a cache backend or to store user sessions in Magento take a closer look on what's happening on your servers. On one of our servers I discovered that the Redis log file was constantly growing and showing tons of error messages like this one:

[1185] 05 Feb 01:17:36 * 1 changes in 900 seconds. Saving...  
[1185] 05 Feb 01:17:36 # Can't save in background: fork: Cannot allocate memory

In addition to that New Relic showed that a big portion of the rendering time was used for "Memcache" (although we're not using Memcache at all on this server).

So, this is what actually was happening: By default Redis is configured to persist its data to the hard drive. But this was broken, what not only resulted in Redis flooding the log file, but also in slowing down Redis remarkably.

After some research I found out, that the operating system's overcommit settings were causing this problem.

These settings control how Linux handles virtual memory limits. Find more about this works in detail hereand here.

After changing this setting from "0" to "1" Redis started persisting the data immediately and the overall performance increased dramatically.

# echo 1 > /proc/sys/vm/overcommit_memory

(Instructions for Redhat)

But persisting the data also needs some resources which might not be needed in case you're using Redis as a cache backend (losing cache data is acceptable). By turning of data persistence the CPU didn't show any peaks anymore:

################################ SNAPSHOTTING #################################  
#  
# Save the DB on disk:  
#  
# save <seconds> <changes>  
#  
# Will save the DB if both the given number of seconds and the given  
# number of write operations against the DB occurred.  
#  
# In the example below the behaviour will be to save:  
# after 900 sec (15 min) if at least 1 key changed  
# after 300 sec (5 min) if at least 10 keys changed  
# after 60 sec if at least 10000 keys changed  
#  
# Note: you can disable saving at all commenting all the "save" lines.  
# save 900 1  
# save 300 10  
# save 60 10000

Due to Redis' "mostly single threaded design" it is recommended to run multiple Redis instances (even if they are on the same server) for storing the sessions and for storing cache content. And if you're using Magento Enterprise Edition's full page cache you could even add another Redis instance for this one instead of running multiple databases within the same Redis instance.

Plus this allows you to have different configurations: You wouldn't want to lose any sessions, so configure that Redis instance to persist data. On the cache backends you shouldn't care too much about losing cache data, so save some resources by not persisting these Redis instances.