Explain what is Cache Stampede

Technology CommunityCategory: Software ArchitectureExplain what is Cache Stampede
VietMX Staff asked 3 months ago

cache stampede (or cache miss storm) is a type of cascading failure that can occur when massively parallel computing systems with caching mechanisms come under very high load. This behaviour is sometimes also called dog-piling.

Under very heavy load, when the cached version of the page (or resource) expires, there may be sufficient concurrency in the server farm that multiple threads of execution will all attempt to render the content (or get a resource) of that page simultaneously.

To give a concrete example, assume the page in consideration takes 3 seconds to render and we have a traffic of 10 requests per second. Then, when the cached page expires, we have 30 processes simultaneously recomputing the rendering of the page and updating the cache with the rendered page.

Consider:

 function fetch(key, ttl) {
   value ← cache_read(key)
   if (!value) {
     value ← recompute_value()
     cache_write(key, value, ttl)
   }
   return value
 }

If the function recompute_value() takes a long time and the key is accessed frequently, many processes will simultaneously call recompute\_value() upon expiration of the cache value.

In typical web applications, the function recompute_value() may query a database, access other services, or perform some complicated operation (which is why this particular computation is being cached in the first place). When the request rate is high, the database (or any other shared resource) will suffer from an overload of requests/queries, which may in turn cause a system collapse.