Custom response caching in ASP.NET Core (with cache invalidation)

This post looks at custom output caching and in particular, allowing cache invalidation (cache busting) at the server level in order to allow the use of response caching for more dynamic pages.

In the last post we looked at the various options available for caching in ASP.NET Core. The best performing solution we discussed was response caching at both the client and server. While response caching is a powerful technique, unless you have the ability to remove cached items ad hoc then you are rather limited in how long you can cache for.

Background

Once a page is cached in the browser (or a proxy), the server cannot force a new version of the page to be used instead of the cached copy. Therefore, pages likely to change cannot be cached for very long at the client. This is not something we can change.

At the server, we should have more control but unfortunately the built-in response caching does not permit cache invalidation. It also uses the same cache duration for both server side caching and for instructing the browser to cache the page. These restrictions make the built-in response caching unusable in some applications.

We are going to look into creating a custom response caching solution. We will assume the following requirements:

• Allow server and client cache durations to be different
• Allow an arbitrary page to be removed from the cache at will
• Allow multiple cached pages to be removed based on a common criterion (tags)

The underlying requirement is the ability to cache at the server indefinitely and to allow application code to invalidate existing cache entries when content changes.

We will also strive for simplicity and speed over flexibility. We will implement the cache for a single web server using IMemoryCache. For web farm scenarios, it is trivial to replace IMemoryCache usage with IDistributedCache but this less feature-rich interface means that the third requirement (tagging) will not be possible without a redesign.

Implementation

To allow our caching to be as fast as possible, we will implement it as middleware which runs before MVC and can short-circuit the full request pipeline if a cached page is found.

For a page not in the cache, we have the following flow:

1. Fail to retrieve page from the cache
2. Execute inner middleware (the MVC pipeline) redirecting the response to a buffer
3. Cache the page (if conditions are met)
4. Render page

For an existing cached page, the sequence is much shorter:

1. Retrieve page from cache
2. Render page

Using middleware is the fastest option for returning cached content but it does make it more difficult to allow cache configuration for individual pages. If all pages are cached with the same criteria then the middleware could be used in isolation but in most real-world sites, we need the ability to configure which pages are cached and for how long. While this could be done as middleware configuration, it is much easier to use attributes on controller actions.

Controlling server caching

To allow us to control how individual pages are cached, we will use a very simple action filter attribute:

public class CacheAttribute : ActionFilterAttribute
{
    public int? ClientDuration { get; set; }
    public int? ServerDuration { get; set; }
    public string Tags { get; set; }

    public override void OnActionExecuting(ActionExecutingContext context)
    {
        // validation omitted

        if (ClientDuration.HasValue)
        {
            context.HttpContext.Items[Constants.ClientDuration] = ClientDuration.Value;
        }

        if (ServerDuration.HasValue)
        {
            context.HttpContext.Items[Constants.ServerDuration] = ServerDuration.Value;
        }

        if (!string.IsNullOrWhiteSpace(Tags))
        {
            context.HttpContext.Items[Constants.Tags] = Tags;
        }

        base.OnActionExecuting(context);
    }
}

The filter code is trivial and does nothing more than add the attribute values to the HttpContext Items collection. We are using the collection to communicate between the action and the caching middleware.

The middleware

The middleware's main Invoke method is fairly readable:

public async Task Invoke(HttpContext context)
{
    var key = BuildCacheKey(context);

    if (_cache.TryGet(key, out CachedPage page))
    {
        await WriteResponse(context, page);

        return;
    }

    ApplyClientHeaders(context);

    if (IsNotServerCachable(context))
    {
        await _next.Invoke(context);

        return;
    }            

    page = await CaptureResponse(context);

    if (page != null)
    {
        var serverCacheDuration = GetCacheDuration(context, Constants.ServerDuration);

        if (serverCacheDuration.HasValue)
        {
            var tags = GetCacheTags(context, Constants.Tags);

            _cache.Set(key, page, serverCacheDuration.Value, tags);
        }
    }            
}

How you choose to build the cache key will depend on your requirements but it can be as simple as using context.Request.Path.

If the page can be retrieved from the cache then it is written to the response and that is the end of the request.

private async Task WriteResponse(HttpContext context, CachedPage page)
{
    foreach (var header in page.Headers)
    {
        context.Response.Headers.Add(header);
    }

    await context.Response.Body.WriteAsync(page.Content, 0, page.Content.Length);
}

If a cached page is not found then there is a bit more work to do. First we set client caching headers if applicable and then perform a basic check to see if we can cache the request. We only want to cache GET methods so if another method is used then we invoke the next middleware component and take no further action in the request.

The next stage is to capture the request output from inner middleware components. We discuss how this is achieved in the next section. The final action is to save the page to the cache if we have configured server side caching (via the action filter discussed above).

Capturing the response

Capturing the page response requires you to swap out the default response body stream with a MemoryStream:

private async Task<CachedPage> CaptureResponse(HttpContext context)
{
    var responseStream = context.Response.Body;

    using (var buffer = new MemoryStream())
    {
        try
        {
            context.Response.Body = buffer;

            await _next.Invoke(context);
        }
        finally
        {
            context.Response.Body = responseStream;
        }

        if (buffer.Length == 0) return null;

        var bytes = buffer.ToArray(); // you could gzip here

        responseStream.Write(bytes, 0, bytes.Length);

        if (context.Response.StatusCode != 200) return null;

        return BuildCachedPage(context, bytes);
    }
}

If nothing has been written to the response or if the status code is not 200 then we return null and do not attempt to cache the response. Otherwise we return a CachedPage instance:

internal class CachedPage
{
    public byte[] Content { get; private set; }
    public List<KeyValuePair<string, StringValues>> Headers { get; private set; }

    public CachedPage(byte[] content)
    {
        Content = content;
        Headers = new List<KeyValuePair<string, StringValues>>();
    }
}

The cached page contains the content itself plus a subset of response headers (we want to remove some headers which it doesn't make sense to save such as date).

Controlling client caching

When it comes to sending caching headers to the browser, we go for the simple approach:

public void ApplyClientHeaders(HttpContext context)
{
    context.Response.OnStarting(() =>
    {
        var clientCacheDuration = GetCacheDuration(context, Constants.ClientDuration);

        if (clientCacheDuration.HasValue && context.Response.StatusCode == 200)
        {
            if (clientCacheDuration == TimeSpan.Zero)
            {
                context.Response.GetTypedHeaders().CacheControl = new CacheControlHeaderValue
                {
                    NoCache = true,
                    NoStore = true,
                    MustRevalidate = true
                };
                context.Response.Headers["Expires"] = "0";
            }
            else
            {
                context.Response.GetTypedHeaders().CacheControl = new CacheControlHeaderValue
                {
                    Public = true,
                    MaxAge = clientCacheDuration
                };
            }
        }

        return Task.CompletedTask;
    });            
}

Remember that the ClientDuration value is set by our action filter. There are three different options:

• Unset - do not send headers
• Zero - set various headers instructing downstream clients not to cache the response
• Other - set the cache headers to the provided value

The cache

One thing we haven't discussed yet is the actual cache. The _cache references in the above code actual point to a wrapper class around the built-in IMemoryCache implementation.

public class CacheClient : ICacheClient
{
    private readonly IMemoryCache _cache;
    
    public CacheClient(IMemoryCache cache)
    {
        _cache = cache ?? throw new ArgumentNullException(nameof(cache));
    }

    internal bool TryGet<T>(string key, out T entry)
    {
        return _cache.TryGetValue(Constants.CacheKeyPrefix + key, out entry);
    }

    internal void Set(string key, object entry, TimeSpan expiry, params string[] tags)
    {
        var options = new MemoryCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = expiry
        };

        var allTokenSource = _cache.GetOrCreate(Constants.CacheTagPrefix + Constants.AllTag, 
            allTagEntry => new CancellationTokenSource());

        options.AddExpirationToken(new CancellationChangeToken(allTokenSource.Token));

        foreach (var tag in tags)
        {
            var tokenSource = _cache.GetOrCreate(Constants.CacheTagPrefix + tag, tagEntry =>
            {
                tagEntry.AddExpirationToken(new CancellationChangeToken(allTokenSource.Token));

                return new CancellationTokenSource();
            });

            options.AddExpirationToken(new CancellationChangeToken(tokenSource.Token));
        }

        _cache.Set(Constants.CacheKeyPrefix + key, entry, options);
    }

The Set method makes use of expiration tokens to allow us to remove cache entries in bulk.

If you have not used CancellationTokenSource before then it may be a little hard to understand but the idea is that we store a CancellationTokenSource for all entries in the cache and additional CancellationTokenSources for each tag we specify. We then use these CancellationTokenSources to generate a token which is passed in when we set the cache entry. We'll look at how to use the CancellationTokenSources to invalidate cache entries in bulk in the next section.

Cache invalidation AKA cache busting

The primary reason for implementing this custom approach to response caching was to satisfy a requirement to be able to remove cache entries before they naturally expire. Our CacheClient class exposes the following methods for this:

public void Remove(string key)
{
    _cache.Remove(Constants.CacheKeyPrefix + key);
}

public void RemoveByTag(string tag)
{
    if (_cache.TryGetValue(Constants.CacheTagPrefix + tag, out CancellationTokenSource tokenSource))
    {
        tokenSource.Cancel();

        _cache.Remove(Constants.CacheTagPrefix + tag);
    }            
}

public void RemoveAll()
{
    RemoveByTag(Constants.AllTag);
}

As you can see, removing a single cache entry is as simple as specifying the key. To make it more user friendly, you may wish to allow action / controller / route values to be specified instead. This is the approach I chose when implementing MvcDonutCaching many years ago.

The RemoveAll and RemoveByTag methods retrieve the CancellationTokenSource from the cache and call the Cancel() method which expires all tokens issued by the source. This results in the removal of these cache entries.

Limitations

This is a very basic example of response caching and lacks many of the features that are present in the native response caching middleware. Perhaps the most notable omission is the ability to vary the caching based on headers, cookies etc. If you wanted to add in VaryBy then it would not be particularly difficult. Effectively all we are doing is changing how we generate the cache key (and allowing configuration via the CacheAttribute.

We are also using IMemoryCache instead of the more scalable IDistributedCache. As noted above, changing the usage is easily done but will result in reduced functionality. IDistributedCache does not support expiration tokens so bulk removal of pages would not be possible using the outlined technique. Of course, there is nothing stopping you from coming up with another solution but naive implementations will almost certainly adversely affect performance.

Summary

This post outlined how to create basic response caching middleware which allows you to manually invalidate entries both individually and in bulk using tags. We used an action filter in conjunction with a middleware component to allow us to save pages (or other action results such as JSON) to an in-memory cache. We then exposed several method on our CacheClient class to allow the removal of cache entries.

It is important to be aware that the example code is not a replacement for the built-in response caching which encompasses far more functionality, but it may be useful in some situations.

Comments

Name Email Comment

Add Comment