Purging Varnish cache from Python web application

Varnish is a popular front end caching server. If you are using it at the front of your web site you have situations where you wish to force the cached content out of the cache E.g. site pages have been updated.

Below is an example how to create an action to purge selected pages from the Varnish cache. The example shows how to purge the whole cache, but you can use regular expressions to limit the effect of the purge.

This tutorial has been written for Ubuntu 8.04, Python 2.6, Varnish 2.x and Plone 4.x. However, it should be trivial to apply the instructions for other systems.

We use Requests library which is a great improvement over Python stdlib’s infamous urllib. After you try Request you never want to use urllib or curl again.

Configuring Varnish

First you need to allow HTTP PURGE request handler in default.vcl from localhost. We’ll create a special PURGE command which takes URLs to be purged out of the cache in a special header:

acl purge_hosts {
        "localhost";
        # XXX: Add your local computer public IP here if you
        # want to test the code against the production server
        # from the development instance
}

...

sub vcl_recv {

        ...

        # PURGE requests call purge() with a regular expression payload from
        if (req.request == "PURGE") {
                # Check IP for access control list
                if (!client.ip ~ purge_hosts) {
                        error 405 "Not allowed.";
                }
                # Purge for the current host using reg-ex from X-Purge-Regex header
                purge("req.http.host == " req.http.host " && req.url ~ " req.http.X-Purge-Regex);
                error 200 "Purged.";
        }
}

Creating a view

Then let’s create a Plone view which will make a request from Plone to Varnish (upstream localhost:80) and issue PURGE command. We do this using Requests Python lib.

This is useful when very fine tuned per page purgingn fails and the site admins simply want to make sure that everything from the front end cache will be gone.

This view is created so that it can be hooked in portal_actions menu and only the site admins have permission to call it.

Example view code:

import requests

from Products.CMFCore.interfaces import ISiteRoot
from five import grok

# XXX: The followig monkey-patch is unneeded soon
# as the author updated Requets library to handle this case

from requests.models import Request

if not "PURGE" in Request._METHODS:
    lst = list(Request._METHODS)
    lst.append("PURGE")
    Request._METHODS = tuple(lst)

class Purge(grok.CodeView):
    """
    Purge upstream cache from all entries.

    This is ideal to hook up for admins e.g. through portal_actions menu.

    You can access it as admin:: http://site.com/@@purge
    """

    grok.context(ISiteRoot)

    # Onlyl site admins can use this
    grok.require("cmf.ManagePortal")

    def render(self):
        """
        Call the parent cache using Requets Python library and issue PURGE command for all URLs.

        Pipe through the response as is.
        """

        site_url = "http://www.site.com/"

        # We are cleaning everything, but we could limit the effect of purging here
        headers = {
                   # Match all pages
                   "X-Purge-Regex" : ".*"
        }

        resp = requests.request("PURGE", site_url, headers=headers)

        self.request.response["Content-type"] = "text/plain"
        text = []

        text.append("HTTP " + str(resp.status_code))

        # Dump response headers as is to the Plone user,
        # so he/she can diagnose the problem
        for key, value in resp.headers.items():
            text.append(str(key) + ": " + str(value))

        # Add payload message from the server (if any)

        text.append(str(resp.body))

        return "\n".join(text)

Now if you are a Plone site admin and visit in URL:

http://site.com/@@purge

You’ll clear the front end cache.

More info

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Leave a Reply

Your email address will not be published. Required fields are marked *