Varnish, caching and HTTP cookies

These short notes related on caching and HTTP cookies are based on my experience with Varnish and Plone CMS and WordPress.

Sanifying cookies for caching

Any cookie set on the server side (session cookie) or on the client-side (e.g. Google Analytics Javascript cookies) is poison for caching the anonymous visitor content.

Common cookies for all CMS systems are usually

  • Session cookie (anonymous user session): ZopeId,  PHPSESSID
  • Logged in user cookie: __ac (Plone)
  • Active language cookie: I18N_LANGUAGE (Plone)
  • Analytics cookies (Google Analytics et. al.): various ugly cookies
  • Some other status information e.g. status message: statusmessages (Plone)

HTTP caching needs to deal with both HTTP request and response cookie handling

  • HTTP request Cookie header. The browser sending HTTP request with Cookie header confuses Varnish cache look-up. This header can be set by Javascript also, not just by the server. Cookie can be preprocessed in vcl_recv.
  • HTTP response Set-Cookie header. This is server-side cookie set. If your server is setting cookies Varnish does not cache these responses by default. Howerver, this might be desirable behavior if e.g. multi-lingual content is served from one URL with language cookies. Set-Cookie can be post-processed in vcl_fetch.

Example how remove all Plone related cookies besides ones dealing with the logged in users (content authors):

sub vcl_recv {

  if (req.http.Cookie) {
      # (logged in user, status message - NO session storage or language cookie)
      set req.http.Cookie = ";" req.http.Cookie;
      set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
      set req.http.Cookie = regsuball(req.http.Cookie, ";(statusmessages|__ac)=", "; \1=");
      set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
      set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

      if (req.http.Cookie == "") {
          remove req.http.Cookie;
      }
  }
  ...

# Let's not remove Set-Cookie header in VCL fetch
sub vcl_fetch {

    # Here we could unset cookies explicitly,
    # but we assume plone.app.caching extension does it jobs
    # and no extra cookies fall through for HTTP responses we'd like to cache
    # (like images)

    if (!obj.cacheable) {
        return (pass);
    }
    if (obj.http.Set-Cookie) {
        return (pass);
    }
    set obj.prefetch =  -30s;
    return (deliver);
}

Another example how to purge Google Analytics cookies only and leave other cookies intact:

sub vcl_recv {

         # Remove Google Analytics cookies - will prevent caching of anon content
         # when using GA Javascript. Also you will lose the information of
         # time spend on the site etc..
         if (req.http.cookie) {
            set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
            if (req.http.cookie ~ "^ *$") {
                remove req.http.cookie;
            }
          }

          ....

 Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

11 thoughts on “Varnish, caching and HTTP cookies

  1. Great post !

    How do you manage cookies used by the load balancer for sticky session ? We use a load balancer after varnish.

  2. For Plone, the only thing I’ve really had to do with Varnish is convert the I18N_LANGUAGE cookie to the Accept-Language header. This then allows you to Vary responses on Accept-Language and you can bypass the Varnish logic that avoids caching requests with cookies (it’s only there for badly behaved backends which do not set their Vary headers correctly.)

    You shouldn’t really be adding cookies to the hash, really that’s what Vary is for. Once you’ve cleaned out your analytics cookies from you can Vary on the Cookie header too. Though I tend to find it easier to just add an X-Anonymous-User header to requests without an authentication cookie and Vary on that (given I will only cache anonymous pages in Varnish.)

  3. i get:

    Message from VCC-compiler:
    Expected an action, ‘if’, ‘{‘ or ‘}’
    (‘Default’ Line 40 Pos 1)
    sub vcl_recv {
    ###———–

    Running VCC-compiler failed, exit 1

    VCL compilation failed

  4. Varnish configuration syntax has changed in the latest version. I recommend you to study Varnish manual and apply the example using the current syntax and best practices.

  5. I need to allow PHPSESSID cookie just for one subdomain. Can you help me?

  6. We have a LB with sticky sessions enabled under after the Varnish. Seems like Varnish is deleting the cookies from the request and hence the we are seeing issues as if sticky sessions are not enabled. We have verified that directly connecting to LB doesn’t show the bug. Below is our varnish config (default backend is the one that is causing issues):

    vcl 4.0;
    # Default backend definition. Set this to point to your content server.
    backend default {
    .host = “192.168.40.180”;
    .port = “80”;
    }

    # Custom backend definitions.
    backend cfprod {
    .host = “192.168.40.20”;
    .port = “80”;
    }
    backend mag_aut {
    .host = “192.168.40.181”;
    .port = “80”;
    }
    sub vcl_recv {
    # Happens before we check if we have this in cache already. Typically you clean up the request here, removing cookies you don’t need, rewriting the request, etc.

    set req.backend_hint = default;
    #Custom rules
    if ( req.http.host ~ “^.*\.prod1-varnish\.prod\.vdc” ) {
    set req.backend_hint = cfprod;
    }
    if ( req.url ~ “^/vwbihpezn” ) {
    set req.backend_hint = mag_aut;
    }
    }

    sub vcl_backend_response {
    # Happens after we have read the response headers from the backend.
    # Here you clean the response headers, removing silly Set-Cookie headers
    # and other mistakes your backend does.
    }
    sub vcl_deliver {
    # Happens when we have all the pieces we need, and are about to send the
    # response to the client.
    # You can do accounting or modifying the final object here.
    }

    Nothing in the configs indicate to clean up the cookies but is there any default behavior of Varnish that can do it?
    Thanks in advance.

Leave a Reply

Your email address will not be published. Required fields are marked *