Advanced Concepts

Cache Varies

Cache objects are stored and retrieved using a Cache Key. An optional component of the cache key is the "vary string." It's this vary string that allows you to save multiple cache objects from the same URL.

This is useful in a number of situations including those where the desktop and mobile views of a page are different, or where the currency displayed on a page varies by the visitor's geographical location.

Vary Types

There are two types of cache varies, which may be used together to make up the vary string component of the cache key: vary cookies and a vary environment value.

Vary Cookies

When you instruct the the LiteSpeed Cache Engine to vary on a cookie, you are not actually setting the cookie. You are telling the Cache Engine to look for the cookie name, and then vary on the value of that cookie. Vary strings can contain multiple vary cookies.

Example

Suppose the server is instructed to vary on the cookie my_cookie, and the following four requests come in:

  • Request A has NO cookies, so the vary string is blank.
  • Request B has my_cookie with a value of Alabama, so the vary string is my_cookie = Alabama.
  • Request C has my_cookie with a value of California, so the vary string is my_cookie = California.
  • Request D has my_cookie with a value of Alabama, so the vary string is my_cookie = Alabama.

As a result of these four requests, three versions of the URL have been cached: one with a blank vary string (A), one with a my_cookie = Alabama vary string (B and D), and one with a my_cookie = California vary string (C).

Tip

By default, LiteSpeed servers will recognize any cookie that starts with _lscache_vary as a vary cookie. This cookie is always added as part of the cache key, unless the no-vary cache-control is set in the response header with X-litespeed-cache-control: no-vary.

Vary Environment Value

Unlike vary cookies, a vary value is not made up of a key/value pair. It is simply an environment value, and it may only be used once. If multiple vary values are set, only the last one is used. A vary value is often used to indicate that the request is coming from a mobile device, or which country the request is coming from, among other things.

Example

  • Request A has no vary value, so the vary string is blank.
  • Request B sets a vary value of ismobile, so the vary string is ismobile.
  • Request C sets vary value US, so the vary string is US.
  • Request D sets vary value US, so the vary string is US.

As a result of these four requests, three versions of the URL have been cached: one with a blank vary string (A), one with a ismobile vary string (B), and one with a US vary string (C and D).

Mixed Varies

Multiple vary cookies and a vary value may all be set and used together to build the vary string.

Example

Suppose the server is instructed to vary on the cookies my_cookie and my_cookie2.

  • Request A has no cookies and no vary value is set, so the vary string is blank.
  • Request B has the cookie my_cookie=Alabama and no vary value is set, so the vary string is my_cookie=Alabama.
  • Request C has the cookie my_cookie=Alabama and vary value ismobile is set, so the vary string is my_cookie=Alabama + ismobile.
  • Request D has the cookies my_cookie=Alabama and my_cookie2=Apple and vary value ismobile is set, so the vary string is my_cookie=Alabama&my_cookie2=Apple + ismobile
  • Request E has no cookies, but vary value ismobile is set, so the vary string is ismobile.

The five requests will all be served a different cache entry, because their final VARY results are different.

Implementing Varies

In general, response headers are preferred for vary cookies, and rewrite rules are not required. Once a vary cookie is set by the browser, it will continue to be sent with each request for that URL until that cookie expires or is unset. Therefore, a rewrite rule is not required for retrieval of the cache object.

It's different with a vary value, as it is not passed along by the browser. You need to use both rewrite rules and response headers together to implement a cache vary based on a vary value. The response header instructs the cache engine to store the vary, and a companion rewrite rule uses the vary value for retrieval of the cache object.

Response Headers

Response headers are used to instruct the server to cache the page with the listed varies in mind.

Adding a vary header with cookie= instructs the server to check for the specified vary cookie(s) for the current URL only.

Adding a vary header with value= instructs the server to add the environment value to the cache entry. This is useful for situations where the environment value should be set, but the rewrite rule missed setting it. Just be aware that using a response header vary value in this way, to alleviate the shortcomings of a set of rewrite rules, is not always going to work. The proper fix for this situation would be to correct the rewrite rule to match the web application.

It is also possible to set both vary cookies and vary values using a single response header.

Examples

This response header will add two vary cookies, my_cookie and my_cookie2, to the list of varies to check for this URL. On the next request for this URL, the server will check for the these two cookies in addition to those set by the rewrite rules. The cache engine will act as if these two response header cookies were part of the rewrite rule.

X-LiteSpeed-Vary: cookie=my_cookie,cookie=my_cookie2
This response header will add the ismobile environment value to the cache key for the URL. This is useful if the web app knows is it building a mobile view, but the rewrite rule did not match a mobile user agent.
X-LiteSpeed-Vary: value=ismobile
This response header will vary the URL on the my_cookie cookie and add the ismobile environment value to the current response.
X-LiteSpeed-Vary: cookie=my_cookie,value=ismobile

Rewrite Rules

Rewrite rules can be used to vary the request when it comes in, and serve the correct version of the cached URL in response.

If no matching cache object is found and the request is forwarded to the web app, the varies set by the rewrite rules can be accessed by the web app via environment variables. In PHP, these environment variables are $_SERVER['LSCACHE_VARY_COOKIE'] (the list of cookie names that the server checked) and $_SERVER['LSCACHE_VARY_VALUE'] (the vary value used). These values may be empty, if no varies of either kind are set.

Examples

This rule instructs the server to check for the my_cookie cookie, and if it exists, vary on the value of it:

RewriteRule .? - [E=Cache-Vary:my_cookie]
In this rule, the server must check for multiple cookies:
RewriteRule .* - [E="cache-vary:xf_style_id,xf_language_id"]
Here, the rewrite rule checks the user agent to see if it's in a list of mobile browsers. If it matches, it will add the vary value ismobile to the request.
RewriteCond %{HTTP_USER_AGENT} Mobile|Android|Silk/|Kindle|BlackBerry|Opera Mini|Opera Mobi [NC]
RewriteRule .* - [E=Cache-Control:vary=ismobile]

Tip

Examine the examples closely. Note that for vary cookies, you would use E=Cache-Vary:, and for the vary environment value, you must use E=Cache-Control:vary=.

ESI

It is assumed that you have a general understanding of ESI use cases. Please visit our blog post introducing ESI to ensure a basic understanding of this topic before proceeding.

LiteSpeed can parse the standard ESI tags as well as some LiteSpeed-specific(LSS) tags. The LiteSpeed-specific tags may be used in cases where minifying applications or PageSpeed modules are not built to parse the standard ESI tags. In addition to HTML tags, LiteSpeed also has LSS attributes to assist cache lookup and storage.

HTML Tags

Inline

The esi:inline block may be used to cache a section of code separately. It requires an opening and closing tag.

Tag Syntax: <esi:inline> or <esi_inline> (LSS)

Attributes:

  • src - The resource url.
  • cache-tag - Used to classify content for cache storage.
  • cache-control - Used to determine whether content should be cached and how to store it, as with a traditional cache-control header.

Include

This tag generates a second request that is loaded after the main page is loaded. The esi:include response headers should contain the various cache headers that are needed to determine how the content is to be cached. It does not require a closing tag. Multiple levels of ESI includes are permitted. The maximum number of levels is 10.

Tag Syntax: <esi:include> or <esi_include> (LSS)

Attributes:

  • src - The resource url.
  • cache-tag - Used for private shared cache lookups. Generally, it should be unique.
  • cache-control - Used for cache lookups. May specify no-vary and either public or private.
  • as-var - Sett this value to store the HTML output during parsing. This is useful if the ESI block needs to be included again later in the main file. This attribute will reuse the HTML rather than going back to the backend to regenerate it.
  • test - Test against a condition before doing the ESI include. If the condition is met, parse the ESI request. Else do not output.
  • combined - Value is either parent or sub. A combined ESI include will parse the rest of the main request for any other ESI includes with the combined=sub attribute. After parsing, a POST request is sent to the backend. The post body will include the list of ESI includes to parse, so everything is returned in a single response. The post should have the cache control values no-cache and esi=on. Each individual ESI include block should be wrapped with an ESI inline.

Remove

This tag may not include nested ESI tags within it. It's used as a backup in case there is no ESI processor, and is placed after another ESI tag (e.g. esi:include). If there is an ESI processor, anything inside the esi:remove tag is ignored. Else, the ESI tags will be ignored, and anything inside this tag will be processed normally.

Tag Syntax: <esi:remove> or <esi_remove> (LSS)

Choose | When | Otherwise

This tag is like an ESI version of an if/else if/else statement. It must have an opening and closing tag.

Tag Syntax: <esi:choose> | <esi:when> | <esi:otherwise> or <esi_choose> | <esi_when> | <esi_otherwise> (LSS)

A list of usable variables and expressions are listed at w3.org.

Try | Attempt |Except

These tags work like an ESI try/catch. Only esi:attempt and esi:except tags are allowed immediately inside esi:try.

Tag Syntax: <esi:try> | <esi:attempt> | <esi:except> or <esi_try> | <esi_attempt> | <esi_except> (LSS)

Comment

This tag may be used to comment ESI logic. It is useful if a developer wishes to leave a comment on an HTML page without it showing up in the processed HTML.

Tag Syntax: <esi:comment> or <esi_comment> (LSS)

Vars

Enables the usage of ESI variables (listed at w3.org)

Tag Syntax: <esi:vars> or <esi_vars> (LSS)

Enabling ESI Support

The server needs to know when to parse for ESI tags in the response body. There are two ways to enable ESI: rewrite rules and response headers.

Regardless of which method you use to enable ESI, the following Apachy-style directive must exist in .htaccess in order to activate the cache engine.

<IfModule LiteSpeed>
RewriteEngine on
CacheLookup on
</IfModule>

Rewrite Rules

Rewrite rules should be used for broad requirements, if many pages need to check for ESI. This rewrite rule activates ESI support for all pages:

RewriteRule .? - [E=esi_on:1]

If you only want to use ESI on a specific page, you can only enable it only for that page:

RewriteRule /path/to/specific-page - [E=esi_on:1]  

Response Headers

Response Headers should be used if your scope is more narrow, as on a page by page basis. The X-LiteSpeed-Cache-Control response header needs to include esi=on.

This notifies the server to parse the response for ESI tags. Upon finding an ESI tag, the server will use the attributes to search for a cache entry and if not found, make the ESI request. If the no-vary cache control attribute is set, the varies will not be used to locate or store in the cache.

The ESI resource needs to set response headers in order to be cached correctly. X-LiteSpeed-Cache-Control needs to be set like a normal cache entry (i.e. public/private/shared/no-cache, max-age), like so:

<?php
header('X-LiteSpeed-Cache-Control: public, max-age=120, esi=on');
...
...
...
?>

ESI Example

You have page with 3 lines of content. Line 1 and line 3 are cache friendly, but line 2 is not.

<?php
echo 'line 1 - cache friendly';
echo "line 2 - cache unfriendly, generate a random number: " . rand(1,999);
echo 'line 3 - cache friendly';
?>

With ESI we can cache this page while punching a hole for line 2 and leaving that content uncached.

Add the following code to .htaccess to enable ESI:

<IfModule LiteSpeed>
RewriteEngine on
CacheLookup on
RewriteRule PAGE_URI - [E=esi_on:1]
RewriteRule PAGE_URI - [E=cache-control:max-age=120]
</IfModule>

Notes

  • PAGE_URI is a placeholder. Replace it with the actual URI of the page.
  • max-age=120 indicates that the content should be cached for 120 seconds.

Create the main PHP file:

<?php
echo 'line 1 - cache friendly';
echo '<esi:include src="/second.php" cache-control="no-cache"/>';
echo 'line 3 - cache friendly';
?>

Create a second PHP file which contains the code that is not cache-friendly:

<?php
echo "line 2 - cache unfriendly, generate a random number: " . rand(1,999);
?>

When you access this page, you will see the X-Litespeed-Cache: hit header, but with each refresh, you will see a new random number. This is proof that line 2 has not been cached.

You can also do this by using PHP to send the cache header.

Create .htaccess with the following code:

<IfModule LiteSpeed>
RewriteEngine on
CacheLookup on
</IfModule>

Create the main PHP file:

<?php
header('X-LiteSpeed-Cache-Control: public, max-age=120, esi=on');
echo 'line 1 - cache friendly';
echo '<esi:include src="/second.php" cache-control="no-cache"/>';
echo 'line 3 - cache friendly';
?>

Understanding Response Status Codes 200 and 304

When content is requested, the response comes with a code of either 200 or 304, if successful.

An HTTP response code of 200 OK indicates that the request has succeeded, and the requested content is included in the response body.

304 Not Modified indicates that the document requested by the browser has not changed since the last request, and no content is being transmitted. This conditional fetching is a way of saving bandwidth by telling the browser "just use the copy you already have cached," whenever possible.

In fact, the LiteSpeed Cache engine uses a similar conditional fetch internally when retrieving the content (during the "LSWS checks the cache key for a cache object" step of the above scenarios).

The cache-control response header is applied to client-side browser cache, and is used to determine how often the browser should check for an updated copy of a document. The result that is returned can either be 304 Not Modified (the content has not changed since the last time the browser requested it), or 200 OK along with a copy of the requested document (the content has changed, and here is a new copy).

Here are a few examples to show how this works.

Public Cache Example

The page includes the following response header:

cache-control: max-age=300, public
This indicates that the browser will check with the server once every 300 seconds, to determine whether the document has changed. The server will respond with 304 Not Modified, if it has not.

If the document has changed, the server will respond with 200 OK, and will deliver the updated content.

No Cache Example

The page includes the following response header:

cache-control: no-cache
You might think this means the content is not cached locally, but that's not the case. All this means is that the browser will always check with the server to determine whether the document has changed. The server will respond with 304 Not Modified, if it has not.

If the document has changed, the server will respond with 200 OK, and will deliver the updated content.

No Store Example

The page includes the following response header:

cache-control: no-cache,no-store
This instructs the brower to never cache or store the HTML response. As such, there is nothing to check against and the server will always respond with 200 OK and a copy of the document.

You can learn more about what the cache-control header does in Mozilla's documentation, and in this document by Google's web.dev team.


Last update: July 31, 2020