From a8eb53fb79981e1b6fb93af3c80a6bdbae7f9d8f Mon Sep 17 00:00:00 2001 From: Chris Rossi Date: Mon, 14 Jul 2014 11:13:28 -0400 Subject: Narrative scifi. --- docs/narr/assets.rst | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index b0a8d18b0..a2976de22 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -286,6 +286,71 @@ the application is being run in development or in production (use a different suggestion for a pattern; any setting name other than ``media_location`` could be used. +.. _cache_busting: + +Cache Busting +------------- + +In order to maximize performance of a web application, you generally want to +limit the number of times a particular client requests the same static asset. +Ideally a client would cache a particular static asset "forever", requiring +it to be sent to the client a single time. The HTTP protocol allows you to +send headers with an HTTP response that can instruct a client to cache a +particular asset for an amount of time. As long as the client has a copy of +the asset in its cache and that cache hasn't expired, the client will use the +cached copy rather than request a new copy from the server. The drawback to +sending cache headers to the client for a static asset is that at some point +the static asset may change, and then you'll want the client to load a new copy +of the asset. Under normal circumstances you'd just need to wait for the +client's cached copy to expire before they get the new version of the static +resource. + +A commonly used workaround to this problem is a technique known as "cache +busting". Cache busting schemes generally involve generating a URL for a +static asset that changes when the static asset changes. This way headers can +be sent along with the static asset instructing the client to cache the asset +for a very long time. When a static asset is changed, the URL used to refer to +it in a web page also changes, so the client sees it as a new resource and +requests a copy, regardless of any caching policy set for the resource's old +URL. + +:app:`Pyramid` can be configured to produce cache busting URLs for static +assets by passing the optional argument, `cache_bust` to +:meth:`~pyramid.config.Configurator.add_static_view`: + +.. code-block:: python + :linenos: + + # config is an instance of pyramid.config.Configurator + config.add_static_view(name='static', path='mypackage:folder/static', + cache_bust='md5') + +Supplying the `cache_bust` argument instructs :app:`Pyramid` to add a query +string to URLs generated for this static view which includes the md5 checksum +of the static file being served: + +.. code-block:: python + :linenos: + + js_url = request.static_url('mypackage:folder/static/js/myapp.js') + # Returns: 'http://www.example.com/static/js/myapp.js?md5=c9658b3c0a314a1ca21e5988e662a09e` + +When the asset changes, so will its md5 checksum, and therefore so will its +URL. Supplying the `cache_bust` argument also causes the static view to set +headers instructing clients to cache the asset for ten years, unless the +`max_cache_age` argument is also passed, in which case that value is used. + +.. note:: + + `md5` is currently the only possible value for the `cache_bust` argument to + :meth:`~pyramid.config.Configurator.add_static_view`. + +.. note:: + + md5 checksums are cached in RAM so if you change a static resource without + restarting your application, you may still generate URLs with a stale md5 + checksum. + .. index:: single: static assets view -- cgit v1.2.3 From f729a1e7f1efc27a6df1ae0eaca7fdffdd86ec2f Mon Sep 17 00:00:00 2001 From: Chris Rossi Date: Thu, 17 Jul 2014 16:04:28 -0400 Subject: Write the documentation. --- docs/narr/assets.rst | 72 ++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 59 insertions(+), 13 deletions(-) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index a2976de22..97d473761 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -315,7 +315,7 @@ requests a copy, regardless of any caching policy set for the resource's old URL. :app:`Pyramid` can be configured to produce cache busting URLs for static -assets by passing the optional argument, `cache_bust` to +assets by passing the optional argument, ``cachebuster`` to :meth:`~pyramid.config.Configurator.add_static_view`: .. code-block:: python @@ -323,27 +323,22 @@ assets by passing the optional argument, `cache_bust` to # config is an instance of pyramid.config.Configurator config.add_static_view(name='static', path='mypackage:folder/static', - cache_bust='md5') + cachebuster=True) -Supplying the `cache_bust` argument instructs :app:`Pyramid` to add a query -string to URLs generated for this static view which includes the md5 checksum -of the static file being served: +Setting the ``cachebuster`` argument instructs :app:`Pyramid` to use a cache +busting scheme which adds the md5 checksum for a static asset as a path segment +in the asset's URL: .. code-block:: python :linenos: js_url = request.static_url('mypackage:folder/static/js/myapp.js') - # Returns: 'http://www.example.com/static/js/myapp.js?md5=c9658b3c0a314a1ca21e5988e662a09e` + # Returns: 'http://www.example.com/static/c9658b3c0a314a1ca21e5988e662a09e/js/myapp.js` When the asset changes, so will its md5 checksum, and therefore so will its -URL. Supplying the `cache_bust` argument also causes the static view to set +URL. Supplying the ``cachebuster`` argument also causes the static view to set headers instructing clients to cache the asset for ten years, unless the -`max_cache_age` argument is also passed, in which case that value is used. - -.. note:: - - `md5` is currently the only possible value for the `cache_bust` argument to - :meth:`~pyramid.config.Configurator.add_static_view`. +``max_cache_age`` argument is also passed, in which case that value is used. .. note:: @@ -351,6 +346,57 @@ headers instructing clients to cache the asset for ten years, unless the restarting your application, you may still generate URLs with a stale md5 checksum. +Customizing the Cache Buster +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Revisiting from the previous section: + +.. code-block:: python + :linenos: + + # config is an instance of pyramid.config.Configurator + config.add_static_view(name='static', path='mypackage:folder/static', + cachebuster=True) + +Setting ``cachebuster`` to ``True`` instructs :app:`Pyramid` to use a default +cache busting implementation that should work for many situations. The +``cachebuster`` may be set to any object that implements the interface, +:class:`~pyramid.interfaces.ICacheBuster`. The above configuration is exactly +equivalent to: + +.. code-block:: python + :linenos: + + from pyramid.static import ( + Md5AssetTokenGenerator, + PathSegmentCacheBuster) + + # config is an instance of pyramid.config.Configurator + cachebuster = PathSegmentCacheBuster(Md5AssetTokenGenerator()) + config.add_static_view(name='static', path='mypackage:folder/static', + cachebuster=cachebuster) + +:app:`Pyramid` includes two ready to use cache buster implementations: +:class:`~pyramid.static.PathSegmentCacheBuster`, which inserts an asset token +in the path portion of the asset's URL, and +:class:`~pyramid.static.QueryStringCacheBuster`, which adds an asset token to +the query string of the asset's URL. Both of these classes have constructors +which accept a token generator function as an argument, allowing for the way a +token is generated to be decoupled from the way it is inserted into a URL. +:app:`Pyramid` provides a single asset token generator, +:meth:`~pyramid.static.Md5AssetTokenGenerator`. + +In order to implement your own cache buster, see the +:class:`~pyramid.interfaces.ICacheBuster` interface and the existing +implementations in the :mod:`~pyramid.static` module. + +.. note:: + + Many HTTP caching proxy implementations will fail to cache any URL which + has a query string. For this reason, you should probably prefer + :class:`~pyramid.static.PathSegementCacheBuster` to + :class:`~pyramid.static.QueryStringCacheBuster`. + .. index:: single: static assets view -- cgit v1.2.3 From 002da7991f4433e5fd5a07489038a6bd2720a526 Mon Sep 17 00:00:00 2001 From: Chris Rossi Date: Thu, 17 Jul 2014 16:10:52 -0400 Subject: Add index entry. --- docs/narr/assets.rst | 3 +++ 1 file changed, 3 insertions(+) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index 97d473761..642211f5b 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -286,6 +286,9 @@ the application is being run in development or in production (use a different suggestion for a pattern; any setting name other than ``media_location`` could be used. +.. index:: + single: Cache Busting + .. _cache_busting: Cache Busting -- cgit v1.2.3 From aa96dda157d39c57c0d2fe8399db0b2175fa83d2 Mon Sep 17 00:00:00 2001 From: Chris Rossi Date: Fri, 18 Jul 2014 17:18:56 -0400 Subject: Take mcdonc's advice. This should be easier for users to understand. --- docs/narr/assets.rst | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index 642211f5b..7987d03a6 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -370,28 +370,18 @@ equivalent to: .. code-block:: python :linenos: - from pyramid.static import ( - Md5AssetTokenGenerator, - PathSegmentCacheBuster) + from pyramid.static import PathSegmentCacheBuster # config is an instance of pyramid.config.Configurator - cachebuster = PathSegmentCacheBuster(Md5AssetTokenGenerator()) config.add_static_view(name='static', path='mypackage:folder/static', - cachebuster=cachebuster) + cachebuster=PathSegmentCacheBuster()) :app:`Pyramid` includes two ready to use cache buster implementations: :class:`~pyramid.static.PathSegmentCacheBuster`, which inserts an asset token in the path portion of the asset's URL, and :class:`~pyramid.static.QueryStringCacheBuster`, which adds an asset token to -the query string of the asset's URL. Both of these classes have constructors -which accept a token generator function as an argument, allowing for the way a -token is generated to be decoupled from the way it is inserted into a URL. -:app:`Pyramid` provides a single asset token generator, -:meth:`~pyramid.static.Md5AssetTokenGenerator`. - -In order to implement your own cache buster, see the -:class:`~pyramid.interfaces.ICacheBuster` interface and the existing -implementations in the :mod:`~pyramid.static` module. +the query string of the asset's URL. Both of these classes generate md5 +checksums as asset tokens. .. note:: @@ -400,6 +390,29 @@ implementations in the :mod:`~pyramid.static` module. :class:`~pyramid.static.PathSegementCacheBuster` to :class:`~pyramid.static.QueryStringCacheBuster`. +In order to implement your own cache buster, you can write your own class from +scratch which implements the :class:`~pyramid.interfaces.ICacheBuster` +interface. Alternatively you may choose to subclass one of the existing +implementations. One of the most likely scenarios is you'd want to change the +way the asset token is generated. To do this just subclass an existing +implementation and replace the :meth:`~pyramid.interfaces.ICacheBuster.token` +method. Here is an example which just uses a global setting for the asset +token: + +.. code-block:: python + :linenos: + + from pyramid.static import PathSegmentCacheBuster + + class MyCacheBuster(PathSegmentCacheBuster): + + def __init__(self, config): + # config is an instance of pyramid.config.Configurator + self._token = config.registry.settings['myapp.cachebust_token'] + + def token(self, pathspec): + return self._token + .. index:: single: static assets view -- cgit v1.2.3 From 6596304446f8369dfbcf264d143fe85d75832dba Mon Sep 17 00:00:00 2001 From: Chris Rossi Date: Mon, 21 Jul 2014 16:46:35 -0400 Subject: Add 'prevent_cachebuster' setting. --- docs/narr/assets.rst | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index 7987d03a6..fea3fae48 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -349,6 +349,15 @@ headers instructing clients to cache the asset for ten years, unless the restarting your application, you may still generate URLs with a stale md5 checksum. +Disabling the Cache Buster +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It can be useful in some situations (e.g. development) to globally disable all +configured cache busters without changing calls to +:meth:`~pyramid.config.Configurator.add_static_view`. To do this set the +``PYRAMID_PREVENT_CACHEBUSTER`` environment variable or the +``pyramid.prevent_cachebuster`` configuration value to a true value. + Customizing the Cache Buster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -387,7 +396,7 @@ checksums as asset tokens. Many HTTP caching proxy implementations will fail to cache any URL which has a query string. For this reason, you should probably prefer - :class:`~pyramid.static.PathSegementCacheBuster` to + :class:`~pyramid.static.PathSegmentCacheBuster` to :class:`~pyramid.static.QueryStringCacheBuster`. In order to implement your own cache buster, you can write your own class from -- cgit v1.2.3 From 15b979413c700fbc289328b25aaa4ba1c4cbdda9 Mon Sep 17 00:00:00 2001 From: Chris Rossi Date: Thu, 24 Jul 2014 17:13:08 -0400 Subject: cachebuster -> cachebust --- docs/narr/assets.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index fea3fae48..7fb0ec40b 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -318,7 +318,7 @@ requests a copy, regardless of any caching policy set for the resource's old URL. :app:`Pyramid` can be configured to produce cache busting URLs for static -assets by passing the optional argument, ``cachebuster`` to +assets by passing the optional argument, ``cachebust`` to :meth:`~pyramid.config.Configurator.add_static_view`: .. code-block:: python @@ -326,9 +326,9 @@ assets by passing the optional argument, ``cachebuster`` to # config is an instance of pyramid.config.Configurator config.add_static_view(name='static', path='mypackage:folder/static', - cachebuster=True) + cachebust=True) -Setting the ``cachebuster`` argument instructs :app:`Pyramid` to use a cache +Setting the ``cachebust`` argument instructs :app:`Pyramid` to use a cache busting scheme which adds the md5 checksum for a static asset as a path segment in the asset's URL: @@ -339,7 +339,7 @@ in the asset's URL: # Returns: 'http://www.example.com/static/c9658b3c0a314a1ca21e5988e662a09e/js/myapp.js` When the asset changes, so will its md5 checksum, and therefore so will its -URL. Supplying the ``cachebuster`` argument also causes the static view to set +URL. Supplying the ``cachebust`` argument also causes the static view to set headers instructing clients to cache the asset for ten years, unless the ``max_cache_age`` argument is also passed, in which case that value is used. @@ -355,8 +355,8 @@ Disabling the Cache Buster It can be useful in some situations (e.g. development) to globally disable all configured cache busters without changing calls to :meth:`~pyramid.config.Configurator.add_static_view`. To do this set the -``PYRAMID_PREVENT_CACHEBUSTER`` environment variable or the -``pyramid.prevent_cachebuster`` configuration value to a true value. +``PYRAMID_PREVENT_CACHEBUST`` environment variable or the +``pyramid.prevent_cachebust`` configuration value to a true value. Customizing the Cache Buster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -368,11 +368,11 @@ Revisiting from the previous section: # config is an instance of pyramid.config.Configurator config.add_static_view(name='static', path='mypackage:folder/static', - cachebuster=True) + cachebust=True) -Setting ``cachebuster`` to ``True`` instructs :app:`Pyramid` to use a default +Setting ``cachebust`` to ``True`` instructs :app:`Pyramid` to use a default cache busting implementation that should work for many situations. The -``cachebuster`` may be set to any object that implements the interface, +``cachebust`` may be set to any object that implements the interface, :class:`~pyramid.interfaces.ICacheBuster`. The above configuration is exactly equivalent to: @@ -383,7 +383,7 @@ equivalent to: # config is an instance of pyramid.config.Configurator config.add_static_view(name='static', path='mypackage:folder/static', - cachebuster=PathSegmentCacheBuster()) + cachebust=PathSegmentCacheBuster()) :app:`Pyramid` includes two ready to use cache buster implementations: :class:`~pyramid.static.PathSegmentCacheBuster`, which inserts an asset token -- cgit v1.2.3 From f674a8f691d260d44e0f76e3afecfb15484c45b9 Mon Sep 17 00:00:00 2001 From: Chris Rossi Date: Mon, 28 Jul 2014 17:26:11 -0400 Subject: Mo' features, mo' problems. --- docs/narr/assets.rst | 87 +++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 62 insertions(+), 25 deletions(-) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index 7fb0ec40b..33677988d 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -379,25 +379,19 @@ equivalent to: .. code-block:: python :linenos: - from pyramid.static import PathSegmentCacheBuster + from pyramid.static import PathSegmentMd5CacheBuster # config is an instance of pyramid.config.Configurator config.add_static_view(name='static', path='mypackage:folder/static', - cachebust=PathSegmentCacheBuster()) + cachebust=PathSegmentMd5CacheBuster()) -:app:`Pyramid` includes two ready to use cache buster implementations: -:class:`~pyramid.static.PathSegmentCacheBuster`, which inserts an asset token -in the path portion of the asset's URL, and -:class:`~pyramid.static.QueryStringCacheBuster`, which adds an asset token to -the query string of the asset's URL. Both of these classes generate md5 -checksums as asset tokens. - -.. note:: - - Many HTTP caching proxy implementations will fail to cache any URL which - has a query string. For this reason, you should probably prefer - :class:`~pyramid.static.PathSegmentCacheBuster` to - :class:`~pyramid.static.QueryStringCacheBuster`. +:app:`Pyramid` includes a handful of ready to use cache buster implementations: +:class:`~pyramid.static.PathSegmentMd5CacheBuster`, which inserts an md5 +checksum token in the path portion of the asset's URL, +:class:`~pyramid.static.QueryStringMd5CacheBuster`, which adds an md5 checksum +token to the query string of the asset's URL, and +:class:`~pyramid.static.QueryStringConstantCacheBuster`, which adds an +arbitrary token you provide to the query string of the asset's URL. In order to implement your own cache buster, you can write your own class from scratch which implements the :class:`~pyramid.interfaces.ICacheBuster` @@ -405,22 +399,65 @@ interface. Alternatively you may choose to subclass one of the existing implementations. One of the most likely scenarios is you'd want to change the way the asset token is generated. To do this just subclass an existing implementation and replace the :meth:`~pyramid.interfaces.ICacheBuster.token` -method. Here is an example which just uses a global setting for the asset -token: +method. Here is an example which just uses Git to get the hash of the +currently checked out code: .. code-block:: python :linenos: - - from pyramid.static import PathSegmentCacheBuster - class MyCacheBuster(PathSegmentCacheBuster): - - def __init__(self, config): - # config is an instance of pyramid.config.Configurator - self._token = config.registry.settings['myapp.cachebust_token'] + import os + import subprocess + from pyramid.static import PathSegmentMd5CacheBuster + + class GitCacheBuster(PathSegmentMd5CacheBuster): + """ + Assuming your code is installed as a Git checkout, as opposed to as an + egg from an egg repository like PYPI, you can use this cachebuster to + get the current commit's SHA1 to use as the cache bust token. + """ + def __init__(self): + here = os.path.dirname(os.path.abspath(__file__)) + self.sha1 = subprocess.check_output( + ['git', 'rev-parse', 'HEAD'], + cwd=here).strip() def token(self, pathspec): - return self._token + return self.sha1 + +Choosing a Cache Buster +~~~~~~~~~~~~~~~~~~~~~~~ + +The default cache buster implementation, +:class:`~pyramid.static.PathSegmentMd5CacheBuster`, works very well assuming +that you're using :app:`Pyramid` to serve your static assets. The md5 checksum +is fine grained enough that browsers should only request new versions of +specific assets that have changed. Many caching HTTP proxies will fail to +cache a resource if the URL contains a query string. In general, therefore, +you should prefer a cache busting strategy which modifies the path segment to +a strategy which adds a query string. + +It is possible, however, that your static assets are being served by another +web server or externally on a CDN. In these cases modifying the path segment +for a static asset URL would cause the external service to fail to find the +asset, causing your customer to get a 404. In these cases you would need to +fall back to a cache buster which adds a query string. It is even possible +that there isn't a copy of your static assets available to the :app:`Pyramid` +application, so a cache busting implementation that generates md5 checksums +would fail since it can't access the assets. In such a case, +:class:`~pyramid.static.QueryStringConstantCacheBuster` is a reasonable +fallback. The following code would set up a cachebuster that just uses the +time at start up as a cachebust token: + +.. code-block:: python + :linenos: + + import time + from pyramid.static import QueryStringConstantCacheBuster + + config.add_static_view( + name='http://mycdn.example.com/', + path='mypackage:static', + cachebust=QueryStringConstantCacheBuster(str(time.time()))) .. index:: single: static assets view -- cgit v1.2.3 From 6b88bdf7680151345debec0c8651f164a149a53a Mon Sep 17 00:00:00 2001 From: Chris McDonough Date: Mon, 28 Jul 2014 21:06:34 -0400 Subject: add versionadded notes --- docs/narr/assets.rst | 2 ++ 1 file changed, 2 insertions(+) (limited to 'docs/narr/assets.rst') diff --git a/docs/narr/assets.rst b/docs/narr/assets.rst index 33677988d..95863848b 100644 --- a/docs/narr/assets.rst +++ b/docs/narr/assets.rst @@ -294,6 +294,8 @@ could be used. Cache Busting ------------- +.. versionadded:: 1.6 + In order to maximize performance of a web application, you generally want to limit the number of times a particular client requests the same static asset. Ideally a client would cache a particular static asset "forever", requiring -- cgit v1.2.3