Dogpile.Cache Documentation

Transcription

1 Dogpile.Cache Documentation Release Mike Bayer Mar 05, 2018

2

3 Contents 1 Front Matter Project Homepage Installation Bugs Usage Guide Overview Rudimentary Usage Region Configuration Using a Region Creating Backends Changing Backend Behavior Recipes Invalidating a group of related keys Asynchronous Data Updates with ORM Events Prefixing all keys in Redis Encoding/Decoding data into another format dogpile Core Do I Need to Learn the dogpile Core API Directly? Rudimentary Usage Example: Using dogpile directly for Caching Using a File or Distributed Lock with Dogpile API Region Backend API Backends Exceptions Plugins Utilities dogpile Core Changelog i

4 Indices and tables 73 Python Module Index 75 ii

5 Dogpile consists of two subsystems, one building on top of the other. dogpile provides the concept of a dogpile lock, a control structure which allows a single thread of execution to be selected as the creator of some resource, while allowing other threads of execution to refer to the previous version of this resource as the creation proceeds; if there is no previous version, then those threads block until the object is available. dogpile.cache is a caching API which provides a generic interface to caching backends of any variety, and additionally provides API hooks which integrate these cache backends with the locking mechanism of dogpile. New backends are very easy to create and use; users are encouraged to adapt the provided backends for their own needs, as high volume caching requires lots of tweaks and adjustments specific to an application and its environment. Contents 1

6 2 Contents

7 CHAPTER 1 Front Matter Information about the dogpile.cache project. 1.1 Project Homepage dogpile.cache is hosted on Bitbucket - the lead project page is at Source code is tracked here using Git. Releases and project status are available on Pypi at The most recent published version of this documentation should be at Installation Install released versions of dogpile.cache from the Python package index with pip or a similar tool: pip install dogpile.cache 1.3 Bugs Bugs and feature enhancements to dogpile.cache should be reported on the Bitbucket issue tracker. 3

8 4 Chapter 1. Front Matter

9 CHAPTER 2 Usage Guide 2.1 Overview At the time of this writing, popular key/value servers include Memcached, Redis and many others. While these tools all have different usage focuses, they all have in common that the storage model is based on the retrieval of a value based on a key; as such, they are all potentially suitable for caching, particularly Memcached which is first and foremost designed for caching. With a caching system in mind, dogpile.cache provides an interface to a particular Python API targeted at that system. A dogpile.cache configuration consists of the following components: A region, which is an instance of CacheRegion, and defines the configuration details for a particular cache backend. The CacheRegion can be considered the front end used by applications. A backend, which is an instance of CacheBackend, describing how values are stored and retrieved from a backend. This interface specifies only get(), set() and delete(). The actual kind of CacheBackend in use for a particular CacheRegion is determined by the underlying Python API being used to talk to the cache, such as Pylibmc. The CacheBackend is instantiated behind the scenes and not directly accessed by applications under normal circumstances. Value generation functions. These are user-defined functions that generate new values to be placed in the cache. While dogpile.cache offers the usual set approach of placing data into the cache, the usual mode of usage is to only instruct it to get a value, passing it a creation function which will be used to generate a new value if and only if one is needed. This get-or-create pattern is the entire key to the Dogpile system, which coordinates a single value creation operation among many concurrent get operations for a particular key, eliminating the issue of an expired value being redundantly re-generated by many workers simultaneously. 2.2 Rudimentary Usage dogpile.cache includes a Pylibmc backend. A basic configuration looks like: 5

10 from dogpile.cache import make_region region = make_region().configure( 'dogpile.cache.pylibmc', expiration_time = 3600, arguments = { 'url': [" "], } def load_user_info(user_id): return some_database.lookup_user_by_id(user_id) pylibmc In this section, we re illustrating Memcached usage using the pylibmc backend, which is a high performing Python library for Memcached. It can be compared to the python-memcached client, which is also an excellent product. Pylibmc is written against Memcached s native API so is markedly faster, though might be considered to have rougher edges. The API is actually a bit more verbose to allow for correct multithreaded usage. Above, we create a CacheRegion using the make_region() function, then apply the backend configuration via the CacheRegion.configure() method, which returns the region. The name of the backend is the only argument required by CacheRegion.configure() itself, in this case dogpile.cache.pylibmc. However, in this specific case, the pylibmc backend also requires that the URL of the memcached server be passed within the arguments dictionary. The configuration is separated into two sections. Upon construction via make_region(), the CacheRegion object is available, typically at module import time, for usage in decorating functions. Additional configuration details passed to CacheRegion.configure() are typically loaded from a configuration file and therefore not necessarily available until runtime, hence the two-step configurational process. Key arguments passed to CacheRegion.configure() include expiration_time, which is the expiration time passed to the Dogpile lock, and arguments, which are arguments used directly by the backend - in this case we are using arguments that are passed directly to the pylibmc module. 2.3 Region Configuration The make_region() function currently calls the CacheRegion constructor directly. class dogpile.cache.region.cacheregion(name=none, function_key_generator=<function function_key_generator>, function_multi_key_generator=<function function_multi_key_generator>, key_mangler=none, async_creation_runner=none) A front end to a particular cache backend. Parameters name Optional, a string name for the region. This isn t used internally but can be accessed via the.name parameter, helpful for configuring a region from a config file. function_key_generator Optional. A function that will produce a cache key given a data creation function and arguments, when using the CacheRegion. 6 Chapter 2. Usage Guide

11 cache_on_arguments() method. The structure of this function should be two levels: given the data creation function, return a new function that generates the key based on the given arguments. Such as: def my_key_generator(namespace, fn, **kw): fname = fn. name def generate_key(*arg): return namespace + "_" + fname + "_".join(str(s) for s in arg) return generate_key region = make_region( function_key_generator = my_key_generator ).configure( "dogpile.cache.dbm", expiration_time=300, arguments={ "filename":"file.dbm" } ) The namespace is that passed to CacheRegion.cache_on_arguments(). It s not consulted outside this function, so in fact can be of any form. For example, it can be passed as a tuple, used to specify arguments to pluck from **kw: def my_key_generator(namespace, fn): def generate_key(*arg, **kw): return ":".join( [kw[k] for k in namespace] + [str(x) for x in arg] ) return generate_key Where the decorator might be used 'y')) def my_function(a, b, **kw): return my_data() See also: function_key_generator() - default key generator kwarg_function_key_generator() - optional gen that also uses keyword arguments function_multi_key_generator Optional. Similar to function_key_generator parameter, but it s used in CacheRegion. cache_multi_on_arguments(). Generated function should return list of keys. For example: def my_multi_key_generator(namespace, fn, **kw): namespace = fn. name + (namespace or '') def generate_keys(*args): return [namespace + ':' + str(a) for a in args] return generate_keys 2.3. Region Configuration 7

12 key_mangler Function which will be used on all incoming keys before passing to the backend. Defaults to None, in which case the key mangling function recommended by the cache backend will be used. A typical mangler is the SHA1 mangler found at sha1_mangle_key() which coerces keys into a SHA1 hash, so that the string length is fixed. To disable all key mangling, set to False. Another typical mangler is the built-in Python function str, which can be used to convert non-string or Unicode keys to bytestrings, which is needed when using a backend such as bsddb or dbm under Python 2.x in conjunction with Unicode keys. async_creation_runner A callable that, when specified, will be passed to and called by dogpile.lock when there is a stale value present in the cache. It will be passed the mutex and is responsible releasing that mutex when finished. This can be used to defer the computation of expensive creator functions to later points in the future by way of, for example, a background thread, a long-running queue, or a task manager system like Celery. For a specific example using async_creation_runner, new values can be created in a background thread like so: import threading def async_creation_runner(cache, somekey, creator, mutex): ''' Used by dogpile.core:lock when appropriate ''' def runner(): try: value = creator() cache.set(somekey, value) finally: mutex.release() thread = threading.thread(target=runner) thread.start() region = make_region( async_creation_runner=async_creation_runner, ).configure( 'dogpile.cache.memcached', expiration_time=5, arguments={ 'url': ' :11211', 'distributed_lock': True, } ) Remember that the first request for a key with no associated value will always block; async_creator will not be invoked. However, subsequent requests for cached-but-expired values will still return promptly. They will be refreshed by whatever asynchronous means the provided async_creation_runner callable implements. By default the async_creation_runner is disabled and is set to None. New in version 0.4.2: added the async_creation_runner feature. One you have a CacheRegion, the CacheRegion.cache_on_arguments() method can be used to decorate functions, but the cache itself can t be used until CacheRegion.configure() is called. The interface for that method is as follows: 8 Chapter 2. Usage Guide

13 CacheRegion.configure(backend, expiration_time=none, arguments=none, _config_argument_dict=none, _config_prefix=none, wrap=none, replace_existing_backend=false, region_invalidator=none) Configure a CacheRegion. The CacheRegion itself is returned. Parameters backend Required. This is the name of the CacheBackend to use, and is resolved by loading the class from the dogpile.cache entrypoint. expiration_time Optional. The expiration time passed to the dogpile system. May be passed as an integer number of seconds, or as a datetime.timedelta value. The CacheRegion.get_or_create() method as well as the CacheRegion. cache_on_arguments() decorator (though note: not the CacheRegion.get() method) will call upon the value creation function after this time period has passed since the last generation. arguments Optional. The structure here is passed directly to the constructor of the CacheBackend in use, though is typically a dictionary. wrap Optional. A list of ProxyBackend classes and/or instances, each of which will be applied in a chain to ultimately wrap the original backend, so that custom functionality augmentation can be applied. New in version See also: Changing Backend Behavior replace_existing_backend if True, the existing cache backend will be replaced. Without this flag, an exception is raised if a backend is already configured. New in version region_invalidator Optional. Override default invalidation strategy with custom implementation of RegionInvalidationStrategy. New in version The CacheRegion can also be configured from a dictionary, using the CacheRegion. configure_from_config() method: CacheRegion.configure_from_config(config_dict, prefix) Configure from a configuration dictionary and a prefix. Example: local_region = make_region() memcached_region = make_region() # regions are ready to use for function # decorators, but not yet for actual caching # later, when config is available myconfig = { "cache.local.backend":"dogpile.cache.dbm", "cache.local.arguments.filename":"/path/to/dbmfile.dbm", "cache.memcached.backend":"dogpile.cache.pylibmc", "cache.memcached.arguments.url":" , ", (continues on next page) 2.3. Region Configuration 9

14 } local_region.configure_from_config(myconfig, "cache.local.") memcached_region.configure_from_config(myconfig, "cache.memcached.") (continued from previous page) 2.4 Using a Region The CacheRegion object is our front-end interface to a cache. It includes the following methods: CacheRegion.get(key, expiration_time=none, ignore_expiration=false) Return a value from the cache, based on the given key. If the value is not present, the method returns the token NO_VALUE. NO_VALUE evaluates to False, but is separate from None to distinguish between a cached value of None. By default, the configured expiration time of the CacheRegion, or alternatively the expiration time supplied by the expiration_time argument, is tested against the creation time of the retrieved value versus the current time (as reported by time.time()). If stale, the cached value is ignored and the NO_VALUE token is returned. Passing the flag ignore_expiration=true bypasses the expiration time check. Changed in version 0.3.0: CacheRegion.get() now checks the value s creation time against the expiration time, rather than returning the value unconditionally. The method also interprets the cached value in terms of the current invalidation time as set by the invalidate() method. If a value is present, but its creation time is older than the current invalidation time, the NO_VALUE token is returned. Passing the flag ignore_expiration=true bypasses the invalidation time check. New in version 0.3.0: Support for the CacheRegion.invalidate() method. Parameters key Key to be retrieved. While it s typical for a key to be a string, it is ultimately passed directly down to the cache backend, before being optionally processed by the key_mangler function, so can be of any type recognized by the backend or by the key_mangler function, if present. expiration_time Optional expiration time value which will supersede that configured on the CacheRegion itself. New in version ignore_expiration if True, the value is returned from the cache if present, regardless of configured expiration times or whether or not invalidate() was called. New in version CacheRegion.get_or_create(key, creator, expiration_time=none, should_cache_fn=none) Return a cached value based on the given key. If the value does not exist or is considered to be expired based on its creation time, the given creation function may or may not be used to recreate the value and persist the newly generated value in the cache. Whether or not the function is used depends on if the dogpile lock can be acquired or not. If it can t, it means a different thread or process is already running a creation function for this key against the cache. When the dogpile lock cannot be acquired, the method will block if no previous value is available, until the lock is released and a new value available. If a previous value is available, that value is returned immediately without blocking. 10 Chapter 2. Usage Guide

15 If the invalidate() method has been called, and the retrieved value s timestamp is older than the invalidation timestamp, the value is unconditionally prevented from being returned. The method will attempt to acquire the dogpile lock to generate a new value, or will wait until the lock is released to return the new value. Changed in version 0.3.0: The value is unconditionally regenerated if the creation time is older than the last call to invalidate(). See also: Parameters key Key to be retrieved. While it s typical for a key to be a string, it is ultimately passed directly down to the cache backend, before being optionally processed by the key_mangler function, so can be of any type recognized by the backend or by the key_mangler function, if present. creator function which creates a new value. expiration_time optional expiration time which will overide the expiration time already configured on this CacheRegion if not None. To set no expiration, use the value -1. should_cache_fn optional callable function which will receive the value returned by the creator, and will then return True or False, indicating if the value should actually be cached or not. If it returns False, the value is still returned, but isn t cached. E.g.: def dont_cache_none(value): return value is not None value = region.get_or_create("some key", create_value, should_cache_fn=dont_cache_none) Above, the function returns the value of create_value() if the cache is invalid, however if the return value is None, it won t be cached. New in version CacheRegion.cache_on_arguments() - applies get_or_create() to any function using a decorator. CacheRegion.get_or_create_multi() - multiple key/value version CacheRegion.set(key, value) Place a new value in the cache under the given key. CacheRegion.delete(key) Remove a value from the cache. This operation is idempotent (can be called multiple times, or on a non-existent key, safely) CacheRegion.cache_on_arguments(namespace=None, expiration_time=none, should_cache_fn=none, to_str=<type str >, function_key_generator=none) A function decorator that will cache the return value of the function using a key derived from the function itself and its arguments. The decorator internally makes use of the CacheRegion.get_or_create() method to access the cache and conditionally call the function. See that method for additional behavioral details. E.g.: 2.4. Using a Region 11

16 @someregion.cache_on_arguments() def generate_something(x, y): return somedatabase.query(x, y) The decorated function can then be called normally, where data will be pulled from the cache region unless a new value is needed: result = generate_something(5, 6) The function is also given an attribute invalidate(), which provides for invalidation of the value. Pass to invalidate() the same arguments you d pass to the function itself to represent a particular value: generate_something.invalidate(5, 6) Another attribute set() is added to provide extra caching possibilities relative to the function. This is a convenience method for CacheRegion.set() which will store a given value directly without calling the decorated function. The value to be cached is passed as the first argument, and the arguments which would normally be passed to the function should follow: generate_something.set(3, 5, 6) The above example is equivalent to calling generate_something(5, 6), if the function were to produce the value 3 as the value to be cached. New in version 0.4.1: Added set() method to decorated function. Similar to set() is refresh(). This attribute will invoke the decorated function and populate a new value into the cache with the new value, as well as returning that value: newvalue = generate_something.refresh(5, 6) New in version 0.5.0: Added refresh() method to decorated function. original() on other hand will invoke the decorated function without any caching: newvalue = generate_something.original(5, 6) New in version 0.6.0: Added original() method to decorated function. Lastly, the get() method returns either the value cached for the given key, or the token NO_VALUE if no such key exists: value = generate_something.get(5, 6) New in version 0.5.3: Added get() method to decorated function. The default key generation will use the name of the function, the module name for the function, the arguments passed, as well as an optional namespace parameter in order to generate a cache key. Given a function one inside the module def one(a, b): return a + b Above, calling one(3, 4) will produce a cache key as follows: myapp.tools:one foo Chapter 2. Usage Guide

17 The key generator will ignore an initial argument of self or cls, making the decorator suitable (with caveats) for use with instance or class methods. Given the example: class def one(self, a, b): return a + b The cache key above for MyClass().one(3, 4) will again produce the same cache key of myapp. tools:one foo the name self is skipped. The namespace parameter is optional, and is used normally to disambiguate two functions of the same name within the same module, as can occur when decorating instance or class methods as below: class def somemethod(self, x, y): "" class def somemethod(self, x, y): "" Above, the namespace parameter disambiguates between somemethod on MyClass and MyOtherClass. Python class declaration mechanics otherwise prevent the decorator from having awareness of the MyClass and MyOtherClass names, as the function is received by the decorator before it becomes an instance method. The function key generation can be entirely replaced on a per-region basis using the function_key_generator argument present on make_region() and CacheRegion. If defaults to function_key_generator(). Parameters namespace optional string argument which will be established as part of the cache key. This may be needed to disambiguate functions of the same name within the same source file, such as those associated with classes - note that the decorator itself can t see the parent class on a function as the class is being declared. expiration_time if not None, will override the normal expiration time. May be specified as a callable, taking no arguments, that returns a value to be used as the expiration_time. This callable will be called whenever the decorated function itself is called, in caching or retrieving. Thus, this can be used to determine a dynamic expiration time for the cached function result. Example use cases include cache the result until the end of the day, week or time period and cache until a certain date or time passes. Changed in version 0.5.0: expiration_time may be passed as a callable to CacheRegion.cache_on_arguments(). should_cache_fn passed to CacheRegion.get_or_create(). New in version to_str callable, will be called on each function argument in order to convert to a string. Defaults to str(). If the function accepts non-ascii unicode arguments on Python 2.x, the unicode() builtin can be substituted, but note this will produce unicode cache keys which may require key mangling before reaching the cache. New in version Using a Region 13

18 See also: function_key_generator a function that will produce a cache key. This function will supersede the one configured on the CacheRegion itself. New in version CacheRegion.cache_multi_on_arguments() CacheRegion.get_or_create() 2.5 Creating Backends Backends are located using the setuptools entrypoint system. To make life easier for writers of ad-hoc backends, a helper function is included which registers any backend in the same way as if it were part of the existing sys.path. For example, to create a backend called DictionaryBackend, we subclass CacheBackend: from dogpile.cache.api import CacheBackend, NO_VALUE class DictionaryBackend(CacheBackend): def init (self, arguments): self.cache = {} def get(self, key): return self.cache.get(key, NO_VALUE) def set(self, key, value): self.cache[key] = value def delete(self, key): self.cache.pop(key) Then make sure the class is available underneath the entrypoint dogpile.cache. If we did this in a setup.py file, it would be in setup() as: entry_points=""" [dogpile.cache] dictionary = mypackage.mybackend:dictionarybackend """ Alternatively, if we want to register the plugin in the same process space without bothering to install anything, we can use register_backend: from dogpile.cache import register_backend register_backend("dictionary", "mypackage.mybackend", "DictionaryBackend") Our new backend would be usable in a region like this: from dogpile.cache import make_region region = make_region("myregion") region.configure("dictionary") data = region.set("somekey", "somevalue") 14 Chapter 2. Usage Guide

19 The values we receive for the backend here are instances of CachedValue. This is a tuple subclass of length two, of the form: (payload, metadata) Where payload is the thing being cached, and metadata is information we store in the cache - a dictionary which currently has just the creation time and a version identifier as key/values. If the cache backend requires serialization, pickle or similar can be used on the tuple - the metadata portion will always be a small and easily serializable Python structure. 2.6 Changing Backend Behavior The ProxyBackend is a decorator class provided to easily augment existing backend behavior without having to extend the original class. Using a decorator class is also adventageous as it allows us to share the altered behavior between different backends. Proxies are added to the CacheRegion object using the CacheRegion.configure() method. Only the overridden methods need to be specified and the real backend can be accessed with the self.proxied object from inside the ProxyBackend. For example, a simple class to log all calls to.set() would look like this: from dogpile.cache.proxy import ProxyBackend import logging log = logging.getlogger( name ) class LoggingProxy(ProxyBackend): def set(self, key, value): log.debug('setting Cache Key: %s' % key) self.proxied.set(key, value) ProxyBackend can be be configured to optionally take arguments (as long as the ProxyBackend. init () method is called properly, either directly or via super(). In the example below, the RetryDeleteProxy class accepts a retry_count parameter on initialization. In the event of an exception on delete(), it will retry this many times before returning: from dogpile.cache.proxy import ProxyBackend class RetryDeleteProxy(ProxyBackend): def init (self, retry_count=5): super(retrydeleteproxy, self). init () self.retry_count = retry_count def delete(self, key): retries = self.retry_count while retries > 0: retries -= 1 try: self.proxied.delete(key) return except: pass 2.6. Changing Backend Behavior 15

20 The wrap parameter of the CacheRegion.configure() accepts a list which can contain any combination of instantiated proxy objects as well as uninstantiated proxy classes. Putting the two examples above together would look like this: from dogpile.cache import make_region retry_proxy = RetryDeleteProxy(5) region = make_region().configure( 'dogpile.cache.pylibmc', expiration_time = 3600, arguments = { 'url':[" "], }, wrap = [ LoggingProxy, retry_proxy ] ) In the above example, the LoggingProxy object would be instantated by the CacheRegion and applied to wrap requests on behalf of the retry_proxy instance; that proxy in turn wraps requests on behalf of the original dogpile.cache.pylibmc backend. New in version 0.4.4: Added support for the ProxyBackend class. 16 Chapter 2. Usage Guide

21 CHAPTER 3 Recipes 3.1 Invalidating a group of related keys This recipe presents a way to track the cache keys related to a particular region, for the purposes of invalidating a series of keys that relate to a particular id. Three cached functions, user_fn_one(), user_fn_two(), user_fn_three() each perform a different function based on a user_id integer value. The region applied to cache them uses a custom key generator which tracks each cache key generated, pulling out the integer id and replacing with a template. When all three functions have been called, the key generator is now aware of these three keys: user_fn_one_%d, user_fn_two_%d, and user_fn_three_%d. The invalidate_user_id() function then knows that for a particular user_id, it needs to hit all three of those keys in order to invalidate everything having to do with that id. from dogpile.cache import make_region from itertools import count user_keys = set() def my_key_generator(namespace, fn): fname = fn. name def generate_key(*arg): # generate a key template: # "fname_%d_arg1_arg2_arg3..." key_template = fname + "_" + \ "%d" + \ "_".join(str(s) for s in arg[1:]) # store key template user_keys.add(key_template) # return cache key user_id = arg[0] return key_template % user_id (continues on next page) 17

22 return generate_key (continued from previous page) def invalidate_user_id(region, user_id): for key in user_keys: region.delete(key % user_id) region = make_region( function_key_generator=my_key_generator ).configure( "dogpile.cache.memory" ) counter = def user_fn_one(user_id): return "user fn one: %d, %d" % (next(counter), def user_fn_two(user_id): return "user fn two: %d, %d" % (next(counter), def user_fn_three(user_id): return "user fn three: %d, %d" % (next(counter), user_id) print user_fn_one(5) print user_fn_two(5) print user_fn_three(7) print user_fn_two(7) invalidate_user_id(region, 5) print "invalidated:" print user_fn_one(5) print user_fn_two(5) print user_fn_three(7) print user_fn_two(7) 3.2 Asynchronous Data Updates with ORM Events This recipe presents one technique of optimistically pushing new data into the cache when an update is sent to a database. Using SQLAlchemy for database querying, suppose a simple cache-decorated function returns the results of a database def get_some_data(argument): # query database to get data data = Session().query(DBClass).filter(DBClass.argument == argument).all() return data We would like this particular function to be re-queried when the data has changed. We could call get_some_data. invalidate(argument, hard=false) at the point at which the data changes, however this only leads to the invalidation of the old value; a new value is not generated until the next call, and also means at least one client has 18 Chapter 3. Recipes

23 to block while the new value is generated. We could also call get_some_data.refresh(argument), which would perform the data refresh at that moment, but then the writer is delayed by the re-query. A third variant is to instead offload the work of refreshing for this query into a background thread or process. This can be acheived using a system such as the CacheRegion.async_creation_runner. However, an expedient approach for smaller use cases is to link cache refresh operations to the ORM session s commit, as below: from sqlalchemy import event from sqlalchemy.orm import Session def cache_refresh(session, refresher, *args, **kwargs): """ Refresh the functions cache data in a new thread. Starts refreshing only after the session was committed so all database data is available. """ assert isinstance(session, Session), \ "Need a session, not a sessionmaker or "after_commit") def do_refresh(session): t = Thread(target=refresher, args=args, kwargs=kwargs) t.daemon = True t.start() Within a sequence of data persistence, cache_refresh can be called given a particular SQLAlchemy Session and a callable to do the work: def add_new_data(session, argument): # add some data session.add(something_new(argument)) # add a hook to refresh after the Session is committed. cache_refresh(session, get_some_data.refresh, argument) Note that the event to refresh the data is associated with the Session being used for persistence; however, the actual refresh operation is called with a different Session, typically one that is local to the refresh operation, either through a thread-local registry or via direct instantiation. 3.3 Prefixing all keys in Redis If you use a redis instance as backend that contains other keys besides the ones set by dogpile.cache, it is a good idea to uniquely prefix all dogpile.cache keys, to avoid potential collisions with keys set by your own code. This can easily be done using a key mangler function: from dogpile.cache import make_region region = make_region( key_mangler=lambda key: "myapp:dogpile:" + key ) 3.4 Encoding/Decoding data into another format 3.3. Prefixing all keys in Redis 19

24 A Note on Data Encoding Under the hood, dogpile.cache wraps cached data in an instance of dogpile.cache.api.cachedvalue and then pickles that data for storage along with some bookkeeping metadata. If you implement a ProxyBackend to encode/decode data, that transformation will happen on the pre-pickled data- dogpile does not store the data raw and will still pass a pickled payload to the backend. This behavior can negate the hopeful improvements of some encoding schemes. Since dogpile is managing cached data, you may be concerned with the size of your payloads. A possible method of helping minimize payloads is to use a ProxyBackend to recode the data on-the-fly or otherwise transform data as it enters or leaves persistent storage. In the example below, we define 2 classes to implement msgpack encoding. Msgpack ( is a serialization format that works exceptionally well with json-like data and can serialize nested dicts into a much smaller payload than Python s own pickle. _EncodedProxy is our base class for building data encoders, and inherits from dogpile s own ProxyBackend. You could just use one class. This class passes 4 of the main key/value functions into a configurable decoder and encoder. The MsgpackProxy class simply inherits from _EncodedProxy and implements the necessary value_decode and value_encode functions. Encoded ProxyBackend Example: from dogpile.cache.proxy import ProxyBackend import msgpack class _EncodedProxy(ProxyBackend): """base class for building value-mangling proxies""" def value_decode(self, value): raise NotImplementedError("override me") def value_encode(self, value): raise NotImplementedError("override me") def set(self, k, v): v = self.value_encode(v) self.proxied.set(k, v) def get(self, key): v = self.proxied.get(key) return self.value_decode(v) def set_multi(self, mapping): """encode to a new dict to preserve unencoded values in-place when called by `get_or_create_multi` """ mapping_set = {} for (k, v) in mapping.iteritems(): mapping_set[k] = self.value_encode(v) return self.proxied.set_multi(mapping_set) def get_multi(self, keys): results = self.proxied.get_multi(keys) translated = [] for record in results: try: translated.append(self.value_decode(record)) except Exception as e: (continues on next page) 20 Chapter 3. Recipes

25 raise return translated (continued from previous page) class MsgpackProxy(_EncodedProxy): """custom decode/encode for value mangling""" def value_decode(self, v): if not v or v is NO_VALUE: return NO_VALUE # you probably want to specify a custom decoder via `object_hook` v = msgpack.unpackb(payload, encoding="utf-8") return CachedValue(*v) def value_encode(self, v): # you probably want to specify a custom encoder via `default` v = msgpack.packb(payload, use_bin_type=true) return v # extend our region configuration from above with a 'wrap' region = make_region().configure( 'dogpile.cache.pylibmc', expiration_time = 3600, arguments = { 'url': [" "], }, wrap = [MsgpackProxy, ] ) 3.4. Encoding/Decoding data into another format 21

26 22 Chapter 3. Recipes

27 CHAPTER 4 dogpile Core dogpile provides a locking interface around a value creation and value retrieval pair of functions. Changed in version 0.6.0: The dogpile package encapsulates the functionality that was previously provided by the separate dogpile.core package. The primary interface is the Lock object, which provides for the invocation of the creation function by only one thread and/or process at a time, deferring all other threads/processes to the value retrieval function until the single creation thread is completed. 4.1 Do I Need to Learn the dogpile Core API Directly? It s anticipated that most users of dogpile will be using it indirectly via the dogpile.cache caching front-end. If you fall into this category, then the short answer is no. Using the core dogpile APIs described here directly implies you re building your own resource-usage system outside, or in addition to, the one dogpile.cache provides. 4.2 Rudimentary Usage The primary API dogpile provides is the Lock object. This object allows for functions that provide mutexing, value creation, as well as value retrieval. An example usage is as follows: from dogpile import Lock, NeedRegenerationException import threading import time # store a reference to a "resource", some # object that is expensive to create. the_resource = [None] (continues on next page) 23

28 (continued from previous page) def some_creation_function(): # call a value creation function value = create_some_resource() # get creationtime using time.time() creationtime = time.time() # keep track of the value and creation time in the "cache" the_resource[0] = tup = (value, creationtime) # return the tuple of (value, creationtime) return tup def retrieve_resource(): # function that retrieves the resource and # creation time. # if no resource, then raise NeedRegenerationException if the_resource[0] is None: raise NeedRegenerationException() # else return the tuple of (value, creationtime) return the_resource[0] # a mutex, which needs here to be shared across all invocations # of this particular creation function mutex = threading.lock() with Lock(mutex, some_creation_function, retrieve_resource, 3600) as value: # some function that uses # the resource. Won't reach # here until some_creation_function() # has completed at least once. value.do_something() Above, some_creation_function() will be called when Lock is first invoked as a context manager. The value returned by this function is then passed into the with block, where it can be used by application code. Concurrent threads which call Lock during this initial period will be blocked until some_creation_function() completes. Once the creation function has completed successfully the first time, new calls to Lock will call retrieve_resource() in order to get the current cached value as well as its creation time; if the creation time is older than the current time minus an expiration time of 3600, then some_creation_function() will be called again, but only by one thread/process, using the given mutex object as a source of synchronization. Concurrent threads/processes which call Lock during this period will fall through, and not be blocked; instead, the stale value just returned by retrieve_resource() will continue to be returned until the creation function has finished. The Lock API is designed to work with simple cache backends like Memcached. It addresses such issues as: Values can disappear from the cache at any time, before our expiration time is reached. The NeedRegenerationException class is used to alert the Lock object that a value needs regeneration ahead of the usual expiration time. There s no function in a Memcached-like system to check for a key without actually retrieving it. The usage of the retrieve_resource() function allows that we check for an existing key and also return the existing value, if any, at the same time, without the need for two separate round trips. The creation function used by Lock is expected to store the newly created value in the cache, as well as to 24 Chapter 4. dogpile Core

29 return it. This is also more efficient than using two separate round trips to separately store, and re-retrieve, the object. 4.3 Example: Using dogpile directly for Caching The following example approximates Beaker s cache decoration function, to decorate any function and store the value in Memcached. Note that normally, we d just use dogpile.cache here, however for the purposes of example, we ll illustrate how the Lock object is used directly. We create a Python decorator function called cached() which will provide caching for the output of a single function. It s given the key which we d like to use in Memcached, and internally it makes usage of Lock, along with a thread based mutex (we ll see a distributed mutex in the next section): import pylibmc import threading import time from dogpile import Lock, NeedRegenerationException mc_pool = pylibmc.threadmappedpool(pylibmc.client("localhost")) def cached(key, expiration_time): """A decorator that will cache the return value of a function in memcached given a key.""" mutex = threading.lock() def get_value(): with mc_pool.reserve() as mc: value_plus_time = mc.get(key) if value_plus_time is None: raise NeedRegenerationException() # return a tuple (value, createdtime) return value_plus_time def decorate(fn): def gen_cached(): value = fn() with mc_pool.reserve() as mc: # create a tuple (value, createdtime) value_plus_time = (value, time.time()) mc.put(key, value_plus_time) return value_plus_time def invoke(): with Lock(mutex, gen_cached, get_value, expiration_time) as value: return value return invoke return decorate Using the above, we can decorate any function key", 3600) def generate_my_expensive_value(): return slow_database.lookup("stuff") 4.3. Example: Using dogpile directly for Caching 25

30 The Lock object will ensure that only one thread at a time performs slow_database.lookup(), and only every 3600 seconds, unless Memcached has removed the value, in which case it will be called again as needed. In particular, dogpile.core s system allows us to call the memcached get() function at most once per access, instead of Beaker s system which calls it twice, and doesn t make us call get() when we just created the value. For the mutex object, we keep a threading.lock object that s local to the decorated function, rather than using a global lock. This localizes the in-process locking to be local to this one decorated function. In the next section, we ll see the usage of a cross-process lock that accomplishes this differently. 4.4 Using a File or Distributed Lock with Dogpile The examples thus far use a threading.lock() object for synchronization. If our application uses multiple processes, we will want to coordinate creation operations not just on threads, but on some mutex that other processes can access. In this example we ll use a file-based lock as provided by the lockfile package, which uses a unix-symlink concept to provide a filesystem-level lock (which also has been made threadsafe). Another strategy may base itself directly off the Unix os.flock() call, or use an NFS-safe file lock like flufl.lock, and still another approach is to lock against a cache server, using a recipe such as that described at Using Memcached as a Distributed Locking Service. What all of these locking schemes have in common is that unlike the Python threading.lock object, they all need access to an actual key which acts as the symbol that all processes will coordinate upon. So here, we will also need to create the mutex which we pass to Lock using the key argument: import lockfile import os from hashlib import sha1 #... other imports and setup from the previous example def cached(key, expiration_time): """A decorator that will cache the return value of a function in memcached given a key.""" lock_path = os.path.join("/tmp", "%s.lock" % sha1(key).hexdigest()) #... get_value() from the previous example goes here def decorate(fn): #... gen_cached() from the previous example goes here def invoke(): # create an ad-hoc FileLock mutex = lockfile.filelock(lock_path) with Lock(mutex, gen_cached, get_value, expiration_time) as value: return value return invoke return decorate For a given key some_key, we generate a hex digest of the key, then use lockfile.filelock() to create a lock against the file /tmp/53def077a4264bd3183d4eb21b1f56f883e1b572.lock. Any number of Lock objects in various processes will now coordinate with each other, using this common filename as the baton against which creation of a new value proceeds. 26 Chapter 4. dogpile Core

31 Unlike when we used threading.lock, the file lock is ultimately locking on a file, so multiple instances of FileLock() will all coordinate on that same file - it s often the case that file locks that rely upon flock() require non-threaded usage, so a unique filesystem lock per thread is often a good idea in any case Using a File or Distributed Lock with Dogpile 27

32 28 Chapter 4. dogpile Core

33 CHAPTER 5 API 5.1 Region class dogpile.cache.region.cacheregion(name=none, function_key_generator=<function function_key_generator>, function_multi_key_generator=<function function_multi_key_generator>, key_mangler=none, async_creation_runner=none) A front end to a particular cache backend. Parameters name Optional, a string name for the region. This isn t used internally but can be accessed via the.name parameter, helpful for configuring a region from a config file. function_key_generator Optional. A function that will produce a cache key given a data creation function and arguments, when using the CacheRegion. cache_on_arguments() method. The structure of this function should be two levels: given the data creation function, return a new function that generates the key based on the given arguments. Such as: def my_key_generator(namespace, fn, **kw): fname = fn. name def generate_key(*arg): return namespace + "_" + fname + "_".join(str(s) for s in arg) return generate_key region = make_region( function_key_generator = my_key_generator ).configure( "dogpile.cache.dbm", expiration_time=300, (continues on next page) 29

34 ) arguments={ "filename":"file.dbm" } (continued from previous page) The namespace is that passed to CacheRegion.cache_on_arguments(). It s not consulted outside this function, so in fact can be of any form. For example, it can be passed as a tuple, used to specify arguments to pluck from **kw: def my_key_generator(namespace, fn): def generate_key(*arg, **kw): return ":".join( [kw[k] for k in namespace] + [str(x) for x in arg] ) return generate_key Where the decorator might be used 'y')) def my_function(a, b, **kw): return my_data() See also: function_key_generator() - default key generator kwarg_function_key_generator() - optional gen that also uses keyword arguments function_multi_key_generator Optional. Similar to function_key_generator parameter, but it s used in CacheRegion. cache_multi_on_arguments(). Generated function should return list of keys. For example: def my_multi_key_generator(namespace, fn, **kw): namespace = fn. name + (namespace or '') def generate_keys(*args): return [namespace + ':' + str(a) for a in args] return generate_keys key_mangler Function which will be used on all incoming keys before passing to the backend. Defaults to None, in which case the key mangling function recommended by the cache backend will be used. A typical mangler is the SHA1 mangler found at sha1_mangle_key() which coerces keys into a SHA1 hash, so that the string length is fixed. To disable all key mangling, set to False. Another typical mangler is the built-in Python function str, which can be used to convert non-string or Unicode keys to bytestrings, which is needed when using a backend such as bsddb or dbm under Python 2.x in conjunction with Unicode keys. async_creation_runner A callable that, when specified, will be passed to and called by dogpile.lock when there is a stale value present in the cache. It will be passed the mutex and is responsible releasing that mutex when finished. This can be used to defer the computation of expensive creator functions to later points in the future by way of, for example, a background thread, a long-running queue, or a task manager system like Celery. 30 Chapter 5. API