django-redis-cache Documentation

Transcription

1 django-redis-cache Documentation Release Sean Bleier Nov 15, 2018

2

3 Contents 1 Intro and Quick Start Intro Quick Start API Usage Standard Django Cache API Cache Methods Provided by django-redis-cache Advanced Configuration Example Setting Pluggable Backends Location Schemes Database Number Password Master/Slave Setup Pluggable Parser Classes Pickle Version Socket Timeout and Socket Create Timeout Connection Pool Pluggable Serializers Pluggable Compressors Indices and tables 15 i

4 ii

5 Contents: Contents 1

6 2 Contents

7 CHAPTER 1 Intro and Quick Start 1.1 Intro django-redis-cache is a cache backend for the Django webframework. It uses the redis server, which is a in-memory key-value data structure server. Similar to the great Memcached in performance, it has several features that makes it more appealing. Multiple data structures types, e.g. string, list, set, sorted sets, and hashes. Atomic pipelines: guaranteed that multiple commands will run sequentially and uninterrupted. Pub/Sub: subscribe to a channel and listen for messages from other processes. Can back data to disk, which can keep a cache warm even if the process is killed. Lua scripting Clustering (as of 3.0) Many more. Many of these features are irrelevant to caching, but can be used by other areas of a web stack and therefore offer a compelling case to simplify your infrastructure. 1.2 Quick Start Recommended: redis >= 2.4 redis-py >= python >= Install redis. You can use install_redis.sh to install a local copy of redis. Start the server by running./src/redis-server 3

8 2. Run pip install django-redis-cache. 3. Modify your Django settings to use redis_cache. 'BACKEND': 'redis_cache.rediscache', 'LOCATION': 'localhost:6379', Warning: By default, django-redis-cache set keys in the database 1 of Redis. By default, a session with redis-cli start on database 0. Switch to database 1 with SELECT 1. 4 Chapter 1. Intro and Quick Start

9 CHAPTER 2 API Usage 2.1 Standard Django Cache API get(self, key[, default=none]): Retrieves a value from the cache. Parameters key Location of the value default Value to return if key does not exist in cache. Return type Value that was cached. add(self, key, value[, timeout=default_timeout]): Add a value to the cache, failing if the key already exists. Parameters key Location of the value value Value to cache timeout (Number of seconds or DEFAULT_TIMEOUT) Number of seconds to hold value in cache. Return type True if object was added and False if it already exists. set(self, key, value, timeout=default_timeout): Sets a value to the cache, regardless of whether it exists. If timeout == None, then cache is set indefinitely. DEFAULT_TIMEOUT. Parameters key Location of the value value Value to cache Otherwise, timeout defaults to the defined 5

10 timeout (Number of seconds or DEFAULT_TIMEOUT) Number of seconds to hold value in cache. delete(self, key): Removes a key from the cache Parameters key Location of the value delete_many(self, keys[, version=none]): Removes multiple keys at once. Parameters key Location of the value version Version of keys clear(self[, version=none]): Flushes the cache. If version is provided, all keys under the version number will be deleted. Otherwise, all keys will be flushed. Parameters version Version of keys get_many(self, keys[, version=none]): Retrieves many keys at once. Parameters keys an iterable of keys to retrieve. Return type Dict of keys mapping to their values. set_many(self, data[, timeout=none, version=none]): Set many values in the cache at once from a dict of key/value pairs. This is much more efficient than calling set() multiple times and is atomic. Parameters data dict of key/value pairs to cache. timeout (Number of seconds or None) Number of seconds to hold value in cache. incr(self, key[, delta=1]): Add delta to value in the cache. If the key does not exist, raise a ValueError exception. Parameters key Location of the value delta (Integer) Integer used to increment a value. incr_version(self, key[, delta=1, version=none]): Adds delta to the cache version for the supplied key. Returns the new version. Parameters key Location of the value delta (Integer) Integer used to increment a value. version (Integer or None) Version of key 6 Chapter 2. API Usage

11 2.2 Cache Methods Provided by django-redis-cache has_key(self, key): Returns True if the key is in the cache and has not expired. Parameters key Location of the value Return type bool ttl(self, key): Returns the time-to-live of a key. If the key is not volatile, i.e. it has not set an expiration, then the value returned is None. Otherwise, the value is the number of seconds remaining. If the key does not exist, 0 is returned. Parameters key Location of the value Return type Integer or None delete_pattern(pattern[, version=none]): Deletes keys matching the glob-style pattern provided. Parameters pattern Glob-style pattern used to select keys to delete. version Version of the keys get_or_set(self, key, func[, timeout=none]): Retrieves a key value from the cache and sets the value if it does not exist. Parameters key Location of the value func Callable used to set the value if key does not exist. timeout (Number of seconds or None) Number of seconds to hold value in cache. reinsert_keys(self): Helper function to reinsert keys using a different pickle protocol version. persist(self, key): Removes the timeout on a key. Equivalent to setting a timeout of None in a set command. :param key: Location of the value :rtype: bool expire(self, key, timeout): Set the expire time on a key Parameters key Location of the value Return type bool 2.2. Cache Methods Provided by django-redis-cache 7

12 8 Chapter 2. API Usage

13 CHAPTER 3 Advanced Configuration 3.1 Example Setting 'BACKEND': 'redis_cache.rediscache', 'LOCATION': ' :6379', 'DB': 1, 'PASSWORD': 'yadayada', 'PARSER_CLASS': 'redis.connection.hiredisparser', 'CONNECTION_POOL_CLASS': 'redis.blockingconnectionpool', 'PICKLE_VERSION': -1, 3.2 Pluggable Backends django-redis-cache comes with a couple pluggable backends, one for a unified keyspace and one for a sharded keyspace. The former can be in the form of a single redis server or several redis servers setup in a primary/secondary configuration. The primary is used for writing and secondaries are replicated versions of the primary for read-access. Default Backend: redis_cache.rediscache # Unified keyspace 'BACKEND': 'redis_cache.rediscache', (continues on next page) 9

14 (continued from previous page) # Sharded keyspace 'BACKEND': 'redis_cache.shardedrediscache', 3.3 Location Schemes The LOCATION contains the information for the redis server s location, which can be the address/port or the server path to the unix domain socket. The location can be a single string or a list of strings. Several schemes for defining the location can be used. Here is a list of example schemes: :6379 /path/to/socket redis://[:password]@localhost:6379/0 rediss://[:password]@localhost:6379/0 unix://[:password]@/path/to/socket.sock?db=0 3.4 Database Number The DB option will allow key/values to exist in a different keyspace. The DB value can either be defined in the OPTIONS or in the LOCATION scheme. Default DB: 1 'DB': 1, Password If the redis server is password protected, you can specify the PASSWORD option. 'PASSWORD': 'yadayada', (continues on next page) 10 Chapter 3. Advanced Configuration

15 (continued from previous page) 3.6 Master/Slave Setup It s possible to have multiple redis servers in a master/slave or primary/secondary configuration. Here we have the primary server acting as a read/write server and secondary servers as read-only. 'LOCATION': [ ' :6379', # Primary ' :6380', # Secondary ' :6381', # Secondary ], 'PASSWORD': 'yadayada', 'MASTER_CACHE': ' :6379', 3.7 Pluggable Parser Classes redis-py comes with two parsers: HiredisParser and PythonParser. The former uses the hiredis library to parse responses from the redis server, while the latter uses Python. Hiredis is a library that uses C, so it is much faster than the python parser, but requires installing the library separately. Default Parser: redis.connection.pythonparser The default parser is the Python parser because there is no other dependency, but I would recommend using hiredis: pip install hiredis 'PARSER_CLASS': 'redis.connection.hiredisparser', 3.6. Master/Slave Setup 11

16 3.8 Pickle Version When using the pickle serializer, you can use PICKLE_VERSION to specify the protocol version of pickle you want to use to serialize your python objects. Default Pickle Version: -1 The default pickle protocol is -1, which is the highest and latest version. This value should be pinned to a specific protocol number, since -1 means different things between versions of Python. 'PICKLE_VERSION': 2, 3.9 Socket Timeout and Socket Create Timeout When working with a TCP connection, it may be beneficial to set the SOCKET_TIMEOUT and SOCKET_CONNECT_TIMEOUT options to prevent your app from blocking indefinitely. If provided, the socket will time out when the established connection exceeds SOCKET_TIMEOUT seconds. Similarly, the socket will time out if it takes more than SOCKET_CONNECT_TIMEOUT seconds to establish. Default Socket Timeout: None Default Socket Connect Timeout: None CACHES={ 'SOCKET_TIMEOUT': 5, 'SOCKET_CONNECT_TIMEOUT': 5, 3.10 Connection Pool There is an associated overhead when creating connections to a redis server. Therefore, it s beneficial to create a pool of connections that the cache can reuse to send or retrieve data from the redis server. CONNECTION_POOL_CLASS can be used to specify a class to use for the connection pool. In addition, you can provide custom keyword arguments using the CONNECTION_POOL_CLASS_KWARGS option that will be passed into the class when it s initialized. Default Connection Pool: redis.connectionpool 12 Chapter 3. Advanced Configuration

17 'CONNECTION_POOL_CLASS': 'redis.blockingconnectionpool', 'CONNECTION_POOL_CLASS_KWARGS': { 'max_connections': 50, 'timeout': 20, 3.11 Pluggable Serializers You can use SERIALIZER_CLASS to specify a class that will serialize/deserialize data. In addition, you can provide custom keyword arguments using the SERIALIZER_CLASS_KWARGS option that will be passed into the class when it s initialized. The default serializer in django-redis-cache is the pickle serializer. It can serialize most python objects, but is slow and not always safe. Also included are serializer using json, msgpack, and yaml. Not all serializers can handle Python objects, so they are limited to primitive data types. Default Serializer: redis_cache.serializers.pickleserializer 'SERIALIZER_CLASS': 'redis_cache.serializers.pickleserializer', 'SERIALIZER_CLASS_KWARGS': { 'pickle_version': Pluggable Compressors You can use COMPRESSOR_CLASS to specify a class that will compress/decompress data. COMPRESSOR_CLASS_KWARGS option to initialize the compressor class. Use the The default compressor is NoopCompressor which does not compress your data. However, if you want to compress your data, you can use one of the included compressor classes: Default Compressor: redis_cache.compressors.noopcompressor # zlib compressor (continues on next page) Pluggable Serializers 13

18 (continued from previous page) 'COMPRESSOR_CLASS': 'redis_cache.compressors.zlibcompressor', 'COMPRESSOR_CLASS_KWARGS': { 'level': 5, # 0-9; 0 - no compression; 1 - fastest, biggest; 9 - slowest, smallest # bzip2 compressor 'COMPRESSOR_CLASS': 'redis_cache.compressors.bzip2compressor', 'COMPRESSOR_CLASS_KWARGS': { 'compresslevel': 5, # 1-9; 1 - fastest, biggest; 9 - slowest, smallest 14 Chapter 3. Advanced Configuration

19 CHAPTER 4 Indices and tables genindex modindex search 15