Introduction

Have you ever run the same expensive computation multiple times and wondered if Python could "remember" the result? That's the core idea of memoization—store results of function calls so repeated calls with the same inputs return instantly. Python's standard library provides robust tools for this, primarily in the functools module.

In this guide you'll learn:

• What memoization is and when to use it
• How to use functools.lru_cache and functools.cache
• Patterns for caching functions that accept complex or mutable inputs
• How memoization interacts with dataclasses, multiprocessing, and async functions
• Best practices, pitfalls, and advanced tips

Prerequisites: intermediate Python (functions, decorators, basic concurrency). Examples assume Python 3.8+ (some features require 3.9+ for functools.cache; frozen dataclasses require no special version).

Prerequisites and Concepts

Before we dive into examples, let's clarify the key concepts.

• Memoization: Caching function results keyed by the function's input arguments.
• Hashability: Standard lru_cache requires function arguments to be hashable because it uses a dictionary-like mapping.
• LRU (Least Recently Used): Strategy used by lru_cache to limit memory—evicts the least recently used items when full.
• Thread/Process Safety: Caches are in-process; threads share memory but processes do not.
• Asynchronous functions: async def functions return coroutines—naive use of lru_cache won't await and correctly cache results.

Related tools we will mention:

• dataclasses: Helpful to create small, hashable containers for complex inputs (see "A Practical Guide to Python's dataclasses: Simplifying Class Creation and Data Management").
• multiprocessing: Caches don't automatically share across processes; see "Mastering Python's multiprocessing for Parallel Processing" for deeper context.
• async/await: For I/O-bound tasks, caching coroutine results requires async-aware strategies (see "Exploring Python's async and await: Real-World Applications in I/O-Bound Tasks").

Core Tools in functools

Two functions in functools are essential:

• functools.lru_cache(maxsize=128, typed=False): A decorator implementing an LRU cache.
• functools.cache (Python 3.9+): An unbounded cache (like lru_cache(maxsize=None)).

Common useful attributes/methods:

• wrapped_function.cache_info() returns (hits, misses, maxsize, currsize)
• wrapped_function.cache_clear() clears the cache

Simple Example: Fibonacci (Naive vs Memoized)

Let's start with a classic: the recursive Fibonacci function.

from functools import lru_cache
import time
def fib_naive(n: int) -> int:
    if n < 2:
        return n
    return fib_naive(n - 1) + fib_naive(n - 2)
@lru_cache(maxsize=128)
def fib_memo(n: int) -> int:
    if n < 2:
        return n
    return fib_memo(n - 1) + fib_memo(n - 2)
Timing
start = time.time()
print("Naive:", fib_naive(30))
print("Naive time:", time.time() - start)
start = time.time()
print("Memo:", fib_memo(100))
print("Memo time:", time.time() - start)
print("Cache info:", fib_memo.cache_info())

Explanation line-by-line:

• from functools import lru_cache: import the decorator.
• fib_naive: standard exponential-time recursive Fibonacci.
• @lru_cache(maxsize=128): decorates fib_memo to cache up to 128 unique calls.
• Calls to fib_memo reuse cached values for overlapping subproblems, turning an exponential algorithm into effectively linear time.
• fib_memo.cache_info() shows hits & misses and cache size.

Edge cases:

• maxsize must be large enough to hold unique patterns for your use-case; too small -> frequent evictions.
• For very large n, recursion depth may still be an issue; memoization helps time but not Python recursion limits.

Try it: run both functions and compare timings. You'll see dramatic improvement.

When Arguments Are Not Hashable — Solutions

lru_cache requires arguments to be hashable. What if you have lists, dicts, or complex objects? Several strategies:

1. Convert mutable inputs to an immutable representation (e.g., tuple or frozenset).
2. Use a frozen dataclass to represent structured data as a hashable object.
3. Implement a custom decorator that creates a hashable key.

Strategy A: Convert args to tuples

from functools import lru_cache
def normalize_args(arg):
    # Example for lists/dicts - simplistic; adapt for real needs
    if isinstance(arg, list):
        return tuple(arg)
    if isinstance(arg, dict):
        return tuple(sorted(arg.items()))
    return arg
@lru_cache(maxsize=256)
def expensive_sum(args):
    # assume args is given as a tuple or acceptable hashable representation
    return sum(args)
Use normalized input
data = [1, 2, 3]
result = expensive_sum(tuple(data))  # convert list -> tuple

Explanation:

• Convert lists/dicts into tuples/sorted tuples of items so they become hashable.
• For nested structures, you might need recursive normalization.

Strategy B: Frozen dataclasses (recommended for structured inputs)

Use dataclasses to define a clean, hashable container for inputs.

from dataclasses import dataclass
from functools import lru_cache
@dataclass(frozen=True)
class QueryParams:
    user_id: int
    filters: tuple  # immutable representation
    
@lru_cache(maxsize=512)
def query_costly(params: QueryParams) -> dict:
    # imagine hitting a database or computing analytics
    # params is hashable because dataclass is frozen
    return {"user": params.user_id, "result": sum(params.filters)}

Explanation:

• @dataclass(frozen=True) results in an immutable, hashable object.
• filters is stored as a tuple (converted before creating QueryParams instance), ensuring the entire dataclass is hashable.
• Using dataclasses also improves readability and maintainability — see "A Practical Guide to Python's dataclasses: Simplifying Class Creation and Data Management."

Edge-cases:

• If any field is mutable, the dataclass won't be fully hashable—ensure fields are immutable.

Custom Memoization Decorator for Unhashable Inputs

If you have complex, variable inputs, implement a decorator that builds a hashable key safely. Here's a robust example.

from functools import wraps
import pickle
from typing import Callable, Any
def memoize_via_pickle(func: Callable) -> Callable:
    cache = {}
    @wraps(func)
    def wrapper(args, kwargs):
        # create a byte-key using pickle (works for many Python objects)
        # Note: pickle can be slow and may not be secure for untrusted inputs
        key = pickle.dumps((args, kwargs))
        if key in cache:
            return cache[key]
        result = func(args, *kwargs)
        cache[key] = result
        return result
    def cache_clear():
        cache.clear()
    wrapper.cache_clear = cache_clear
    return wrapper

@memoize_via_pickle
def complex_op(records):
    # expensive operation that accepts lists/dicts
    return sum(item['value'] for item in records)

Explanation:

• pickle.dumps((args, kwargs)) serializes the call into bytes to create a key; works for many Python objects.
• cache stores results keyed by serialized arguments.
• Security & performance: avoid using this for untrusted input (pickle vulnerability); serialization overhead may be significant.

Async Functions and Memoization

Can you use lru_cache on async def functions? Not directly. Applying lru_cache to an async function caches the coroutine object itself, not the awaited result. We need an async-aware cache.

Here's an async memoization decorator that caches results properly and avoids duplicate concurrent executions for the same key:

import asyncio
from functools import wraps
def async_memoize(func):
    cache = {}
    locks = {}
    @wraps(func)
    async def wrapper(args, *kwargs):
        key = (args, tuple(sorted(kwargs.items())))
        if key in cache:
            return cache[key]
        # Ensure only one coroutine computes the result for a given key
        lock = locks.setdefault(key, asyncio.Lock())
        async with lock:
            # Another coroutine may have already filled the cache
            if key in cache:
                return cache[key]
            result = await func(args, *kwargs)
            cache[key] = result
            # Optional: clean up locks to avoid memory leak
            locks.pop(key, None)
            return result
    wrapper.cache_clear = lambda: cache.clear()
    return wrapper

Example usage
import aiohttp
@async_memoize
async def fetch_json(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            return await resp.json()

Explanation:

• cache stores completed results.
• locks ensures only one coroutine fetches a given key at a time (prevents thundering herd).
• key is constructed from args and kwargs; ensure arguments are hashable or normalized.
• This approach is pure-Python and works with asyncio. For production, consider battle-tested async cache libraries.

Edge cases:

• If fetching fails, you may want to decide whether to cache exceptions or reattempt on next call.
• For long-running processes, memory growth from unbounded caches should be guarded or cleaned.

Multiprocessing and Memoization

Important: caches live in a process's memory. With multiprocessing, worker processes have separate memory—so a cache in the parent won't be available in children, and vice versa.

Scenario: You use multiprocessing.Pool to parallelize CPU-bound work; memoization in the worker function isn't shared across workers by default. That means each worker gets its own lru_cache instance.

Options:

• Pre-warm each worker's cache using initializer in Pool.
• Use a shared memory cache (e.g., multiprocessing.Manager().dict()), but be aware of lock overhead and serialization costs.
• For CPU-bound tasks, sometimes using a shared cache via a lightweight server (e.g., Redis) is more scalable.

Example: pre-warm worker caches

from multiprocessing import Pool
from functools import lru_cache
@lru_cache(maxsize=1024)
def heavy_compute(x):
    # expensive CPU-bound computation
    return x  x  # placeholder
def init_worker():
    # dummy call(s) to pre-populate cache if you have predictable patterns
    heavy_compute(1)
    heavy_compute(2)
def worker(x):
    return heavy_compute(x)
if __name__ == "__main__":
    with Pool(initializer=init_worker) as p:
        print(p.map(worker, [1, 2, 3, 4]))

Notes:

• Pre-warming helps only if you know keys in advance.
• multiprocessing overhead may negate some caching benefits for fine-grained tasks.

For more detailed patterns, read "Mastering Python's multiprocessing for Parallel Processing: A Case Study on Performance Improvement".

Measuring and Tuning Cache Performance

Always measure before and after. Use cache_info() with lru_cache:

@lru_cache(maxsize=256)
def compute(x):
    # ...
    return x
compute(1); compute(2); compute(1)
print(compute.cache_info())  # Example output: CacheInfo(hits=1, misses=2, maxsize=256, currsize=2)

Metrics:

• hits: number of times cached result returned
• misses: calls that required computation
• maxsize vs currsize: tuning maxsize balances memory vs reuse

Performance considerations:

• Serialization for complex keys (e.g., pickle) adds overhead — avoid if function is cheap.
• For long-running apps, an unbounded cache (functools.cache) may cause memory leaks — prefer LRU with sensible maxsize.
• Consider TTL (time-based expiration) with third-party libraries if cached results become stale.

Best Practices

• Use @lru_cache for pure functions: those without side effects; caching side-effectful functions can be a bug source.
• Keep cache keys deterministic and based only on function input.
• Use @dataclass(frozen=True) for structured inputs to make them safe and readable.
• Avoid caching for very cheap functions — the overhead may outweigh benefits.
• Use cache_clear() and/or strategy to limit cache growth (LRU or TTL).
• Use thread/process-aware designs when concurrent execution is involved.

Common Pitfalls

• Caching functions with mutable default arguments — these are shared and cause confusion.
• Using lru_cache on methods without accounting for self: bound methods include self (the object) in the key, so identical-looking calls on different instances won't share cache. Consider using @functools.cache on @staticmethod or use caching on functions that accept choices of state explicitly.
• Assuming caches are shared across processes or machines.
• Caching results that depend on external state (e.g., file contents) without invalidation.

Advanced Tips

• Use the typed=True option in lru_cache if you want f(3) and f(3.0) to be cached separately.
• Combine dataclasses and lru_cache for readable, hashable keys:

- When calling, ensure fields in dataclass are immutable (e.g., tuples).

• For async workloads, consider using dedicated async cache libraries (e.g., aiocache) if you need features like TTL, persistence, or eviction policies.
• If caching expensive results across processes/machines, use an external cache (Redis, Memcached) — but weigh serialization costs.
• Profile memory usage of your caches (e.g., tracemalloc) in long-running services.

Practical Real-World Example: Caching a Computation Pipeline

Imagine a pipeline that computes user analytics from a set of events. Each run aggregates events for a user and applies filters. We'll use a frozen dataclass for parameters and lru_cache for memoization.

from dataclasses import dataclass
from functools import lru_cache
from typing import Tuple, Dict
@dataclass(frozen=True)
class AnalyticsParams:
    user_id: int
    filters: Tuple[str, ...]
@lru_cache(maxsize=1024)
def compute_user_analytics(params: AnalyticsParams) -> Dict:
    # Simulated heavy work
    # In reality, this might query a DB and run computations
    print(f"Computing for {params}")  # helps show cache misses
    result = {"user": params.user_id, "score": sum(len(f) for f in params.filters)}
    return result
Usage:
params = AnalyticsParams(user_id=42, filters=("clicks", "purchases"))
print(compute_user_analytics(params))  # computes
print(compute_user_analytics(params))  # cached

Explanation:

• AnalyticsParams is frozen, so instances are immutable and hashable.
• compute_user_analytics caches results keyed by params.
• print helps demonstrate cache usage.

Conclusion

Memoization via functools is a practical, high-impact optimization technique. Use:

• lru_cache for most cases (bounded cache with eviction)
• functools.cache for simple unbounded caches (use with caution)
• Frozen dataclasses for complex inputs
• Custom async memoizers for async def functions
• Careful design for multiprocessing scenarios (caches are per-process)

Want to go further? Explore:

• "A Practical Guide to Python's dataclasses: Simplifying Class Creation and Data Management" — for building clear, hashable data inputs
• "Exploring Python's async and await: Real-World Applications in I/O-Bound Tasks" — for async patterns and caches
• "Mastering Python's multiprocessing for Parallel Processing: A Case Study on Performance Improvement" — for parallelism and cache-sharing strategies

References:

• functools.lru_cache docs: https://docs.python.org/3/library/functools.html#functools.lru_cache
• dataclasses docs: https://docs.python.org/3/library/dataclasses.html
• asyncio docs: https://docs.python.org/3/library/asyncio.html

Call to action: Try adding @lru_cache or a small memoizer to one of your slow functions. Measure before and after, and share your results or questions—let's optimize Python together!