30. Stdlib Power Tools – itertools, functools, collections
Introduction
Python’s standard library includes powerful modules that simplify common programming tasks. In this lesson, we explore three such modules: itertools, functools, and collections. These modules provide efficient tools for iteration, functional programming, and data structures.
Itertools
The itertools module provides a set of fast, memory-efficient tools for working with iterators. These tools are useful for looping, filtering, and combining data.
Key Functions and Examples:
1. count(start=0, step=1): Infinite counter
Example: itertools.count(10, 2) → 10, 12, 14, …
2. cycle(iterable): Repeats elements infinitely
3. repeat(item, times): Repeats an item multiple times
4. chain(*iterables): Combines multiple iterables
5. combinations(iterable, r): r-length tuples without replacement
6. permutations(iterable, r): r-length tuples with replacement
functools
The functools module provides higher-order functions that act on or return other functions. It supports functional programming techniques such as memoization and partial application.
Key Functions and Examples:
1. lru_cache(maxsize=128): Caches function results
Example:
@lru_cache
def fib(n): return fib(n-1) + fib(n-2)
2. partial(func, *args, **kwargs): Fixes arguments of a function
3. reduce(function, iterable): Applies function cumulatively
Example:
@lru_cache
def fib(n): return fib(n-1) + fib(n-2)
Collections:
The collections module provides specialized container datatypes that enhance the built-in types. These include named tuples, default dictionaries, counters, and more.
Key Classes and Examples:
1. namedtuple(): Tuple with named fields
Example: Point = namedtuple(‘Point’, [‘x’, ‘y’])
2. defaultdict(default_factory): Dictionary with default values
3. Counter(): Counts hashable items
4. deque(): Double-ended queue
5. ChainMap(): Combines multiple dictionaries
Use Cases
– itertools: Efficient looping and combinatorics
– functools: Memoization, function composition, partial functions
– collections: Enhanced data structures for cleaner code
Performance Benefits
These modules are implemented in C and optimized for performance. They reduce memory usage, improve execution speed, and simplify complex operations.
Best Practices
– Use itertools for large or infinite sequences
– Use functools.lru_cache for expensive recursive functions
– Use collections.defaultdict and Counter for cleaner data handling
Best Practices
– itertools functions may produce infinite sequences; use with caution
– functools.reduce can be less readable than loops
– collections.namedtuple is immutable and cannot be modified