Python's standard library includes the collections module, which provides specialized data structures beyond the built-in types. These tools solve specific problems more efficiently than using lists, dictionaries, or tuples alone.
In this lesson, you'll explore Counter for counting objects, defaultdict for default values, deque for efficient queue operations, and namedtuple for structured data. These specialized containers will make your code more efficient and readable. By the end, you'll know when to reach for these tools instead of basic data structures.
What You'll Learn
- Overview of the collections module
- Counter for counting objects
- defaultdict for default values
- deque for efficient queues
- namedtuple for structured data
- When to use each specialized container
Introduction to Collections
The collections module provides alternatives to Python's built-in containers. Each is optimized for specific use cases:
from collections import Counter, defaultdict, deque, namedtuple
# We'll explore each of these in detail
These specialized containers aren't always necessary, but when you need them, they make your code significantly more efficient and readable.
Counter: Counting Made Easy
Counter is a dictionary subclass designed for counting hashable objects. It's perfect for frequency analysis:
from collections import Counter
# Count items in a list
words = ["apple", "banana", "apple", "orange", "banana", "apple"]
word_count = Counter(words)
print(word_count) # Counter({'apple': 3, 'banana': 2, 'orange': 1})
# Count characters in a string
text = "hello world"
char_count = Counter(text)
print(char_count)
# Counter({'l': 3, 'o': 2, 'h': 1, 'e': 1, ' ': 1, 'w': 1, 'r': 1, 'd': 1})
# Most common items
print(word_count.most_common(2)) # [('apple', 3), ('banana', 2)]
print(char_count.most_common(3)) # [('l', 3), ('o', 2), ('h', 1)]
Counter supports mathematical operations and works like a dictionary:
# Counter operations
c1 = Counter(["a", "b", "c", "a", "b"])
c2 = Counter(["a", "b", "b", "c"])
print(c1 + c2) # Counter({'a': 3, 'b': 4, 'c': 2})
print(c1 - c2) # Counter({'a': 1, 'c': 1})
print(c1 & c2) # Intersection: Counter({'a': 1, 'b': 2, 'c': 1})
print(c1 | c2) # Union: Counter({'a': 2, 'b': 2, 'c': 1})
# Access like a dictionary
print(c1["a"]) # 2
print(c1["x"]) # 0 (doesn't raise KeyError)
defaultdict: Automatic Defaults
defaultdict automatically creates default values for missing keys, eliminating the need to check if keys exist:
from collections import defaultdict
# Regular dictionary - verbose
word_groups = {}
for word in ["apple", "banana", "carrot", "broccoli"]:
first_letter = word[0]
if first_letter not in word_groups:
word_groups[first_letter] = []
word_groups[first_letter].append(word)
# defaultdict - clean and simple
word_groups = defaultdict(list)
for word in ["apple", "banana", "carrot", "broccoli"]:
word_groups[word[0]].append(word)
print(dict(word_groups))
# {'a': ['apple'], 'b': ['banana', 'broccoli'], 'c': ['carrot']}
You can use any callable as the default factory:
# Default to 0 for counting
counts = defaultdict(int)
for num in [1, 2, 2, 3, 3, 3]:
counts[num] += 1
print(dict(counts)) # {1: 1, 2: 2, 3: 3}
# Default to empty set
friends = defaultdict(set)
friends["Alice"].add("Bob")
friends["Alice"].add("Charlie")
print(dict(friends)) # {'Alice': {'Bob', 'Charlie'}}
# Custom default function
def default_score():
return {"points": 0, "games": 0}
players = defaultdict(default_score)
players["Alice"]["points"] = 100
print(dict(players)) # {'Alice': {'points': 100, 'games': 0}}
deque: Efficient Queues and Stacks
deque (double-ended queue) provides efficient append and pop operations from both ends. It's faster than lists for queue operations:
from collections import deque
# Create a deque
queue = deque([1, 2, 3])
print(queue) # deque([1, 2, 3])
# Efficient operations on both ends
queue.append(4) # Add to right end
queue.appendleft(0) # Add to left end
print(queue) # deque([0, 1, 2, 3, 4])
right = queue.pop() # Remove from right
left = queue.popleft() # Remove from left
print(f"Removed: {left} and {right}") # Removed: 0 and 4
print(queue) # deque([1, 2, 3])
Deques are perfect for implementing queues and stacks:
# Queue (FIFO - First In, First Out)
queue = deque()
queue.append("task1")
queue.append("task2")
queue.append("task3")
while queue:
task = queue.popleft() # Process in order
print(f"Processing: {task}")
# Stack (LIFO - Last In, First Out)
stack = deque()
stack.append("item1")
stack.append("item2")
stack.append("item3")
while stack:
item = stack.pop() # Process in reverse order
print(f"Popping: {item}")
Deques support rotation and have a maximum length option:
# Rotate elements
numbers = deque([1, 2, 3, 4, 5])
numbers.rotate(2) # Rotate right by 2
print(numbers) # deque([4, 5, 1, 2, 3])
numbers.rotate(-1) # Rotate left by 1
print(numbers) # deque([5, 1, 2, 3, 4])
# Bounded deque (max length)
limited = deque(maxlen=3)
limited.append(1)
limited.append(2)
limited.append(3)
print(limited) # deque([1, 2, 3], maxlen=3)
limited.append(4) # Automatically removes oldest
print(limited) # deque([2, 3, 4], maxlen=3)
namedtuple: Structured Data
namedtuple creates tuple subclasses with named fields, making code more readable:
from collections import namedtuple
# Define a Point namedtuple
Point = namedtuple("Point", ["x", "y"])
# Create instances
p1 = Point(3, 4)
p2 = Point(x=1, y=2)
print(p1) # Point(x=3, y=4)
print(p1.x) # 3
print(p1.y) # 4
print(p1[0]) # 3 (still works like tuple)
# Calculate distance
def distance(p1, p2):
return ((p2.x - p1.x)**2 + (p2.y - p1.y)**2)**0.5
dist = distance(p1, p2)
print(f"Distance: {dist:.2f}") # Distance: 2.83
Namedtuples are perfect for representing structured data without creating full classes:
# Represent a person
Person = namedtuple("Person", ["name", "age", "city"])
people = [
Person("Alice", 25, "London"),
Person("Bob", 30, "Manchester"),
Person("Charlie", 35, "Birmingham")
]
# Access by name (much clearer than indices)
for person in people:
print(f"{person.name} is {person.age} and lives in {person.city}")
# Convert to dictionary
alice_dict = people[0]._asdict()
print(alice_dict) # {'name': 'Alice', 'age': 25, 'city': 'London'}
# Replace fields (creates new instance)
alice_older = people[0]._replace(age=26)
print(alice_older) # Person(name='Alice', age=26, city='London')
Practical Examples
Let's combine these tools in real-world scenarios:
# Example 1: Word frequency analysis
from collections import Counter
text = "the quick brown fox jumps over the lazy dog the fox is quick"
words = text.split()
word_freq = Counter(words)
print(f"Most common: {word_freq.most_common(3)}")
# [('the', 3), ('quick', 2), ('fox', 2)]
# Example 2: Grouping data by category
from collections import defaultdict
products = [
{"name": "apple", "category": "fruit", "price": 1.20},
{"name": "banana", "category": "fruit", "price": 0.80},
{"name": "carrot", "category": "vegetable", "price": 0.50},
{"name": "broccoli", "category": "vegetable", "price": 1.00}
]
by_category = defaultdict(list)
for product in products:
by_category[product["category"]].append(product)
for category, items in by_category.items():
print(f"{category}: {len(items)} items")
# Example 3: Processing queue with deque
from collections import deque
tasks = deque(["task1", "task2", "task3"])
completed = []
while tasks:
current = tasks.popleft()
print(f"Processing {current}")
completed.append(current)
print(f"Completed: {completed}")
# Example 4: Using namedtuple for coordinates
from collections import namedtuple
Rectangle = namedtuple("Rectangle", ["width", "height"])
rect = Rectangle(10, 5)
area = rect.width * rect.height
print(f"Area: {area}") # Area: 50
Try It Yourself
Practice with these exercises:
-
Character Analysis: Use Counter to find the most common character in a long string, ignoring spaces.
-
Grouping Students: Use defaultdict to group students by their grade level from a list of student dictionaries.
-
Task Queue: Implement a task processing system using deque where tasks have priorities (process high priority first).
-
Coordinate System: Create a namedtuple for 3D coordinates (x, y, z) and write a function to calculate the distance between two 3D points.
-
Sliding Window: Use a bounded deque to keep track of the last 5 numbers in a data stream.
Summary
The collections module provides specialized data structures that solve specific problems efficiently. Counter simplifies counting and frequency analysis. defaultdict eliminates key-checking boilerplate. deque offers efficient queue and stack operations. namedtuple creates readable structured data without full class definitions.
These tools aren't always necessary—regular dictionaries, lists, and tuples work fine for many cases. But when you need their specific features, they make your code cleaner, faster, and more Pythonic. Understanding when to use each tool is a sign of an experienced Python programmer.
What's Next?
In the next lesson, we'll explore advanced function features. You'll learn about lambda functions, higher-order functions like map() and filter(), variable-length arguments, and function annotations. These concepts will make your functions more flexible and powerful.