In a developer's everyday life, you will often see code that works with iterations because iterators are built into the language and tightly integrated into the standard library.
We usually assemble iterators and operations on them into data conveyors. Only at the end of each pipeline is reduce()
or something else that doesn't pass elements on. Most of these pipelines consist of two types of operations:
- Converting individual elements with the
map()
function. It converts the entire stream using another function that handles the individual items - Changing the composition of the elements via filtration or multiplication. The
filter()
function can filter the data. And themap()
paired with thechain()
from theitertools
module turns each element into several without changing the nesting level
For example, imagine we want a list of numbers like this:
[0, 0, 2, 2, 4, 4...]
There are two copies each of increasing even numbers. Let's write a suitable pipeline:
# Getting a stream of even numbers
def is_even(x):
return x % 2 == 0
list(filter(is_even, range(20)))
# [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
# Doubling each of them
def dup(x):
return [x, x]
list(map(dup, filter(is_even, range(20))))
# [[0, 0], [2, 2], [4, 4], [6, 6], [8, 8], [10, 10], [12, 12], [14, 14], [16, 16], [18, 18]]
# Making the pipeline flat again
from itertools import chain
list(chain(*map(dup, filter(is_even, range(20)))))
# [0, 0, 2, 2, 4, 4, 6, 6, 8, 8, 10, 10, 12, 12, 14, 14, 16, 16, 18, 18]
# Making a single-line variant
list(chain(*map(lambda x: [x, x], filter(lambda x: x % 2 == 0, range(20)))))
# [0, 0, 2, 2, 4, 4, 6, 6, 8, 8, 10, 10, 12, 12, 14, 14, 16, 16, 18, 18]
As you can see, the task is done by connecting ready-made elements rather than by writing all the code manually in the form of a for
loop. We can already see the issue with our constructor: if there are no ready-made functions on elements or predicates, we will declare them beforehand or use lambda
. Both options are inconvenient.
When another person reads our code with individual functions, they have to keep jumping back and forth through the code. And lambda
looks unwieldy. But don't despair, Python has a syntax that can simplify working with conveyors.