Built-in map, filter, and reduce — Python: Functions

• The functools.reduce function
• The filter function
• The map function
• Side effects
• Impure functions and reduce
• Impure functions and map
• Impure functions and filter
• The essential rule

All three higher-level functions we know are build-in in Python. However, map and filter are ready to use, and reduce must be imported from the functools module. Now, we will discuss the differences between each specific function and the simple options we implemented earlier.

The functools.reduce function

Looking at the function declaration, we see:

reduce(function, sequence[, initial]) -> value

Here it is worth paying attention to the fact that the initial value of the accumulator is an optional argument [, initial]. If it is not specified, then reduce will use the first element of the sequence as the initial value. In this case, we remember that the reduce call will result in an error if the sequence is empty.

Let us observe an example of the functools.reduce function:

from functools import reduce

numbers = [2, 3, 8]

def get_maximum(first_num, second_num):
    return first_num if first_num > second_num else second_num

reduce(get_maximum, numbers, 10)  # 10
reduce(get_maximum, numbers, 4)  # 8

The filter function

Now let us take a look at filter:

filter(function or None, iterable) -> filter object

Here is the code:

numbers = [2, 3, 8, 15, 34, 42]

def is_even(num):
    return num % 2 == 0

filter(is_even, numbers)  # <filter object at ...>
list(filter(is_even, numbers))  # [2, 8, 34, 42]

The map function

The built-in map differs from our naive implementation:

map(func, *iterables) --> map object

Here map object is also an iterator. It is understandable. But the greedy argument *iterables accepts several iterable objects instead of one. If map passes more than one source of elements, then the func function will be called first for all the first elements, then for the second, and so on until the elements in at least one source run out.

It is somewhat similar to the zip function. The only difference is that zip always returns tuples, and map applies an arbitrary function from the number of sources corresponding to the number of arguments.

Here is an example of applying the map function to a pair of sources:

from operator import mul
map(mul, "abc", [3, 5, 7])  # <map object at ...>
list(map(mul, "abc", [3, 5, 7]))  # ['aaa', 'bbbbb', 'ccccccc']

Side effects

When using the alternatives to the for loop, we should follow one rule — the functions they support should be as pure as possible. Let us observe the definition:

• Pure functions return results that depend only on the arguments and do not produce any consequences unrelated to the results
• Impure functions have side effects — they print something to the console, write it to a file, and send it over the network
• Impure functions have calls depending on the outside world

What is wrong with using side effects with the map? The function can return a lazy iterator, so the effects occur not when we call it but when we take elements. We should note that side effects violate the laws characteristic of the mathematical versions of map, filter, and reduce.

Impure functions and reduce

When using functions with side effects for right and left folds, the equality of the result ceases to apply. It means the left fold will produce side effects for the elements from start to finish, and the right one will do so from finish to start. We will see a different overall effect — for example, the order of lines written to the file.

It happens that way even if the result returned from the function is the same. But, it may not be true when functions depend on the outside world, even this may not be true.

Impure functions and map

If we take an impure function, we will be breaking map's rules:

map(f, map(g, l)) == map(f*g, l)

Here we wrote f*g. It is a piece of pseudocode. It works as a composition of functions, obtaining this function:

x -> f(g(x))

There is no built-in operator for this composition in Python, but the corresponding function is easy to write.

Impure functions and filter

This kind of optimization is also possible for filter:

filter(f, filter(g, l)) == filter(g&f, l)

Here, g&f represents the function that does:

x -> g(x) and f(x)

Again, there is no such operator in Python, but it is simple to implement this kind of function. The effects are also mixed with filter after optimization, like the map function above.

The essential rule

Pure and impure functions have different properties and applications.

Pure functions are preferable. They are easier to understand, easier to test, and they work well in multithreaded applications. Impure functions could help when we change the program state or perform I/O. However, they can make debugging and testing the application difficult.

We recommend that you use clean functions whenever possible. And it is better to limit the use of impure functions. We should use them only when it is necessary.