Modular tests — Python: Automated testing

• What is unit testing
• Testing basic features
• Testing additional features
• Borderline cases

The information we've looked at is already enough for everyday testing. Before we dive into more complex topics and features of Pytest, let's go through the testing path of the library. We'll talk about organizing tests and good and bad practices. It will help you get the right attitude towards testing in general.

What is unit testing

In this lesson, we'll analyze the basics of unit testing**. This type of testing examines program modules in isolation from all other parts. They test basic language constructs such as functions, modules, and classes. These tests don't guarantee that the entire application will work, but they help when a program module has complex logic.

Let's try to test a stack. You'll no doubt remember that a stack is a list of items organized by LIFO (Last In First Out). Typically, stacks themselves are often used to implement algorithms. Developers often use them in low-level code, such as within programming languages or operating systems:

stack = []  # Python implements stacks using lists
not stack  # The list is empty
# True
stack.append(1)  # [1]
stack.append(2)  # [1, 2]
stack.append(3)  # [1, 2, 3]
not stack  # The list is not empty
# False
stack
# [1, 2, 3]
stack.pop()  # In the stack [1, 2]
# 3
stack.pop()  # In the stack [1]
# 2
stack.pop()  # The stack is empty
# 1
not stack
# True

Testing basic features

Let's write our first test. The first test should always check a positive scenario, one that involves the main functionality of the component:

def test_stack():
    stack = []
    # We add two elements to the stack and then extract them
    # Why two? It's more reliable than one, but not as excessive as three
    stack.append('one')
    stack.append('two')

    assert stack.pop() == 'two'
    assert stack.pop() == 'one'

This test checks whether the two main methods work, excluding borderline cases. The test runs two statements that examine the values extracted from the stack. On the Internet, you may read opinions saying that multiple checks within a test are wrong.

Many people believe that tests must be as detailed as possible and that you must create a new test for each check:

def test_stack1():
    stack = []
    stack.append('one')
    stack.append('two')

    assert stack.pop() == 'two'

def test_stack2():
    stack = []
    stack.append('one')
    stack.append('two')

    stack.pop()
    assert stack.pop() == 'one'

It's worthwhile to highlight other scenarios in a separate test that require different data or perform a different sequence of actions. But this approach doesn't always work. It often leads to code bloat and duplication for no apparent benefit.

Testing additional features

The next test is a test for additional stack functions. It includes a check to see if the stack is empty:

def test_emptiness():
    stack = []
    assert not stack
    stack.append('one')
    assert bool(stack)  # Not `not stack`

    stack.pop()
    assert not stack

This test covers three situations at once:

• The initial state of the stack
• The state of the stack after adding elements
• The state of the stack after removing all elements

In principle, this is sufficient. However, there may be situations where the `not' check fails. Do you have to try to find all the options? No. Every line of code in the project is a potential place to change in case of edits. If there's any doubt about whether a check is needed, it's better not to write it. That means you'll find the minimum that's worth it. Rare situations require test coverage only if they are critical to performance.

Borderline cases

The last thing we can test is the behavior of the pop() function when there are no elements on the stack.

By design, the stack throws an exception if we attempt to pop elements from it when it is empty. In other words, this situation is considered an error; the programmer should always make sure that the stack isn't empty:

import pytest

def test_pop_with_empty_stack():
    stack = []
    with pytest.raises(IndexError):
        stack.pop()

Borderline cases aren't always so easy to see. It is unlikely that any programmer can write all the necessary tests at once.

Imagine an error occurred in code that didn't have a test. In that case, write a test that reproduces that bug and then fix it. It is the only way to maintain a sufficient level of reliability without turning development into continuous bug fixing.

Recommended materials

1. Checklist of good engineering practices in companies