Iterators, generators, closures and decorators · about 45 min

Type hints

Why this matters

In a 50-line script you can keep every variable's type in your head. In a 50,000-line codebase you cannot: is user a dict, a User object, or a user id? Does this function return None when nothing is found? Type hints let you write the answers into the code itself: def find_user(uid: int) -> User | None. Editors use them for autocompletion and inline errors, and static type checkers such as mypy and pyright read them to find bugs (calling a method on something that might be None, passing a str where an int is needed) without running the program. They are optional, gradual, and now standard in professional Python.

The syntax

Annotate parameters with : type, return values with -> type, and variables with name: type = value:

def greet(name: str, times: int = 1) -> str:
    return ", ".join([f"hi {name}"] * times)

count: int = 0
names: list[str] = []

A function that returns nothing useful is annotated -> None.

Not enforced at run time

This is the most important fact about type hints: CPython ignores them when running your code. They are evaluated once when the function is defined, stored, and otherwise unused:

def double(x: int) -> int:
    return x * 2

print(double("ab"))
print(double.__annotations__)

abab
{'x': <class 'int'>, 'return': <class 'int'>}

No error. Hints are information for tools and readers; the interpreter does not slow down to check them. typing.get_type_hints(func) returns the same dict, resolving any annotations written as strings. (Libraries such as dataclasses read annotations to know the fields; some frameworks use them for validation, but that is the library acting, not the language.)

Checking happens with a separate tool, run like a linter, before you run the program:

$ mypy app.py
app.py:4: error: Argument 1 to "double" has incompatible type "str"; expected "int"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

mypy is not installed in the learnprog sandbox (no extra packages), so exercises here check annotations by inspecting them. On your own machine, pip install mypy in a virtual environment and try it on your code.

Types you will write

Built-in classes are types: int, float, str, bool, bytes, and your own classes. Containers take their element types in square brackets (since Python 3.9):

scores: list[int] = [90, 85]
ages: dict[str, int] = {"ada": 36}
point: tuple[float, float] = (1.0, 2.5)        # exactly two floats
row: tuple[str, ...] = ("a", "b", "c")         # any length, all str
tags: set[str] = {"x"}

Unions say "one of these" with | (Python 3.10+). The most common union is "or None":

def find(words: list[str], target: str) -> int | None:
    for i, w in enumerate(words):
        if w == target:
            return i
    return None

i = find(["a", "b"], "b")
print(i + 1)      # a type checker complains: i may be None

The checker forces you to handle None first (if i is not None:), which prevents the very common TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'. Older code writes Optional[int] (from typing), which means the same as int | None.

Abstract collection types from collections.abc describe what you need rather than a concrete class. Accept the most general type that works, return a concrete one:

from collections.abc import Iterable, Sequence, Mapping

def total(nums: Iterable[float]) -> float:    # list, tuple, set, generator... all fine
    return sum(nums)

def middle(items: Sequence[str]) -> str:      # needs len() and indexing
    return items[len(items) // 2]

Callables: Callable[[arg types], return type]:

from collections.abc import Callable

def apply(func: Callable[[int], int], values: list[int]) -> list[int]:
    return [func(v) for v in values]

Any (from typing) switches checking off for a value. It is an escape hatch; use it sparingly.

Generics: TypeVar

What is the return type of a function that returns the first element of any list? Not Any: we want "the same type as the elements". A type variable expresses that:

from collections.abc import Sequence
from typing import TypeVar

T = TypeVar("T")

def first(items: Sequence[T]) -> T:
    return items[0]

n = first([1, 2, 3])        # checker infers int
s = first(["a", "b"])       # checker infers str

Python 3.12 added a shorter syntax that declares the type variable inline: def first[T](items: Sequence[T]) -> T:. Classes can be generic too (class Stack[T]: or, in older style, class Stack(Generic[T])), so that Stack[int] is a stack of ints.

Structural typing: Protocol

Python code relies on duck typing: anything with a read() method can be used as a file. A Protocol writes that down for the type checker: "any object with these methods", regardless of its class:

from typing import Protocol

class Greeter(Protocol):
    def greet(self, name: str) -> str: ...

class English:
    def greet(self, name: str) -> str:
        return f"Hello, {name}"

class Pirate:
    def greet(self, name: str) -> str:
        return f"Ahoy, {name}"

def welcome(g: Greeter) -> None:
    print(g.greet("Ada"))

welcome(English())
welcome(Pirate())

Hello, Ada
Ahoy, Ada

Neither class inherits from Greeter; they match by structure. Contrast this with the abstract base classes of the OOP module, which require explicit inheritance (nominal typing). Adding @typing.runtime_checkable lets isinstance(obj, Greeter) check, at run time, that the methods exist (it does not check their signatures).

Gradual typing in practice

You do not have to annotate everything at once. Unannotated code is treated as Any and not checked, so teams add hints module by module, starting with public function signatures. Annotate function parameters and returns; local variables are usually inferred (count = 0 is obviously an int). Annotate empty containers, where inference cannot know the element type: seen: set[str] = set().

A detail you will meet in older code: from __future__ import annotations makes all annotations lazily evaluated strings, which allowed forward references to classes not yet defined. Python 3.14 makes annotations lazily evaluated by default.

Common mistakes

Expecting run-time errors from wrong types: nothing happens unless a tool checks.
list[int]() or annotations used as values: annotations describe; build values normally ([]).
Mutable default with a hint: def f(items: list[str] = []) still has the shared-default bug; use items: list[str] | None = None.
Over-precise parameters: def total(nums: list[int]) rejects a perfectly good tuple or generator in the checker's eyes; use Iterable[int].
Old spellings: List[int], Dict[str, int], Optional[X] from typing still work but are legacy; prefer list[int], dict[str, int], X | None.

Recap

Annotate with param: T, -> T and var: T; annotations are stored in __annotations__ and not enforced by CPython.
Static checkers (mypy, pyright) use them to catch bugs before running.
Use built-in generics (list[int], dict[str, int], tuple[str, ...]), unions (int | None), collections.abc types (Iterable, Sequence, Callable).
TypeVar (or def f[T]) links input and output types; Protocol describes duck-typed interfaces structurally.

Going further: read the mypy "cheat sheet" (mypy.readthedocs.io) and the typing module documentation; PEP 484 introduced type hints and PEP 544 introduced protocols.

Exercises

Exercise 1 · quiz

Are hints enforced?

What happens when this runs?

def double(x: int) -> int:
    return x * 2

print(double("ab"))

TypeError: argument x must be int
It prints abab
SyntaxError, because the annotation does not match
It prints 0

Answer

It prints abab

Python does not check annotations at run time. They are stored in double.__annotations__ and otherwise ignored by the interpreter, so "ab" * 2 runs and prints abab. A static type checker such as mypy or pyright would flag the call before running, which is the whole point of hints: catching mistakes by analysis, not at run time.

Exercise 2 · write code

Annotate a module

solution.py contains working but unannotated functions. Add complete type hints (every parameter and every return type) without changing behavior:

word_counts(text): takes a str, returns a dict[str, int].
top(counts, n): takes a dict[str, int] and an int, returns a list[tuple[str, int]].
find(words, target): takes a list[str] and a str, returns an int | None (the index, or None).
apply(func, values): takes a function from int to int (Callable[[int], int], imported from collections.abc) and a list[int], returns a list[int].

The check inspects the annotations with typing.get_type_hints, so use the modern built-in generic syntax shown here exactly.

File: solution.py

Starting point

from collections.abc import Callable


def word_counts(text):
    counts = {}
    for w in text.lower().split():
        counts[w] = counts.get(w, 0) + 1
    return counts


def top(counts, n):
    return sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))[:n]


def find(words, target):
    for i, w in enumerate(words):
        if w == target:
            return i
    return None


def apply(func, values):
    return [func(v) for v in values]

Hint 1

Parameters: def top(counts: dict[str, int], n: int). Return type after the parentheses: -> list[tuple[str, int]]:.

Hint 2

Optional results: -> int | None.

Hint 3

A function parameter: func: Callable[[int], int] (argument types in a list, then the return type).

Reference solution

from collections.abc import Callable


def word_counts(text: str) -> dict[str, int]:
    counts: dict[str, int] = {}
    for w in text.lower().split():
        counts[w] = counts.get(w, 0) + 1
    return counts


def top(counts: dict[str, int], n: int) -> list[tuple[str, int]]:
    return sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))[:n]


def find(words: list[str], target: str) -> int | None:
    for i, w in enumerate(words):
        if w == target:
            return i
    return None


def apply(func: Callable[[int], int], values: list[int]) -> list[int]:
    return [func(v) for v in values]

Exercise 3 · write code

Generic functions with TypeVar

In solution.py, write two generic functions using a type variable, so a type checker knows the output type matches the input type:

first(items: Sequence[T], default: T) -> T returns items[0], or default if items is empty.
group_by(items: Iterable[T], key: Callable[[T], K]) -> dict[K, list[T]] groups items into a dict of lists by key(item), keeping each group in input order, with keys in order of first appearance.

Define the type variables with T = TypeVar("T") and K = TypeVar("K") from typing, and import Sequence, Iterable, Callable from collections.abc. (Python 3.12+ also allows def first[T](...); either style is accepted as long as the annotations are present.)

group_by(["apple", "avocado", "banana"], key=lambda w: w[0]) returns {"a": ["apple", "avocado"], "b": ["banana"]}.

File: solution.py

Starting point

from collections.abc import Callable, Iterable, Sequence
from typing import TypeVar


def first(items, default):
    return items[0]


def group_by(items, key):
    return {}

Hint 1

items[0] if items else default handles the empty case.

Hint 2

groups.setdefault(k, []).append(item) creates each list on first use; dicts keep insertion order.

Hint 3

Annotations: def first(items: Sequence[T], default: T) -> T:.

Reference solution

from collections.abc import Callable, Iterable, Sequence
from typing import TypeVar

T = TypeVar("T")
K = TypeVar("K")


def first(items: Sequence[T], default: T) -> T:
    return items[0] if items else default


def group_by(items: Iterable[T], key: Callable[[T], K]) -> dict[K, list[T]]:
    groups: dict[K, list[T]] = {}
    for item in items:
        groups.setdefault(key(item), []).append(item)
    return groups

Exercise 4 · write code

Structural typing with Protocol

In solution.py:

Define a Protocol named HasArea (from typing) with one method area(self) -> float. Decorate it with @runtime_checkable so isinstance works with it.
Write two unrelated classes, Square(side) and Circle(radius), each with an area() method (use math.pi for the circle). Neither may inherit from HasArea.
Write total_area(shapes: Iterable[HasArea]) -> float returning the sum of the areas, rounded to 2 decimals with round(..., 2).

total_area([Square(2), Circle(1)]) returns 7.14. And isinstance(Square(1), HasArea) is True even though Square never mentions HasArea, while isinstance("text", HasArea) is False.

File: solution.py

Starting point

import math
from collections.abc import Iterable
from typing import Protocol, runtime_checkable


class Square:
    def __init__(self, side):
        self.side = side


def total_area(shapes):
    return 0.0

Hint 1

class HasArea(Protocol): with def area(self) -> float: ... as the body.

Hint 2

Put @runtime_checkable on the line above class HasArea.

Hint 3

Square and Circle are ordinary classes (class Square:), they just happen to have a matching area method.

Reference solution

import math
from collections.abc import Iterable
from typing import Protocol, runtime_checkable


@runtime_checkable
class HasArea(Protocol):
    def area(self) -> float: ...


class Square:
    def __init__(self, side: float) -> None:
        self.side = side

    def area(self) -> float:
        return self.side ** 2


class Circle:
    def __init__(self, radius: float) -> None:
        self.radius = radius

    def area(self) -> float:
        return math.pi * self.radius ** 2


def total_area(shapes: Iterable[HasArea]) -> float:
    return round(sum(s.area() for s in shapes), 2)