Functions as objects, lambdas and closures

Why this matters

In Python a function is a value, just like a number or a list. You can store it in a variable, put it in a dict, pass it to another function, and return it from a function. This single fact powers a huge range of techniques: sorted(key=...), callbacks, event handlers, dispatch tables, and the decorators of the next lesson. Closures, functions that remember variables from where they were defined, let you package behavior together with private state without writing a class. Understanding them precisely also explains one of Python's most famous gotchas, late binding in loops.

Functions are objects

def does two things at run time: it creates a function object and binds a name to it.

def greet(name):
    return f"hello, {name}"

print(type(greet))
say = greet                 # another name for the same object
print(say("ada"))
print(greet.__name__, say is greet)

actions = {"greet": greet, "shout": str.upper}
print(actions["shout"]("hi"))

<class 'function'>
hello, ada
greet True
HI

say = greet copies no code; both names refer to one function object (names bind to objects, as always). The function object carries its compiled bytecode (greet.__code__), its name, its defaults (__defaults__) and a reference to the globals of its module. Calling it, greet("ada"), creates a new frame and runs the code.

Higher-order functions

A higher-order function takes a function as an argument or returns one. You have used several:

words = ["banana", "Kiwi", "apple"]
print(sorted(words))                    # by code point: uppercase first
print(sorted(words, key=str.lower))     # pass a function as the key
print(list(map(len, words)))
print(max(words, key=len))

['Kiwi', 'apple', 'banana']
['apple', 'banana', 'Kiwi']
[6, 4, 5]
banana

Note key=str.lower, not key=str.lower(): you hand sorted the function itself, and it calls the function once per element. Writing your own is just as easy:

def apply_twice(f, x):
    return f(f(x))

print(apply_twice(lambda s: s + "!", "hey"))

hey!!

lambda

lambda params: expression creates a small anonymous function whose body is a single expression, returned automatically:

square = lambda x: x * x
print(square(7))
pairs = [(3, "c"), (1, "a"), (2, "b")]
print(sorted(pairs, key=lambda p: p[1]))

49
[(1, 'a'), (2, 'b'), (3, 'c')]

A lambda is exactly a function object, just written inline and named <lambda>. It cannot contain statements (no if blocks, loops or assignments, though a conditional expression a if c else b is fine). Use it for short throwaway functions, mostly as arguments. If you are assigning a lambda to a name, a def is clearer and gives better tracebacks.

Closures

A function defined inside another function can use the outer function's local variables:

def make_greeter(greeting):
    def greet(name):
        return f"{greeting}, {name}!"
    return greet

hello = make_greeter("Hello")
hola = make_greeter("Hola")
print(hello("Ada"), hola("Ada"))

Hello, Ada! Hola, Ada!

Strange, if you think about it: by the time hello("Ada") runs, make_greeter has returned and its frame is gone. Where does greeting come from?

When the compiler sees that the inner function uses a variable of the enclosing function, it stores that variable in a cell: a small object that holds a reference to the value, shared by the outer frame and the inner function. Each call to make_greeter creates a new cell, and the inner function object keeps a reference to it in __closure__:

print(hello.__code__.co_freevars)
print(hello.__closure__[0].cell_contents)
print(hola.__closure__[0].cell_contents)

('greeting',)
Hello
Hola

 make_greeter("Hello") call               hello (function object)
 +----------------------+                 +-----------------------+
 | greeting --> [cell] -+------+          | __code__: greet code  |
 +----------------------+      +--------> | __closure__: (cell,)  |
    (frame discarded)          cell --> "Hello"

A function plus the cells it captured is a closure. Closures give you private, persistent state: nothing outside can reach greeting except through the function.

Late binding: closures capture variables, not values

A closure stores a reference to the variable (the cell), and looks up its value when the inner function runs:

def make():
    x = 1
    def show():
        return x
    x = 2
    return show

print(make()())

2

That explains the classic loop bug:

funcs = [lambda: i for i in range(3)]
print([f() for f in funcs])

[2, 2, 2]

All three lambdas share one variable i, whose final value is 2. Two standard fixes bind the value at creation time:

funcs = [lambda i=i: i for i in range(3)]           # default arg evaluated now
print([f() for f in funcs])

from functools import partial
def show(i):
    return i
funcs = [partial(show, i) for i in range(3)]        # partial stores the value
print([f() for f in funcs])

[0, 1, 2]
[0, 1, 2]

functools.partial(f, a) returns a callable that calls f(a, ...), with the argument values frozen in. It is a handy alternative to small lambdas.

Changing captured variables: nonlocal

Reading an enclosing variable is automatic. Assigning to a name inside a function makes it local to that function (the scope rule from the functions module), so this fails:

def make_counter():
    count = 0
    def inc():
        count += 1
        return count
    return inc

make_counter()()

UnboundLocalError: cannot access local variable 'count' where it is not associated with a value

count += 1 is an assignment, so the compiler treats count as a local of inc, and it is read before being set. Declare it nonlocal to rebind the enclosing variable instead:

def make_counter():
    count = 0
    def inc():
        nonlocal count
        count += 1
        return count
    return inc

c = make_counter()
print(c(), c(), c())

1 2 3

Mutating an object does not need nonlocal (items.append(x) is not an assignment to items), but rebinding a name does. A closure with nonlocal state is a lightweight alternative to a class with one method; when you need several methods or the state grows, write a class.

Common mistakes

• Calling instead of passing: sorted(words, key=str.lower()) raises TypeError: unbound method str.lower() needs an argument. Pass str.lower without parentheses.
• Returning the call result instead of the function: return inner() returns a value; return inner returns the function.
• Late binding in loops: lambdas or inner functions created in a loop all see the final value. Use a default argument or partial.
• Assigning to a captured variable without nonlocal: UnboundLocalError.
• Long lambdas: if it does not fit comfortably on one line, use def.

Recap

• Functions are objects: assign them, store them, pass them, return them.
• Higher-order functions take or return functions; key= arguments are the everyday example.
• lambda args: expr makes a small anonymous function.
• A closure is a function plus the cells holding variables it uses from enclosing scopes; values are looked up when it runs (late binding).
• nonlocal name lets an inner function rebind an enclosing variable.

Going further: read the functools docs for partial and reduce, and inspect fn.__code__.co_freevars and fn.__closure__ for your own closures.