Testing and debugging · about 45 min

Debugging with pdb and breakpoint()

Why this matters

Print debugging means editing the code, rerunning it, reading the output, and repeating, each time guessing in advance which values you will need. A debugger lets you pause a running program at any line and look around: inspect every variable, evaluate expressions, step through the code one line at a time, and walk up the call stack, all without editing anything. Python's built-in debugger, pdb, runs in any terminal, on any machine you can SSH into, with nothing to install. Ten minutes learning its handful of commands pays off for the rest of your career.

How a debugger works

CPython lets a program install a trace function that the interpreter calls on events: each new line, each function call and return, each exception (sys.settrace underneath; Python 3.12+ also has the faster sys.monitoring). pdb installs one, and when the current file and line match a breakpoint, or you asked to step, it pauses the program and reads commands from you. The program is frozen mid-flight, with its whole call stack and every frame's local variables still in memory, which is why you can inspect them.

Starting the debugger

From your code, with breakpoint(). Put this line where you want to stop:

def average(scores):
    total = sum(scores)
    breakpoint()
    return total / len(scores)

print(average([80, 90, 100]))

$ python3 avg.py
> /home/ada/avg.py(4)average()
-> return total / len(scores)
(Pdb) p total
270
(Pdb) p len(scores), scores
(3, [80, 90, 100])
(Pdb) c
90.0

The > line shows the file, line number and function; -> shows the line about to run (it has not run yet). (Pdb) is the prompt. breakpoint() is a built-in (Python 3.7+) that calls pdb.set_trace() by default. Setting the environment variable PYTHONBREAKPOINT=0 disables every breakpoint() call, a safety net if you forget one. Still, remove them before committing.

From the command line, without editing the code.

$ python3 -m pdb avg.py
> /home/ada/avg.py(1)<module>()
-> def average(scores):
(Pdb)

The program stops before its first line, and you set breakpoints from there.

The essential commands

Most commands have a one-letter abbreviation.

Command	Does
`l` (list) / `ll`	show source around the current line / the whole current function
`n` (next)	run the current line, stepping over function calls
`s` (step)	run the current line, stepping into a function it calls
`c` (continue)	run until the next breakpoint (or the end)
`r` (return)	run until the current function returns
`unt N` (until)	run until line N in the current frame (handy to finish a loop)
`b N` / `b func` / `b file.py:N`	set a breakpoint at a line or function
`b N, condition`	conditional breakpoint: stop only when the condition is true
`b` / `cl N`	list breakpoints / clear breakpoint number N
`p expr` / `pp expr`	print (pretty-print) the value of an expression
`display expr`	show expr's value every time the program stops, if it changed
`w` (where)	show the call stack, current frame marked with `>`
`u` / `d` (up, down)	move to the calling frame / back down, to inspect its variables
`a` (args)	arguments of the current function
`interact`	open a full Python REPL in the current scope (Ctrl-D to return)
`q` (quit)	abort the program

Pressing Enter on an empty line repeats the last command, so stepping is n, Enter, Enter, Enter. Anything that is not a command is run as Python, so you can type total * 2 or even total = 0 to change a variable and see what happens. When a variable name collides with a command (a variable named n or c), prefix with !: !n evaluates the variable, p n also works.

next versus step

The difference matters constantly:

-> total = compute_total(items)

n: run compute_total completely, stop at the next line here
s: stop at the first line inside compute_total

Use n to walk through code you trust and s to dive into a call you suspect. If you stepped into something uninteresting, r runs to its end and stops as it returns.

Conditional breakpoints

Stopping on every iteration of a 10,000-step loop is useless. Stop only when the interesting thing happens:

(Pdb) b 14, qty == 0
Breakpoint 1 at /home/ada/report.py:14
(Pdb) c
> /home/ada/report.py(14)unit_price()
-> return row["total"] / row["qty"]
(Pdb) p row
{'name': 'widget', 'total': 0, 'qty': 0}

Walking the stack

w prints the call stack like a traceback, newest frame at the bottom, with > at the frame you are inspecting:

(Pdb) w
  /home/ada/report.py(22)<module>()
-> main()
  /home/ada/report.py(18)main()
-> print(format_row(row))
> /home/ada/report.py(14)unit_price()
-> return row["total"] / row["qty"]
(Pdb) u
> /home/ada/report.py(18)main()
-> print(format_row(row))
(Pdb) p row_number
57

u moved to the caller, main, so p now sees main's local variables. The program does not move; you only change which frame you are looking at.

Post-mortem debugging

The best moment to inspect a crash is the moment it happens. Run the script under pdb with an immediate continue:

$ python3 -m pdb -c continue report.py
Traceback (most recent call last):
  ...
ZeroDivisionError: division by zero
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /home/ada/report.py(6)unit_price()
-> return row["total"] / row["qty"]
(Pdb) p row
{'name': 'widget', 'total': 0, 'qty': 0}

pdb catches the uncaught exception and stops in the frame that raised it, with all variables intact. In the REPL, after something has crashed, import pdb; pdb.pm() ("post-mortem") does the same for the most recent exception. In your own code, pdb.post_mortem() inside an except block opens the debugger on the exception being handled.

Scripted sessions

pdb reads commands from stdin like any program, so you can save a session in a file and replay it:

$ printf 'b 7\nc\np v, best\nq\n' > commands.pdb
$ python3 -m pdb buggy.py < commands.pdb

This is useful for reproducing an investigation, and it is how the exercise in this lesson is checked. A file named .pdbrc in your home or current directory is read at startup in the same way, a good place for aliases and favorite settings.

Which tool when

Traceback: always read first. Often enough by itself.
print / logging: quick checks, and for programs where pausing is impossible (services, timing-sensitive code, code running on another machine). Logging stays useful in production.
pdb: when you do not know what to look at yet, when the state is complex, or when each rerun is slow. Exploring live beats guessing in advance.
Editors like VS Code and PyCharm put a graphical front end on the same machinery: click to set breakpoints, hover to see values. The concepts (breakpoints, step over and into, call stack, watch expressions) are identical.

Common mistakes

Thinking the -> line has already run. It is the next line to execute. p a variable assigned on that line and you see its old value, or a NameError.
Using s everywhere and ending up deep inside the standard library. Use n over code you trust, r to climb out.
A variable named like a command (c, n, l, p): typing it runs the command. Use p c or !c.
Leaving breakpoint() in code. A program that hangs waiting for (Pdb) input in production or in a test run is a classic. Search for it before committing, and set PYTHONBREAKPOINT=0 in automated environments.
Debugging the wrong file. Breakpoints by line number refer to the file pdb is running; b other.py:12 for another module.

Recap

breakpoint() in code, or python3 -m pdb script.py, starts pdb; it pauses with the next line shown after ->.
Core commands: n, s, c, r, unt, b N [, cond], p/pp, l/ll, w, u/d, q; Enter repeats; other input is evaluated as Python.
Post-mortem: python3 -m pdb -c continue script.py, or pdb.pm() in the REPL, stops in the crashing frame with its variables.
pdb reads stdin, so sessions can be scripted with a command file.

Going further: in a terminal, run python3 -m pdb on one of your earlier exercise solutions and step through it with n, s and p. Then read the pdb documentation's "Debugger Commands" section for display, commands (run commands automatically at a breakpoint) and retval.

Exercises

Exercise 1 · quiz

next or step?

The debugger is stopped on this line inside main():

-> total = compute_total(items)

You suspect the bug is inside compute_total. Which command takes you to the first line of compute_total?

n (next)
s (step)
c (continue)
r (return)

Answer

s (step)

s (step) executes the current line but stops at the very first opportunity, which is inside the called function. n (next) runs the whole call and stops at the next line of main, which is what you want when you trust the function. c runs until the next breakpoint, and r runs until the current function returns.

Exercise 2 · quiz

After the crash

A long script crashes with an IndexError after several minutes of work, deep inside a helper function. You want to look at the local variables of the frame where it crashed, as they were at the moment of the crash. Which approach does that?

Add print statements and wait for the crash again
Run python3 -m pdb -c continue script.py; when the exception is uncaught, pdb enters post-mortem mode in the crashing frame
Run python3 -O script.py so the error is skipped
Read the traceback; local variables are always listed in it

Answer

Run python3 -m pdb -c continue script.py; when the exception is uncaught, pdb enters post-mortem mode in the crashing frame

python3 -m pdb -c continue script.py runs the script under pdb without stopping at the start; if an exception goes uncaught, pdb stops in post-mortem mode at the frame that raised it. There you can p any variable, w to see the stack, and u/d to inspect callers' variables. In the REPL, import pdb; pdb.pm() does the same for the last exception. A traceback shows lines, not variable values.

Exercise 3 · write code

Drive pdb with a command file

pdb reads its commands from stdin, so a debugging session can be scripted: python3 -m pdb buggy.py < commands.pdb runs the commands in the file, one per line, as if you typed them.

buggy.py (in your working directory) contains:

def running_max(values):
    result = []
    best = 0
    for v in values:
        if v > best:
            best = v
        result.append(best)
    return result


def main():
    temps = [-5, -2, -8, -1]
    print(running_max(temps))


if __name__ == "__main__":
    main()

It should print the running maximum [-5, -2, -2, -1] but prints [0, 0, 0, 0]. Write commands.pdb, a list of pdb commands that:

sets a breakpoint on line 7 (result.append(best)),
continues to it,
prints the expression v, best (this shows (-5, 0) on the first iteration: there is the bug),
quits.

This file is pdb commands, not Python, so the Run button will not help; use Check. The checker runs python3 -m pdb buggy.py < commands.pdb and looks for (-5, 0) printed at the breakpoint.

File: commands.pdb

Starting point

continue

Hint 1

break 7 (or b 7) sets a breakpoint on line 7 of the file being debugged.

Hint 2

continue (or c) runs until a breakpoint is hit.

Hint 3

p v, best prints a tuple of the two values. Then quit (or q).

Reference solution

break 7
continue
p v, best
quit

Exercise 4 · write code

Fix the bug you found

Now fix it. In solution.py, write running_max(values) that returns a new list where element i is the maximum of values[0] through values[i]. It must work for negative numbers, and return [] for an empty list.

>>> running_max([-5, -2, -8, -1])
[-5, -2, -2, -1]
>>> running_max([3, 1, 4, 1, 5])
[3, 3, 4, 4, 5]
>>> running_max([])
[]

Think about what best should start as so that the first value always becomes the best so far.

File: solution.py

Starting point

def running_max(values):
    result = []
    best = 0
    for v in values:
        if v > best:
            best = v
        result.append(best)
    return result

Hint 1

Starting best at 0 assumes every value is at least 0.

Hint 2

Options: start with best = None and treat None as "nothing seen yet", or start with float("-inf"), which is smaller than every number.

Reference solution

def running_max(values):
    """Return the running maximum of values as a new list."""
    result = []
    best = None
    for v in values:
        if best is None or v > best:
            best = v
        result.append(best)
    return result