Errors, exceptions and files · about 35 min

Reading tracebacks and the exception hierarchy

Why this matters

Every program you write will crash, many times, while you develop it. The crash report Python prints, the traceback, is the single most useful debugging tool you have, and it is free. Programmers who read tracebacks carefully fix bugs in minutes; those who skim them guess for an hour. This lesson teaches you to read one line by line, explains what an exception is (an object, with a type that sits in a family tree), and introduces the common exception types you will meet daily. The next lessons build on this to handle and raise exceptions yourself.

What an exception is

When the Python virtual machine hits an operation it cannot perform, such as dividing by zero or looking up a missing dict key, it does not just stop. It creates an exception object describing the problem and starts unwinding: it abandons the current function and looks in the function that called it for code that handles this kind of exception, then in that function's caller, and so on, up the call stack. If nothing handles it all the way up to the top level of the script, Python prints a traceback to stderr and exits with status 1.

call stack while running                 unwinding after the error

  <module>  line 12: report(data)    ▲   no handler: print traceback, exit 1
  report    line 8:  average(values) ▲   no handler here either
  average   line 3:  sum/len  ──X    ▲   ZeroDivisionError raised here

Each box is a frame: the record CPython keeps for one active function call, holding its local variables and the line it is executing. The traceback is a printout of those frames at the moment of the error.

Anatomy of a traceback

Here is a program with a bug:

def average(nums):
    return sum(nums) / len(nums)

def report(data):
    values = [row["score"] for row in data if row["score"] is not None]
    print("average:", average(values))

report([{"score": None}])

Running it prints:

Traceback (most recent call last):
  File "/work/main.py", line 8, in <module>
    report([{"score": None}])
    ~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/work/main.py", line 6, in report
    print("average:", average(values))
                      ~~~~~~~^^^^^^^^
  File "/work/main.py", line 2, in average
    return sum(nums) / len(nums)
           ~~~~~~~~~~^~~~~~~~~~~
ZeroDivisionError: division by zero

Read it from the bottom up:

The last line is the exception type and message: ZeroDivisionError: division by zero. This tells you what went wrong.
The frame just above is where it happened: file, line 2, inside function average, with the source line and markers (~~~^~~~) pointing at the exact operation, the /.
The frames above that show how you got there, "most recent call last": the top-level code (<module>) on line 8 called report, which on line 6 called average.

Then think one step further: the division failed because len(nums) was 0, because values was empty, because every score was None. The traceback shows where the program noticed the problem; the cause is often a frame or two higher. Here a fix belongs in report or average (decide what the average of nothing should be).

Frames from library code appear too. If a traceback is twenty frames long, find the lowest frame that is in your file; that is usually where to start looking.

Exceptions are objects with types

The exception is an ordinary Python object. Its class (type) says what kind of problem it is, and it carries a message and other data. You can create one without raising it:

>>> err = ValueError("bad temperature")
>>> type(err)
<class 'ValueError'>
>>> err.args
('bad temperature',)
>>> str(err)
'bad temperature'
>>> type(err).__name__
'ValueError'

The next lesson catches exceptions with except SomeError as e:, giving you the object in e.

The exception hierarchy

Exception classes form a family tree through inheritance: a more specific class is a subclass of a more general one (classes and inheritance are module 12; here you only need the idea that "a KeyError is a kind of LookupError"). The top of the built-in tree, abridged:

BaseException
 ├── SystemExit                 raised by sys.exit()
 ├── KeyboardInterrupt          the user pressed Ctrl-C
 ├── GeneratorExit
 └── Exception                  everything an ordinary program should handle
      ├── ArithmeticError
      │    ├── ZeroDivisionError
      │    └── OverflowError
      ├── LookupError
      │    ├── IndexError       sequence index out of range
      │    └── KeyError         missing dict key
      ├── OSError               operating-system errors
      │    ├── FileNotFoundError
      │    ├── PermissionError
      │    └── IsADirectoryError
      ├── TypeError             wrong type for an operation
      ├── ValueError            right type, unacceptable value
      │    └── UnicodeError
      ├── NameError             unknown name
      ├── AttributeError        object has no such attribute
      ├── ImportError
      │    └── ModuleNotFoundError
      ├── RuntimeError
      │    └── RecursionError
      └── StopIteration

Why does the tree matter? Because handlers match by family: a handler for LookupError also catches KeyError and IndexError; a handler for OSError catches every file-system error; a handler for Exception catches nearly everything. You can ask the tree directly:

>>> issubclass(KeyError, LookupError)
True
>>> isinstance(KeyError("x"), Exception)
True
>>> KeyError.__mro__
(<class 'KeyError'>, <class 'LookupError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>)

__mro__ ("method resolution order") lists a class and all its ancestors, nearest first.

Note the deliberate split at the top: SystemExit and KeyboardInterrupt are not under Exception. They are not errors in your logic; they are requests to stop. Code that catches Exception therefore still lets the user quit with Ctrl-C.

The everyday exceptions, with real messages

Exception	Typical cause	Message you will see
`NameError`	typo or undefined variable	`name 'totl' is not defined. Did you mean: 'total'?`
`TypeError`	operation on the wrong type, wrong number of arguments	`can only concatenate str (not "int") to str`; `greet() takes 1 positional argument but 2 were given`
`ValueError`	right type, bad value	`invalid literal for int() with base 10: 'abc'`
`KeyError`	missing dict key	`KeyError: 'email'` (the message is just the key)
`IndexError`	index past the end	`list index out of range`
`AttributeError`	no such method or attribute, often on `None`	`'NoneType' object has no attribute 'split'`
`ZeroDivisionError`	division or modulo by zero	`division by zero`
`FileNotFoundError`	opening a missing file	`[Errno 2] No such file or directory: 'data.csv'`
`RecursionError`	recursion without a base case	`maximum recursion depth exceeded`

Modern CPython adds suggestions ("Did you mean ...?") for NameError and AttributeError, and the ^^^ markers that pinpoint the failing sub-expression. Read them; they are often exactly right.

A special note on AttributeError: 'NoneType' object has no attribute ...: it nearly always means a variable you expected to hold an object holds None, often because a function forgot to return, or you used the result of an in-place method like list.sort().

SyntaxError is different

A SyntaxError (and its subclass IndentationError) is raised while compiling the file, before any of it runs (recall lesson 2 of the foundations module). Its report has no "Traceback (most recent call last)" chain of frames, because no functions were running yet: just the file, the line, a caret, and the message.

Common mistakes

Reading only the top. The first lines show where the program started, not where it failed. Start at the bottom.
Ignoring the message. KeyError: 'Email' with a capital E is telling you the key's exact spelling. The message usually contains the bad value.
Fixing the symptom. Wrapping the failing line in "make the error go away" code without asking why the value was bad hides bugs. Trace the bad value upward through the frames.
Assuming the reported line is wrong. For NameError or AttributeError on None, the real mistake is typically where the value was (not) created, earlier.

Recap

An exception is an object raised when an operation fails; Python unwinds the call stack looking for a handler and, finding none, prints a traceback and exits with status 1.
Read tracebacks bottom-up: the type and message, then the failing frame, then the chain of calls that led there.
Exception classes form a hierarchy; handlers for a class also catch its subclasses. Exception is the root for ordinary errors; SystemExit and KeyboardInterrupt sit outside it.
Know the common types: NameError, TypeError, ValueError, KeyError, IndexError, AttributeError, ZeroDivisionError, FileNotFoundError.

Going further: read "Exception hierarchy" at the end of the "Built-in Exceptions" page of the Python library documentation for the full tree. Then deliberately cause each exception in the table above in the REPL and read its message and markers.

Exercises

Exercise 1 · quiz

Where did it fail?

A program prints this traceback:

Traceback (most recent call last):
  File "/work/main.py", line 12, in <module>
    report(data)
  File "/work/main.py", line 8, in report
    print(average(values))
  File "/work/main.py", line 3, in average
    return sum(nums) / len(nums)
           ~~~~~~~~~~^~~~~~~~~~~
ZeroDivisionError: division by zero

In which function did the error actually occur?

<module> (the top level of the script)
report
average
print

Answer

average

"Most recent call last": the bottom frame is where the exception was raised, here line 3 in average. The frames above it show how execution got there: the module called report on line 12, which called average on line 8. The root cause may be higher up (why was the list empty?), but the failure itself happened in average.

Exercise 2 · quiz

Family tree

Which of these are subclasses of LookupError, meaning except LookupError: would catch them? Select all that apply.

Answer

KeyError
IndexError

LookupError is the common base for "the key or index you asked for is not there": KeyError (dicts) and IndexError (sequences). ValueError and TypeError sit directly under Exception. FileNotFoundError is under OSError. You can check any of these yourself with issubclass(KeyError, LookupError).

Exercise 3 · write code

Provoke and identify

The starter defines six small functions, each of which raises a different built-in exception when called. Using try/except Exception as e: (shown in the starter for the first one), call each function in order and print the name of the exception class it raises, one per line, using type(e).__name__.

The output must be exactly:

ZeroDivisionError
KeyError
IndexError
TypeError
ValueError
AttributeError

Do not change the six functions. Fill in the missing calls.

File: main.py

Starting point

def a():
    return 1 / 0

def b():
    return {"x": 1}["y"]

def c():
    return [1, 2, 3][10]

def d():
    return "age: " + 30

def e():
    return int("thirty")

def f():
    return (5).append(6)

try:
    a()
except Exception as err:
    print(type(err).__name__)

Example

expected output

ZeroDivisionError
KeyError
IndexError
TypeError
ValueError
AttributeError

Hint 1

Repeat the try/except block for each function, or loop over the functions: for func in (a, b, c, d, e, f):.

Hint 2

Functions are objects, so func() calls whichever function the loop variable refers to.

Reference solution

def a():
    return 1 / 0

def b():
    return {"x": 1}["y"]

def c():
    return [1, 2, 3][10]

def d():
    return "age: " + 30

def e():
    return int("thirty")

def f():
    return (5).append(6)

for func in (a, b, c, d, e, f):
    try:
        func()
    except Exception as err:
        print(type(err).__name__)

Exercise 4 · write code

Climb the hierarchy

Every exception class knows its ancestors. cls.__mro__ is a tuple of the class followed by its base classes, up to object; cls.__name__ is its name.

Read the name of a built-in exception class from input (for example FileNotFoundError), look it up with getattr(builtins, name) (after import builtins), and print its chain of class names joined by ->, stopping before object. For FileNotFoundError the output is:

FileNotFoundError -> OSError -> Exception -> BaseException

File: main.py

Starting point

import builtins

name = input()
cls = getattr(builtins, name)
print(cls.__name__)

Example

input

FileNotFoundError

expected output

FileNotFoundError -> OSError -> Exception -> BaseException

Example

input

KeyError

expected output

KeyError -> LookupError -> Exception -> BaseException

Hint 1

cls.__mro__ for KeyError is (KeyError, LookupError, Exception, BaseException, object).

Hint 2

Collect c.__name__ for each class except object, then " -> ".join(...).

Reference solution

import builtins

name = input()
cls = getattr(builtins, name)
names = [c.__name__ for c in cls.__mro__ if c is not object]
print(" -> ".join(names))