Modules, packages and the standard library · about 40 min

Modules and how import works

Why this matters

Every useful Python program leans on code written by someone else: the standard library, third-party packages, and other files of your own project. The import statement is the doorway to all of it. Knowing exactly what import does (where it looks, what it creates, and that it runs a file only once) explains many puzzling situations: why your file named random.py broke import random, why a change to a module does not show up in a running REPL, and why from x import y sometimes seems to "not update".

A module is a file, and a module object

A module is simply a .py file. If you save this as greet.py:

"""Greeting helpers."""

DEFAULT_NAME = "world"

def hello(name=DEFAULT_NAME):
    return f"Hello, {name}!"

print("greet.py is running")

then in another file in the same directory you can write:

import greet

print(greet.hello())
print(greet.hello("Ada"))
print(greet.DEFAULT_NAME)
greet.py is running
Hello, world!
Hello, Ada!
world

What did import greet actually do? Three things:

  1. Find a module called greet (more on where it looks below).
  2. Create a module object and execute the file's code from top to bottom inside that module object's namespace. Every top-level def, assignment and import in greet.py creates a name in that namespace. That is why greet.py is running was printed: top-level code runs at import time.
  3. Bind the name greet in the importing file to the module object.

A module object is an ordinary object, and its namespace is (underneath) a dictionary:

import greet
print(type(greet))
print(greet.__name__)
print(sorted(k for k in vars(greet) if not k.startswith("__")))
greet.py is running
<class 'module'>
greet
['DEFAULT_NAME', 'hello']

vars(greet) returns the module's __dict__, the dictionary that maps names to objects. greet.hello is just a lookup of the key "hello" in that dictionary. Seen this way, a module is a namespace: a container that keeps its names separate from yours, so greet.hello and some other module's hello never collide.

The module cache: sys.modules

Importing is expensive (find the file, compile it to bytecode, run it), so Python does it once per process. After the first import, the module object is stored in the dictionary sys.modules, keyed by name. Every later import greet anywhere in the program checks sys.modules first and, finding it, just binds the name to the existing object.

import greet
     |
     v
 "greet" in sys.modules? --yes--> bind name to that object (no code runs)
     |
     no
     v
 search sys.path for greet  -->  create module  -->  run greet.py in it
     -->  store in sys.modules["greet"]  -->  bind name

Consequences:

Some modules are already in sys.modules before your first line runs, because the interpreter itself needed them during startup (sys, os, builtins and a few others). Most of the standard library is loaded only on demand.

Where Python looks: sys.path

To find a module that is not cached, Python searches a list of directories, sys.path, in order, and takes the first match:

import sys
for p in sys.path:
    print(p)
/home/ada/project
/usr/lib/python313.zip
/usr/lib/python3.13
/usr/lib/python3.13/lib-dynload
/usr/local/lib/python3.13/dist-packages
/usr/lib/python3/dist-packages

A few modules (like sys, math and builtins) are compiled directly into the interpreter and are found before the path is searched at all; sys.builtin_module_names lists them.

Shadowing: the classic trap

Because your script's directory comes first, a file of yours with the same name as a standard module hides it:

project/
  random.py      <- you wrote this to experiment with random numbers
  main.py        <- contains: import random; print(random.randint(1, 6))
Traceback (most recent call last):
  File "/home/ada/project/main.py", line 2, in <module>
    print(random.randint(1, 6))
          ^^^^^^^^^^^^^^
AttributeError: module 'random' has no attribute 'randint' (consider renaming '/home/ada/project/random.py' since it has the same name as the standard library module named 'random' and prevents importing that standard library module)

Python 3.13 now spells out the cause, but older versions just say has no attribute. Never name your files after modules you use: random.py, json.py, email.py, test.py, string.py, csv.py. Also delete the stale __pycache__/random.cpython-313.pyc after renaming.

The forms of import

import math                    # name math -> module; use math.sqrt(2)
import collections.abc         # submodule; use collections.abc.Mapping
import numpy as np             # bind the module under a different name
from math import sqrt, pi      # bind sqrt and pi directly in your namespace
from math import sqrt as root  # under a different name
from math import *             # bind every public name (avoid this)

import math binds one name, math, to the module. You then write math.sqrt, which keeps it obvious where sqrt comes from.

from math import sqrt still imports (and caches) the whole math module; it then looks up sqrt in it and binds your name sqrt to that same function object. There is no "partial import" that saves loading time.

That last point has a subtle consequence that the third exercise explores. After from counter import count, your count is a separate name that refers to whatever object counter.count referred to at that moment. If the counter module later rebinds its own count, your name does not follow; names are labels, and you only copied the label's target, not a link to the other label. Reading through the module, counter.count, always gets the current value.

from module import * dumps every public name into your namespace. It makes it impossible to tell where a name came from and can silently overwrite your own names. Use it only in the REPL, if at all.

Style conventions (PEP 8): imports go at the top of the file, one module per line, grouped as standard library, then third-party, then your own modules, with a blank line between groups.

Common mistakes

Recap

Going further: run python3 -v -c "import json" 2>&1 | grep json to watch the import system search and load files, and read the "The import system" chapter of the Python Language Reference.

Exercises

Exercise 1 · quiz

How many times does a module run?

greet.py contains:

print("loading greet")
def hello():
    return "hi"

main.py contains:

import greet
import greet
from greet import hello
print(hello())

How many times is loading greet printed when you run python3 main.py?

Answer
  • 1

The first import greet finds greet.py, creates a module object, runs the file's code once to fill it, and stores it in sys.modules["greet"]. Every later import of greet in the same process (including from greet import hello) finds it in that cache and reuses the same object without running the file again.

Exercise 2 · write code

Use a module you did not write

The file geometry.py (already in your working directory, open it with the Files view or just trust this description) defines:

Write main.py so that it reads two lines from stdin, a width and a height (whole numbers), and prints:

rectangle: <rect_area result>
circle: <circle_area of the width, rounded to 2 decimals with round(..., 2)>

For input 3 and 4 the output is:

rectangle: 12
circle: 28.27

Requirements: use import geometry for rect_area (call it as geometry.rect_area), and from geometry import circle_area for the other. Do not copy the functions into main.py.

File: main.py

Starting point

w = int(input())
h = int(input())

Example

input

3
4

expected output

rectangle: 12
circle: 28.27

Example

input

1
10

expected output

rectangle: 10
circle: 3.14
Hint 1

Put both import lines at the top of the file.

Hint 2

After import geometry, the function is reached through the module: geometry.rect_area(w, h).

Hint 3

After from geometry import circle_area, the bare name circle_area works.

Reference solution
import geometry
from geometry import circle_area

w = int(input())
h = int(input())
print("rectangle:", geometry.rect_area(w, h))
print("circle:", round(circle_area(w), 2))

Exercise 3 · quiz

from-import binds a name, not a link

counter.py:

count = 0

def bump():
    global count
    count += 1

main.py:

from counter import count, bump
bump()
bump()
print(count)

What does python3 main.py print?

Answer
  • 0

from counter import count creates a new name count in main's namespace bound to the object that counter.count referred to at that moment: the integer 0. bump() rebinds the name count inside the counter module to 1, then 2, but that does not touch main's separate name, which still refers to 0. With import counter and print(counter.count) you look the attribute up on the module each time and see 2.

Exercise 4 · write code

Watch the module cache

Write a program that shows the import cache in action. It must print exactly three lines:

before: False
after: True
same object: True
  1. The first line says whether the key "json" is in sys.modules before you import json.
  2. Then import json and print whether it is in sys.modules now.
  3. Finally, check that sys.modules["json"] is the very same object as the json name you just imported (use is).

You need import sys first. Do not import json before step 1.

File: main.py

Starting point

import sys
import json
print("before:", "json" in sys.modules)

Example

expected output

before: False
after: True
same object: True
Hint 1

"json" in sys.modules is a dictionary membership test.

Hint 2

An import statement can appear anywhere, not just at the top of the file; here it must come after the first print.

Reference solution
import sys

print("before:", "json" in sys.modules)
import json
print("after:", "json" in sys.modules)
print("same object:", sys.modules["json"] is json)