Modules, packages and the standard library · about 40 min

Writing your own modules and the main idiom

Why this matters

As soon as a program grows past a page or two, you want to split it into files: one for parsing input, one for the calculations, one for output. And you will often write a file that is useful both as a standalone program and as a library of functions for other code, including your tests. Python has one small idiom that makes this work, if __name__ == "__main__":, and you will see it at the bottom of almost every Python script. This lesson explains what it means and how to organize a module so importing it is safe.

Splitting a program into modules

Suppose a program reads scores and prints a summary. You can put the reusable logic in its own file:

# scoring.py
"""Functions for summarizing test scores."""

PASS_MARK = 50


def passed(scores):
    """Return the scores that are at or above the pass mark."""
    return [s for s in scores if s >= PASS_MARK]


def summary(scores):
    """Return a one-line summary of a list of scores."""
    ok = passed(scores)
    return f"{len(ok)} of {len(scores)} passed, best {max(scores)}"

and keep the program itself small:

# main.py
import scoring

scores = [int(x) for x in input().split()]
print(scoring.summary(scores))

$ echo "40 75 50 90" | python3 main.py
3 of 4 passed, best 90

scoring.py begins with a module docstring, a string literal as the first statement. It becomes scoring.__doc__ and is what help(scoring) shows. Every function has one too. The module only defines things: one constant and two functions. Importing it has no visible effect, which is exactly what you want from a library.

Every module knows its name: name

When Python creates a module object, it sets a few special names in its namespace, called dunder names (for double underscore). The most useful is __name__:

For an imported module, __name__ is the module's name: "scoring".
For the file you run directly with python3 file.py, the module is named "__main__" instead.

# whoami.py
print("my __name__ is", repr(__name__))

$ python3 whoami.py
my __name__ is '__main__'
$ python3 -c "import whoami"
my __name__ is 'whoami'

Underneath, running python3 whoami.py makes the interpreter create a module called __main__, put it in sys.modules["__main__"], and execute whoami.py's code in it. Importing creates a module named after the file. Same code, same machinery, different name.

The idiom

Because __name__ tells you how the file is being used, you can make part of it run only when it is the main program:

# scoring.py
"""Functions for summarizing test scores."""

PASS_MARK = 50


def passed(scores):
    return [s for s in scores if s >= PASS_MARK]


def summary(scores):
    ok = passed(scores)
    return f"{len(ok)} of {len(scores)} passed, best {max(scores)}"


def main():
    scores = [int(x) for x in input().split()]
    print(summary(scores))


if __name__ == "__main__":
    main()

Now the file works both ways:

$ echo "40 75 50 90" | python3 scoring.py
3 of 4 passed, best 90
$ python3 -c "import scoring; print(scoring.passed([10, 60]))"
[60]

When imported, the final if is false, so main() is never called: no input(), no output. When run, the if is true and the program starts.

Why put the program in a main() function rather than directly under the if? Three reasons:

Local variables. Code at module level creates global names (scores would become scoring.scores), which can accidentally be used by other functions. Inside main(), they are local and disappear when it returns.
Testability. A test can call main() itself, with stdin replaced, without starting a separate process.
Readability. The reader sees "definitions first, then one line that starts everything" and knows where the program begins.

A common extension is to have main return an exit status, sys.exit(main()), which you met with exit codes in the previous module.

What runs at import time

Remember that importing a module runs every top-level statement in it, once. So think about each top-level line as "something that happens to anyone who imports me":

At top level	Effect when imported	OK?
`def f(): ...`	creates a function object, binds `f`	yes
`class C: ...`	creates a class	yes
`TIMEOUT = 30`	binds a constant	yes
`import json`	imports (or reuses) json, binds `json`	yes
`print("loaded")`	prints, in every importer, every test run	no
`data = open("big.csv").read()`	reads a file that may not exist where the importer runs	no
`name = input()`	blocks, waiting for a user who is not there	no

Code that does things belongs in functions. The rule of thumb: importing a module should define names and nothing else. Tools depend on this: test runners import your files to find tests, documentation generators import them to read docstrings, and editors import them for autocompletion.

How the pieces of a project fit together

A typical small project:

wordtool/
  wordtool.py      the command-line program: parses arguments, calls the library
  textstats.py     pure functions: count_words(text), top_words(text, n)
  test_textstats.py   tests that import textstats

textstats.py has no input() or print(): it takes data as arguments and returns results. wordtool.py handles the outside world (stdin, files, output). This split, often called "functional core, imperative shell", makes the logic easy to test and reuse. You will lean on it in the testing module.

Common mistakes

Typos in the guard. if __name__ == "main": or if name == "__main__": never runs (or raises NameError). It must be __name__ (two underscores each side) compared with "__main__".
Forgetting to call main(). Defining def main(): without the guard and the call means the script silently does nothing when run.
Running the guard's code at import because it is not indented. Only the lines indented under the if are guarded.
Hidden dependence on globals. Code like def summary(): return len(scores) that relies on a module-level scores works when run as a script and fails with NameError when imported. Pass data in as arguments.
Two copies of one module. If scoring.py is run as the main program and imported by another of your modules, it exists twice: once as __main__ and once as scoring, each with its own globals. Keep entry-point scripts thin so this does not matter.

Recap

Your own modules are just .py files next to your script; give them a docstring and keep them focused.
__name__ is the module's name when imported, and "__main__" for the file being run.
if __name__ == "__main__": main() lets one file be both a library and a program.
Importing should only define names; put actions (input, output, file access) inside functions.

Going further: run python3 -c "import json; print(json.__name__, json.__file__)", then open that file and look at the bottom of json/tool.py in the same directory for a standard-library example of the idiom.

Exercises

Exercise 1 · quiz

What is name?

tools.py contains the single line print(__name__). app.py contains import tools.

What is printed by python3 app.py, and what by python3 tools.py?

app.py prints tools; tools.py prints tools
app.py prints __main__; tools.py prints tools
app.py prints tools; tools.py prints __main__
app.py prints app; tools.py prints __main__

Answer

app.py prints tools; tools.py prints __main__

When a file is imported, its module's __name__ is the module name (tools). When a file is run as the program, Python executes it as a module named __main__. That difference is exactly what if __name__ == "__main__": tests.

Exercise 2 · write code

A module that is also a script

Write temperature.py, usable both as a library and as a program.

As a library it must provide two functions (no printing, no input at import time):

c_to_f(c) returns c * 9 / 5 + 32
f_to_c(f) returns (f - 32) * 5 / 9

As a program (python3 temperature.py) it reads one line like 100 C or 212 F (a number, a space, then C or F), converts it to the other scale, and prints the result rounded to one decimal place followed by the other unit:

212.0 F

Put the program part in a function main() and call it only under if __name__ == "__main__":. The checker also imports your module with no input available, so import temperature must not call input().

File: temperature.py

Starting point

def c_to_f(c):
    return c * 9 / 5 + 32


def f_to_c(f):
    return (f - 32) * 5 / 9


value, unit = input().split()
value = float(value)
if unit == "C":
    print(round(c_to_f(value), 1), "F")
else:
    print(round(f_to_c(value), 1), "C")

Example

input

100 C

expected output

212.0 F

Example

input

98.6 F

expected output

37.0 C

Hint 1

Move the input/print code into def main(): (indent it).

Hint 2

At the very bottom: if __name__ == "__main__": and, indented under it, main().

Reference solution

"""Temperature conversion between Celsius and Fahrenheit."""


def c_to_f(c):
    """Convert degrees Celsius to Fahrenheit."""
    return c * 9 / 5 + 32


def f_to_c(f):
    """Convert degrees Fahrenheit to Celsius."""
    return (f - 32) * 5 / 9


def main():
    value, unit = input().split()
    value = float(value)
    if unit == "C":
        print(round(c_to_f(value), 1), "F")
    else:
        print(round(f_to_c(value), 1), "C")


if __name__ == "__main__":
    main()

Exercise 3 · quiz

Import side effects

Which of these top-level statements in a module meant to be imported are reasonable side effects at import time? Select all that apply.

Defining functions and classes with def and class
Assigning constants such as MAX_RETRIES = 3
Calling input() to ask the user for a file name
Deleting files in the current directory
Importing other modules it needs

Answer

Defining functions and classes with def and class
Assigning constants such as MAX_RETRIES = 3
Importing other modules it needs

Importing should be cheap and safe: it should only define things. def, class, constants and imports just create names. Asking for input, printing, writing or deleting files, or starting long computations at import time surprises every importer (including test runners and tools that import modules just to read their docstrings). Put those actions in functions and call them from main() under the __main__ guard.

Exercise 4 · write code

Make a script importable

stats.py was written as a quick script: it reads numbers from stdin and prints their mean. A colleague now wants to reuse its mean function from report.py (already in your working directory), which contains:

from stats import mean
print("average of 2, 4, 9:", mean([2, 4, 9]))

Running python3 report.py currently breaks, because importing stats runs the script part. Restructure stats.py so that:

python3 stats.py still works exactly as before: it reads one line of space-separated numbers and prints mean: <value> (the value printed as a float, as print shows it);
python3 report.py prints exactly average of 2, 4, 9: 5.0 and nothing else.

Keep mean unchanged.

File: stats.py

Starting point

def mean(numbers):
    return sum(numbers) / len(numbers)


print("stats module loaded")
line = input()
numbers = [float(x) for x in line.split()]
print("mean:", mean(numbers))

Example

input

1 2 3 4

expected output

mean: 2.5

Hint 1

The stray print("stats module loaded") is also output at import time; does anyone need it?

Hint 2

Move the reading and printing into main() and guard the call with if __name__ == "__main__":.

Reference solution

def mean(numbers):
    return sum(numbers) / len(numbers)


def main():
    line = input()
    numbers = [float(x) for x in line.split()]
    print("mean:", mean(numbers))


if __name__ == "__main__":
    main()

Writing your own modules and the __main__ idiom

Why this matters

Splitting a program into modules

Every module knows its name: name

The idiom

What runs at import time

How the pieces of a project fit together

Common mistakes

Recap

Exercises

What is __name__?

A module that is also a script

Import side effects

Make a script importable

Writing your own modules and the main idiom

What is name?