Building a command-line tool with argparse

Why this matters

Much of the useful software you will write is small tools run from a terminal, scripts, cron jobs or other programs: "rename these files", "summarize this log", "convert this CSV". A good command-line interface (CLI) behaves like the Unix tools around it: it takes arguments and options, prints help, reports usage errors clearly, and signals success or failure through its exit status. Python's standard argparse module gives you all of that with a few declarations.

Where arguments come from: sys.argv

When you run python3 tool.py report.txt --verbose, the shell splits the command line into words and the operating system passes them to the new process. Python puts them in sys.argv:

import sys
print(sys.argv)

$ python3 tool.py report.txt --verbose
['tool.py', 'report.txt', '--verbose']

sys.argv[0] is the script name; the rest are strings, exactly as the shell produced them (after its quote handling: "Grace Hopper" arrives as one item). Parsing them by hand works for one argument, but soon you want optional flags in any order, defaults, type checking, help text and error messages. That is argparse's job.

A first parser

import argparse

def main():
    parser = argparse.ArgumentParser(description="Count lines in a file.")
    parser.add_argument("path", help="file to read")
    parser.add_argument("-v", "--verbose", action="store_true", help="print details")
    args = parser.parse_args()

    with open(args.path) as f:
        n = sum(1 for _ in f)
    if args.verbose:
        print(f"{args.path}: {n} lines")
    else:
        print(n)

if __name__ == "__main__":
    main()

Step by step:

• ArgumentParser(description=...) creates a parser; the description appears in the help.
• add_argument("path"): a name without dashes declares a positional argument, required and identified by its position.
• add_argument("-v", "--verbose", ...): names with dashes declare an option; it may appear anywhere, or not at all. -v is the short form, --verbose the long form, and the attribute name comes from the long form (args.verbose; dashes inside become underscores: --dry-run gives args.dry_run).
• action="store_true" makes it a flag: no value follows it; the attribute is True if present, False otherwise.
• parse_args() reads sys.argv[1:], validates everything, and returns a Namespace object whose attributes hold the values.

You get help for free:

$ python3 count.py --help
usage: count.py [-h] [-v] path

Count lines in a file.

positional arguments:
  path           file to read

options:
  -h, --help     show this help message and exit
  -v, --verbose  print details

Types, defaults, choices, nargs

Arguments arrive as strings. type= converts them, and a failed conversion becomes a usage error:

parser.add_argument("-n", "--lines", type=int, default=10, help="lines to show (default: 10)")

$ python3 head.py -n ten file.txt
usage: head.py [-h] [-n LINES] path
head.py: error: argument -n/--lines: invalid int value: 'ten'
$ echo $?
2

type can be any function that takes a string: int, float, pathlib.Path, or your own function that raises argparse.ArgumentTypeError (or ValueError) on bad input.

Other useful parameters:

An example combining several:

parser = argparse.ArgumentParser(prog="convert")
parser.add_argument("files", nargs="+", help="input files")
parser.add_argument("--format", choices=["json", "csv"], default="json")
parser.add_argument("-v", "--verbose", action="count", default=0)
args = parser.parse_args(["a.txt", "b.txt", "-vv", "--format", "csv"])
print(args)

Namespace(files=['a.txt', 'b.txt'], format='csv', verbose=2)

Passing a list to parse_args instead of reading sys.argv is handy for experimenting in the REPL and for testing your parser.

Exit statuses and where output goes

A CLI communicates through three channels:

• stdout: the actual results, so they can be piped into other tools.
• stderr: errors, warnings and progress messages, so they do not pollute piped results.
• exit status: 0 for success, nonzero for failure. By convention 1 means a general error and 2 means incorrect usage.

argparse follows these rules: usage errors go to stderr with status 2 (parser.error("message") lets you raise your own usage error the same way), and --help goes to stdout with status 0. For your own failures, such as a missing input file, print to stderr and exit with 1:

import sys

def main() -> int:
    ...
    try:
        text = open(args.path).read()
    except FileNotFoundError:
        print(f"count: {args.path}: no such file", file=sys.stderr)
        return 1
    ...
    return 0

if __name__ == "__main__":
    sys.exit(main())

Having main() return the status and calling sys.exit(main()) keeps the function testable: tests can call main() and check the return value without the process exiting.

Structure of a well-behaved tool

#!/usr/bin/env python3
"""wordfreq: print the most common words in text files."""
import argparse
import sys
from collections import Counter


def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(description=__doc__)
    p.add_argument("files", nargs="+")
    p.add_argument("-n", "--top", type=int, default=10)
    return p


def count_words(text: str) -> Counter[str]:
    return Counter(text.lower().split())


def main(argv: list[str] | None = None) -> int:
    args = build_parser().parse_args(argv)
    total: Counter[str] = Counter()
    for name in args.files:
        try:
            with open(name, encoding="utf-8") as f:
                total += count_words(f.read())
        except OSError as e:
            print(f"wordfreq: {e}", file=sys.stderr)
            return 1
    for word, n in total.most_common(args.top):
        print(f"{n:6} {word}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

The parser is built in its own function, the real work lives in small pure functions (count_words) that are easy to unit-test, and main(argv) accepts an optional argument list so tests can call main(["file.txt", "-n", "3"]). The #!/usr/bin/env python3 line plus chmod +x wordfreq.py lets you run it as ./wordfreq.py.

Common mistakes

• Forgetting type=int: args.times is the string "3", and range(args.times) raises TypeError: 'str' object cannot be interpreted as an integer.
• action="store_true" with a value: --shout yes makes yes a stray positional and gives error: unrecognized arguments: yes.
• Negative numbers as positionals: use -- before them when in doubt.
• Printing errors to stdout: they then end up in the output file of tool > out.txt. Use file=sys.stderr.
• Calling sys.exit deep inside helper functions: it makes them hard to test and reuse. Raise exceptions and decide the exit status in main.

Recap

• sys.argv holds the raw words; argparse turns them into a validated Namespace.
• Positionals have no dashes and are required; options have dashes and are optional; action="store_true" makes a flag.
• type, default, choices and nargs handle conversion and validation; failures exit with status 2 and a usage message on stderr.
• Results to stdout, errors to stderr, exit 0 on success, 1 on errors, 2 on usage errors.
• Structure tools as build_parser(), small pure functions, and main(argv) -> int called via sys.exit(main()).

Going further: read the argparse tutorial in the Python docs ("Argparse Tutorial" under HOWTOs), especially the section on mutually exclusive groups, and look at subcommands (add_subparsers), which the to-do project at the end of this module uses.