Practical projects · about 75 min

Project: a to-do manager with subcommands and JSON storage

Why this matters

Tools like git, docker and pip are not one command but a family: git commit, git log, git push. Each subcommand has its own arguments, and all of them share state that persists between runs, such as a repository, an image store or an installed-package list. This final project builds a small tool in that style: a to-do manager whose data survives from one run to the next in a JSON file. Along the way you will design a data format, write files safely so a crash cannot destroy data, structure a CLI with argparse subparsers, and test a program across many invocations.

Designing the data

Before code, decide what lives on disk. A JSON document is a good fit for small personal data: human-readable, diffable, and the standard library reads and writes it.

{
  "next_id": 4,
  "tasks": [
    {"done": true, "id": 1, "text": "buy milk"},
    {"done": false, "id": 3, "text": "write report"}
  ]
}

Design decisions, each with a reason:

Every task has a stable integer id. Users refer to tasks by id (done 3). The id must not change when other tasks are removed, so it is stored, not computed from the position in the list.
next_id is stored too. Using len(tasks) + 1 would reuse ids after a removal. A counter that only grows makes ids unique for the life of the store.
A top-level object, not a bare list. An object leaves room to add fields later ("version": 2, settings) without breaking old files. Formats that will live for years benefit from a version number from the start.
sort_keys=True, indent=2 when saving, so the file is stable and easy to read or edit by hand.

Persisting state safely

The naive save is:

with open(path, "w") as f:
    json.dump(store, f)

Opening with "w" truncates the file immediately. If the program crashes, is killed, or the disk fills up before json.dump finishes, the file is left empty or half-written, and the next load fails with JSONDecodeError. All tasks are gone.

The standard remedy is write to a temporary file, then rename:

import json
import os

def save(path: str, store: dict) -> None:
    tmp = path + ".tmp"
    with open(tmp, "w", encoding="utf-8") as f:
        json.dump(store, f, indent=2, sort_keys=True)
        f.write("\n")
    os.replace(tmp, path)

os.replace renames tmp over path. On POSIX systems a rename within one file system is atomic: at every instant path is either the complete old file or the complete new one. If anything fails while writing the temp file, the real file is untouched. (For protection against power loss as well, programs also call f.flush() and os.fsync(f.fileno()) before closing; databases and editors do this.) os.replace overwrites an existing target on every platform, unlike os.rename, which fails on Windows if the target exists.

Loading treats a missing file as an empty store, so the first command just works:

def load(path: str) -> dict:
    try:
        with open(path, encoding="utf-8") as f:
            return json.load(f)
    except FileNotFoundError:
        return {"next_id": 1, "tasks": []}

Note what is not caught: a corrupt file raises JSONDecodeError. Silently returning an empty store there would make the next save overwrite the user's data. Failing loudly is the right default; a friendlier version would print error: todo.json is corrupt and exit 1 without touching the file.

Subcommands with argparse

add_subparsers creates a group of sub-parsers, one per command, each with its own arguments:

import argparse

parser = argparse.ArgumentParser(prog="todo")
parser.add_argument("--file", default="todo.json")
sub = parser.add_subparsers(dest="command", required=True)

p_add = sub.add_parser("add", help="add a task")
p_add.add_argument("words", nargs="+")

p_done = sub.add_parser("done", help="mark a task completed")
p_done.add_argument("id", type=int)

print(parser.parse_args(["add", "buy", "milk"]))
print(parser.parse_args(["--file", "w.json", "done", "3"]))

Namespace(file='todo.json', command='add', words=['buy', 'milk'])
Namespace(file='w.json', command='done', id=3)

dest="command" stores which subcommand was chosen.
required=True makes a missing subcommand a usage error (exit status 2) instead of a Namespace with command=None.
Options of the main parser (--file) go before the subcommand; each subcommand's arguments go after it.
Help is generated per level: todo --help lists the commands, todo done --help shows the arguments of done.

nargs="+" on add lets users type todo add buy milk without quotes; joining the words with spaces rebuilds the text.

Dispatching with set_defaults

A chain of if args.command == "add": ... elif ... grows with every command. Instead, attach the handler function to each sub-parser:

def cmd_add(store, args):
    ...

p_add.set_defaults(func=cmd_add)
p_done.set_defaults(func=cmd_done)

args = parser.parse_args()
status = args.func(store, args)

set_defaults(func=...) puts a func attribute on the Namespace when that subcommand is used. main becomes three lines: parse, load, dispatch. Adding a command means writing one function and registering one parser, and functions are first-class objects, so storing them in the Namespace is natural.

Handlers and exit statuses

Each handler takes the store and the parsed arguments, does its work, prints its result, and returns an exit status:

def cmd_done(store: dict, args) -> int:
    task = find_task(store, args.id)
    if task is None:
        print(f"error: no task {args.id}", file=sys.stderr)
        return 1
    task["done"] = True
    save(args.file, store)
    print(f"completed {task['id']}: {task['text']}")
    return 0

Errors go to stderr and return 1, and the store is saved only after a successful change, so a failed command never modifies the file. main returns the handler's status to sys.exit, so shell scripts can write todo done 3 && echo ok.

Testing a stateful program

A program whose output depends on earlier runs needs tests that run sequences. The check for the final exercise is a shell script that starts from an empty directory and runs command after command, comparing each stdout and exit status. In Python you would do the same with subprocess and a temporary directory:

import subprocess, sys, tempfile, unittest
from pathlib import Path

class TodoSession(unittest.TestCase):
    def run_todo(self, *args):
        return subprocess.run([sys.executable, str(self.script), "--file", str(self.store), *args],
                              capture_output=True, text=True)

    def setUp(self):
        self.tmp = tempfile.TemporaryDirectory()
        self.store = Path(self.tmp.name) / "todo.json"
        self.script = Path(__file__).with_name("todo.py")

    def tearDown(self):
        self.tmp.cleanup()

    def test_add_then_list(self):
        self.assertEqual(self.run_todo("add", "buy", "milk").stdout, "added 1: buy milk\n")
        self.assertEqual(self.run_todo("list").stdout, "[ ] 1 buy milk\n")

    def test_unknown_id(self):
        r = self.run_todo("done", "9")
        self.assertEqual(r.returncode, 1)
        self.assertIn("no task 9", r.stderr)

Each test gets a fresh temporary directory, so tests cannot affect each other, and the --file option, which exists partly for exactly this reason, points the tool at it. Being able to redirect where state lives is a design feature that makes programs testable.

Common mistakes

AttributeError: 'Namespace' object has no attribute 'func': a subcommand was registered without set_defaults(func=...), or the subparsers group is not required and no subcommand was given.
Putting --file after the subcommand: todo add x --file w.json fails with unrecognized arguments because --file belongs to the main parser. (You could add it to each sub-parser instead, if you prefer that style.)
Comparing ids of different types: args.id is a string unless you declare type=int, and "3" == 3 is False, so every lookup fails.
Saving after an error, or saving a store that failed to load: either can destroy data.
Removing from a list while iterating over it (for t in tasks: if ...: tasks.remove(t)) skips elements. Find the task first, then remove it, or build a new list.
Writing in place with "w" for important data: use a temporary file and os.replace.

Recap

Design the on-disk format first: stable ids, a stored next_id, a top-level object with room to grow, sorted and indented JSON.
Save atomically by writing a temp file and os.replace-ing it over the original; treat a missing file as empty, but never silently ignore a corrupt one.
add_subparsers(dest="command", required=True) gives git-style commands; set_defaults(func=handler) dispatches without an if/elif chain.
Handlers print results to stdout, errors to stderr, and return exit statuses that main passes to sys.exit.
Stateful programs are tested by running sequences of commands against a fresh, redirectable store.

Going further: add a due date per task (store ISO strings, parse with date.fromisoformat) and a list --overdue filter, then replace the JSON file with the standard library's sqlite3 module and notice how little of the CLI layer has to change.

Exercises

Exercise 1 · quiz

Choosing task ids

Why does the store keep a next_id counter instead of giving a new task the id len(tasks) + 1?

len() is slow on long lists
After a task is removed, len(tasks) + 1 can repeat an id that is still in use, so two tasks would share an id
JSON cannot store the result of len()
There is no difference; it is only a style choice

Answer

After a task is removed, len(tasks) + 1 can repeat an id that is still in use, so two tasks would share an id

With tasks 1, 2, 3, removing task 1 leaves two tasks, so len + 1 gives 3 again, a duplicate of the existing task 3. Ids must never be reused while anything might still refer to them (a user's notes, a script, another system). A monotonically increasing counter stored with the data guarantees uniqueness; databases do the same with auto-increment keys.

Exercise 2 · quiz

Why write to a temporary file?

save writes the JSON to todo.json.tmp and then calls os.replace("todo.json.tmp", "todo.json"). What problem does this avoid?

JSON files cannot be overwritten directly
If the program crashes or the disk fills up halfway through writing, todo.json would be left truncated and unreadable; with replace, the old file stays intact until a complete new one exists
It makes writing faster because temporary files are cached
It allows two programs to write todo.json at the same time without conflicts

Answer

If the program crashes or the disk fills up halfway through writing, todo.json would be left truncated and unreadable; with replace, the old file stays intact until a complete new one exists

Opening a file with mode "w" truncates it immediately. A crash between the truncation and the end of the write loses all your data. Writing a complete new file next to it and then renaming is atomic on POSIX file systems: other readers see either the old file or the new one, never half of one. It does not, by itself, make concurrent writers safe (the last one wins); that would need file locking.

Exercise 3 · write code

Step 1: load and save the store

In solution.py, implement the storage layer:

load(path) -> dict: if the file does not exist, return a fresh store {"next_id": 1, "tasks": []}. Otherwise return the parsed JSON.
save(path, store) -> None: write the store as JSON with indent=2 and sort_keys=True to path + ".tmp", then move it into place with os.replace(tmp, path). No .tmp file may be left behind.
add_task(store, text) -> dict: append {"id": <next_id>, "text": text, "done": False} to store["tasks"], increment store["next_id"], and return the new task. (It changes the store in memory only.)

path is a string.

File: solution.py

Starting point

import json
import os


def load(path: str) -> dict:
    with open(path) as f:
        return json.load(f)


def save(path: str, store: dict) -> None:
    pass


def add_task(store: dict, text: str) -> dict:
    pass

Hint 1

Catch FileNotFoundError in load and return the fresh store.

Hint 2

In save, the with block must end (closing and flushing the temp file) before os.replace.

Hint 3

add_task reads store["next_id"] for the new id, then adds 1 to it.

Reference solution

import json
import os


def load(path: str) -> dict:
    try:
        with open(path, encoding="utf-8") as f:
            return json.load(f)
    except FileNotFoundError:
        return {"next_id": 1, "tasks": []}


def save(path: str, store: dict) -> None:
    tmp = path + ".tmp"
    with open(tmp, "w", encoding="utf-8") as f:
        json.dump(store, f, indent=2, sort_keys=True)
        f.write("\n")
    os.replace(tmp, path)


def add_task(store: dict, text: str) -> dict:
    task = {"id": store["next_id"], "text": text, "done": False}
    store["tasks"].append(task)
    store["next_id"] += 1
    return task

Exercise 4 · write code

Step 2: the to-do command

Build the complete tool in main.py with argparse subcommands. A global option --file PATH (default todo.json) selects the store, and must come before the subcommand. The storage functions from step 1 are in the starter.

Command	Output on success
`add WORDS...`	`added <id>: <text>` (the words joined with single spaces)
`list`	one line per pending task: `[ ] <id> <text>`; if none, `no tasks`
`list --all`	every task in id order, done ones as `[x] <id> <text>`; if none, `no tasks`
`done ID`	`completed <id>: <text>`
`remove ID`	`removed <id>: <text>`

Every command that changes the store saves it.
For done/remove with an id that does not exist, print error: no task <id> to stderr and exit with status 1 (do not save).
ID is an int (type=int); running with no subcommand must be a usage error (exit status 2): use add_subparsers(dest="command", required=True).

The check runs a whole session of commands in a fresh directory, for example:

$ python3 main.py add buy milk
added 1: buy milk
$ python3 main.py add call bob
added 2: call bob
$ python3 main.py done 1
completed 1: buy milk
$ python3 main.py list
[ ] 2 call bob
$ python3 main.py list --all
[x] 1 buy milk
[ ] 2 call bob

File: main.py

Starting point

import argparse
import json
import os
import sys


def load(path: str) -> dict:
    try:
        with open(path, encoding="utf-8") as f:
            return json.load(f)
    except FileNotFoundError:
        return {"next_id": 1, "tasks": []}


def save(path: str, store: dict) -> None:
    tmp = path + ".tmp"
    with open(tmp, "w", encoding="utf-8") as f:
        json.dump(store, f, indent=2, sort_keys=True)
        f.write("\n")
    os.replace(tmp, path)


def add_task(store: dict, text: str) -> dict:
    task = {"id": store["next_id"], "text": text, "done": False}
    store["tasks"].append(task)
    store["next_id"] += 1
    return task


def main() -> int:
    parser = argparse.ArgumentParser(description="A tiny to-do manager.")
    parser.add_argument("--file", default="todo.json")
    sub = parser.add_subparsers(dest="command", required=True)
    p_add = sub.add_parser("add", help="add a task")
    p_add.add_argument("words", nargs="+")
    args = parser.parse_args()

    store = load(args.file)
    if args.command == "add":
        task = add_task(store, " ".join(args.words))
        save(args.file, store)
        print(f"added {task['id']}: {task['text']}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Hint 1

Give each subcommand its own parser: p = sub.add_parser("done"), p.add_argument("id", type=int).

Hint 2

p.set_defaults(func=cmd_done) attaches a handler; then main just calls args.func(store, args).

Hint 3

Write a find_task(store, task_id) helper used by both done and remove.

Hint 4

list filters with args.all or not t["done"].

Reference solution

import argparse
import json
import os
import sys


def load(path: str) -> dict:
    try:
        with open(path, encoding="utf-8") as f:
            return json.load(f)
    except FileNotFoundError:
        return {"next_id": 1, "tasks": []}


def save(path: str, store: dict) -> None:
    tmp = path + ".tmp"
    with open(tmp, "w", encoding="utf-8") as f:
        json.dump(store, f, indent=2, sort_keys=True)
        f.write("\n")
    os.replace(tmp, path)


def add_task(store: dict, text: str) -> dict:
    task = {"id": store["next_id"], "text": text, "done": False}
    store["tasks"].append(task)
    store["next_id"] += 1
    return task


def find_task(store: dict, task_id: int) -> dict | None:
    for task in store["tasks"]:
        if task["id"] == task_id:
            return task
    return None


def cmd_add(store: dict, args) -> int:
    task = add_task(store, " ".join(args.words))
    save(args.file, store)
    print(f"added {task['id']}: {task['text']}")
    return 0


def cmd_list(store: dict, args) -> int:
    tasks = [t for t in store["tasks"] if args.all or not t["done"]]
    if not tasks:
        print("no tasks")
    for t in sorted(tasks, key=lambda t: t["id"]):
        mark = "x" if t["done"] else " "
        print(f"[{mark}] {t['id']} {t['text']}")
    return 0


def cmd_done(store: dict, args) -> int:
    task = find_task(store, args.id)
    if task is None:
        print(f"error: no task {args.id}", file=sys.stderr)
        return 1
    task["done"] = True
    save(args.file, store)
    print(f"completed {task['id']}: {task['text']}")
    return 0


def cmd_remove(store: dict, args) -> int:
    task = find_task(store, args.id)
    if task is None:
        print(f"error: no task {args.id}", file=sys.stderr)
        return 1
    store["tasks"].remove(task)
    save(args.file, store)
    print(f"removed {task['id']}: {task['text']}")
    return 0


def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(description="A tiny to-do manager.")
    parser.add_argument("--file", default="todo.json", help="JSON store (default: todo.json)")
    sub = parser.add_subparsers(dest="command", required=True)

    p = sub.add_parser("add", help="add a task")
    p.add_argument("words", nargs="+")
    p.set_defaults(func=cmd_add)

    p = sub.add_parser("list", help="show tasks")
    p.add_argument("--all", action="store_true", help="include completed tasks")
    p.set_defaults(func=cmd_list)

    p = sub.add_parser("done", help="mark a task completed")
    p.add_argument("id", type=int)
    p.set_defaults(func=cmd_done)

    p = sub.add_parser("remove", help="delete a task")
    p.add_argument("id", type=int)
    p.set_defaults(func=cmd_remove)
    return parser


def main() -> int:
    args = build_parser().parse_args()
    store = load(args.file)
    return args.func(store, args)


if __name__ == "__main__":
    sys.exit(main())