Working with JSON

Why this matters

JSON (JavaScript Object Notation) is how most programs exchange structured data today: web APIs return it, configuration files use it, logs are written in it one object per line, and tools like jq slice it on the command line. Python's built-in json module converts between JSON text and ordinary Python dicts and lists in one call. The skills are converting correctly, navigating nested data, writing output that humans and diff tools can read, and failing clearly when the input is broken.

The JSON data model

JSON is a text format with six kinds of values, and each maps onto a Python type:

When encoding, Python also accepts tuples (written as arrays) and int, float or bool dict keys (converted to strings). Anything else, such as sets, datetime objects, your own classes or bytes, raises TypeError: Object of type set is not JSON serializable.

JSON is stricter than Python syntax: strings and keys need double quotes, there are no comments, no trailing commas, and True must be true. A JSON document is valid Python surprisingly often, which tempts people to use eval on it. Never do that: eval executes code, and JSON from the outside is untrusted input.

Four functions

import json

text = '{"name": "Ada", "langs": ["python", "c"], "admin": false, "manager": null}'
data = json.loads(text)          # load from a string
print(data)
print(data["langs"][0], type(data["admin"]))

back = json.dumps(data)          # dump to a string
print(back)

{'name': 'Ada', 'langs': ['python', 'c'], 'admin': False, 'manager': None}
python <class 'bool'>
{"name": "Ada", "langs": ["python", "c"], "admin": false, "manager": null}

The s stands for string. The versions without s work with file objects:

with open("config.json", encoding="utf-8") as f:
    config = json.load(f)        # read and parse the whole file

with open("out.json", "w", encoding="utf-8") as f:
    json.dump(config, f, indent=2)

json.load(sys.stdin) reads JSON piped into your program, which makes your script work well with curl ... | python3 tool.py.

Converting Python objects to text is called serialization (or encoding); parsing text back is deserialization (decoding).

Readable, stable output

By default dumps writes everything on one line. Two options make output friendly to humans and to version control:

data = {"zeta": 1, "alpha": [1, 2], "mid": {"b": 2, "a": 1}}
print(json.dumps(data, indent=2, sort_keys=True))

{
  "alpha": [
    1,
    2
  ],
  "mid": {
    "a": 1,
    "b": 2
  },
  "zeta": 1
}

indent=2 pretty-prints; sort_keys=True orders keys alphabetically, so the same data always produces byte-identical output, which keeps diffs small and tests stable. Other useful options: ensure_ascii=False writes non-ASCII characters as themselves ("café") instead of escapes ("café"), and separators=(",", ":") produces the most compact form. json.dump does not add a final newline; write "\n" yourself if you want the file to end cleanly.

Navigating nested data

Real JSON is nested: objects inside arrays inside objects. Work through it one level at a time, and know what type you hold at each step:

import json

doc = json.loads("""
{"repo": "learnprog",
 "commits": [
   {"sha": "d2d5a3a", "author": {"name": "grace"}, "files": ["cli.py", "checker.py"]},
   {"sha": "9f1c0b2", "author": {"name": "ada"}, "files": []}
 ]}
""")

for commit in doc["commits"]:               # a list of dicts
    who = commit["author"]["name"]          # dict inside dict
    print(commit["sha"], who, len(commit["files"]))

d2d5a3a grace 2
9f1c0b2 ada 0

For optional fields, use .get with a default instead of [...], which raises KeyError:

email = user.get("email") or "-"     # missing key and null both become "-"

Note the difference: user.get("email", "-") returns None if the key exists with value null; or "-" covers both cases (and also the empty string, which may or may not be what you want).

When the structure gets deep, converting to dataclasses at the boundary of your program makes the rest of the code clearer and type-checked:

from dataclasses import dataclass

@dataclass
class Commit:
    sha: str
    author: str
    files: list[str]

commits = [Commit(c["sha"], c["author"]["name"], c["files"]) for c in doc["commits"]]

Serializing your own types

To encode objects JSON does not know, pass a default function that converts them into something it does know:

import json
from datetime import date

def encode(obj):
    if isinstance(obj, date):
        return obj.isoformat()
    if isinstance(obj, set):
        return sorted(obj)
    raise TypeError(f"cannot serialize {type(obj).__name__}")

print(json.dumps({"day": date(2026, 11, 1), "tags": {"b", "a"}}, default=encode))

{"day": "2026-11-01", "tags": ["a", "b"]}

For a dataclass, dataclasses.asdict(obj) gives a dict ready for dumps. Decoding back into rich types is your job: JSON only knows the six basic kinds, so a date comes back as a string until you call date.fromisoformat.

Handling bad input

Invalid JSON raises json.JSONDecodeError, a subclass of ValueError that knows where the problem is:

import json

try:
    json.loads('{"port": 8080,}')
except json.JSONDecodeError as e:
    print(e)
    print(e.lineno, e.colno, e.pos)

Illegal trailing comma before end of object: line 1 column 14 (char 13)
1 14 13

Valid JSON may still be the wrong shape: an array where you expected an object, a string where you expected a number. Check types at the boundary with isinstance and report a clear error, rather than letting a confusing TypeError or KeyError surface deep in your code. One subtlety: bool is a subclass of int in Python, so isinstance(True, int) is True; exclude bools explicitly when a number is required.

JSON Lines

Logs and data exports often use JSON Lines (.jsonl): one complete JSON value per line. It can be appended to and processed as a stream, without loading everything into memory:

import json

with open("events.jsonl", encoding="utf-8") as f:
    for lineno, line in enumerate(f, 1):
        if line.strip():
            event = json.loads(line)
            ...

Common mistakes

• Confusing load and loads: json.loads(f) with a file gives TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper; json.load("text") gives AttributeError: 'str' object has no attribute 'read'.
• Single quotes in hand-written JSON: {'a': 1} gives JSONDecodeError: Expecting property name enclosed in double quotes.
• Expecting int keys back: after a round trip, d[1] raises KeyError: 1; the key is now "1".
• Printing a dict instead of JSON: print(data) shows Python syntax ('single quotes', True, None), which is not valid JSON. Use json.dumps(data).
• Serializing sets or datetimes: TypeError: Object of type datetime is not JSON serializable. Convert them, or pass default=.

Recap

• JSON has objects, arrays, strings, numbers, booleans and null, mapping to dict, list, str, int/float, bool and None; object keys are always strings.
• loads/dumps work with strings, load/dump with files; indent=2, sort_keys=True gives readable, stable output.
• Navigate nested data level by level; use .get for optional fields.
• Use default= to serialize custom types; convert back explicitly.
• Catch json.JSONDecodeError (with lineno/colno) and validate the shape with isinstance.

Going further: try python3 -m json.tool file.json, a built-in pretty-printer and validator, and jq '.users[] | .name' in the sandbox to query JSON from the shell.