Practical projects · about 50 min

Processing CSV data

Why this matters

CSV (comma-separated values) is the lingua franca of tabular data: spreadsheet exports, bank statements, database dumps, sensor logs, government open data. Nearly every programmer eventually writes "read this CSV, compute some totals, write a report". It looks trivial, split each line on commas, until a field contains a comma, a quote or a line break. Python's csv module handles those details so your code can focus on the data.

What CSV really is

A CSV file is text. Each line is a record, fields are separated by commas, and the first line is usually a header naming the columns:

date,region,product,units,unit_price
2026-10-01,North,widget,4,10.00
2026-10-02,West,"widget, large",1,30.25

The quoting rules (described in RFC 4180, though real files vary):

A field containing the delimiter, a double quote, or a newline is enclosed in double quotes: "widget, large".
A double quote inside a quoted field is written twice: "said ""hi""" means said "hi".
A quoted field may span several physical lines.

So line.split(",") breaks on "widget, large", and reading line by line breaks multi-line fields. The csv module implements a small state machine that handles all of this.

Two more facts to keep in mind: CSV has no types (every field is a string; "4" must be converted to an int by you), and there are dialects: semicolons as separators in many European locales, tabs (TSV), different quoting habits. csv.reader(f, delimiter=";") or delimiter="\t" handles those.

Reading: csv.reader and newline=""

import csv

with open("sales.csv", newline="", encoding="utf-8") as f:
    reader = csv.reader(f)
    header = next(reader)          # first row
    for row in reader:
        print(row)

['2026-10-01', 'North', 'widget', '4', '10.00']
['2026-10-02', 'West', 'widget, large', '1', '30.25']

csv.reader wraps the file object and yields each record as a list of strings. next(reader) consumes the header.

Why newline=""? In text mode, Python normally translates line endings (\r\n on Windows-made files) into \n as it reads. The csv module wants to see the raw line endings itself, both to recognize \r\n record ends correctly and to preserve newlines inside quoted fields. The csv documentation says to always open CSV files with newline="", for reading and writing. Also give an explicit encoding; spreadsheet exports are often utf-8-sig (UTF-8 with a byte order mark), which you can pass as the encoding to have the mark stripped.

Reading by column name: DictReader

Indexing row[3] is fragile: if someone adds a column, every index shifts. csv.DictReader uses the header row as keys:

import csv

with open("sales.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        revenue = int(row["units"]) * float(row["unit_price"])
        print(row["region"], row["product"], f"{revenue:.2f}")

North widget 40.00
West widget, large 30.25

Each row is a dict mapping column names to strings. Missing trailing fields become None (or the restval you pass), and extra fields are collected under the key None.

Aggregating

Most CSV jobs are "group by something, sum something". collections.defaultdict and Counter make it short:

import csv
from collections import defaultdict

totals = defaultdict(float)
counts = defaultdict(int)
with open("sales.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        totals[row["region"]] += int(row["units"]) * float(row["unit_price"])
        counts[row["region"]] += 1

for region in sorted(totals):
    print(f"{region:<6} {counts[region]:>3} sales {totals[region]:>9.2f}")

East     3 sales    120.50
North    3 sales     72.00
West     1 sales     30.25

Deterministic output. A dict iterates in insertion order, which here is the order regions first appear in the file. That is reproducible but accidental; reports are clearer, and tests more robust, when you sort explicitly. To sort by value with ties broken by name, use a key that returns a tuple: sorted(totals.items(), key=lambda kv: (-kv[1], kv[0])). Note Counter.most_common() breaks ties by insertion order, so it is not enough on its own when ties matter.

Money and floats. 0.1 + 0.2 is 0.30000000000000004 in binary floating point. Formatting with :.2f hides small errors in reports, but for real accounting use decimal.Decimal(row["unit_price"]), which does exact decimal arithmetic.

Writing: csv.writer and DictWriter

import csv

rows = [
    {"region": "North", "product": "widget", "revenue": "40.00"},
    {"region": "West", "product": 'widget, "large"', "revenue": "30.25"},
]
with open("report.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["region", "product", "revenue"])
    writer.writeheader()
    writer.writerows(rows)

report.csv now contains:

region,product,revenue
North,widget,40.00
West,"widget, ""large""",30.25

The writer added the quoting the second product needed. csv.writer(f).writerow([...]) is the list-based equivalent. Values that are not strings are converted with str(), so format numbers yourself if you need a fixed number of decimals.

By default the writer ends each row with \r\n, as the CSV standard says. Without newline="" on a Windows machine, text mode would turn that \n into \r\n again, giving \r\r\n and a blank line between rows in Excel: the classic symptom of a forgotten newline="".

A complete pipeline

import csv
import sys
from collections import defaultdict


def read_sales(path: str) -> list[dict[str, str]]:
    with open(path, newline="", encoding="utf-8") as f:
        return list(csv.DictReader(f))


def revenue_by_product(rows: list[dict[str, str]]) -> dict[str, float]:
    totals: dict[str, float] = defaultdict(float)
    for row in rows:
        totals[row["product"]] += int(row["units"]) * float(row["unit_price"])
    return totals


def write_totals(totals: dict[str, float], out) -> None:
    w = csv.writer(out)
    w.writerow(["product", "revenue"])
    for product in sorted(totals):
        w.writerow([product, f"{totals[product]:.2f}"])


if __name__ == "__main__":
    write_totals(revenue_by_product(read_sales(sys.argv[1])), sys.stdout)

Reading, computing and writing are separate functions, so each can be tested alone, and writing to sys.stdout lets the user redirect the result to a file or pipe it into another tool. For very large files, avoid list(...) and process rows as they stream from the reader: memory then stays constant regardless of file size.

Common mistakes

Splitting lines by hand: breaks on quoted commas. Use the csv module.
Forgetting to convert: row["units"] * 2 is "44", and sum of strings raises TypeError: unsupported operand type(s) for +: 'int' and 'str'.
KeyError: 'region' on the first row: the header has stray spaces (region, product), a different capitalization, or a byte order mark ('region'). Print reader.fieldnames to see the actual keys; open with encoding="utf-8-sig" for the BOM; pass skipinitialspace=True for spaces after commas.
Blank lines between rows in output: open the output file with newline="".
Relying on dict or Counter order for ties: sort with an explicit tie-breaker.
ValueError: invalid literal for int() on an empty field: real data has gaps. Decide on a policy (skip the row, treat as 0, report it) and make it explicit in code.

Recap

CSV fields are strings separated by commas, with double-quote quoting for special characters; use the csv module, never split(",").
Open CSV files with newline="" and an explicit encoding.
csv.reader yields lists; csv.DictReader yields dicts keyed by the header.
Aggregate with defaultdict or Counter, convert types explicitly, and sort explicitly (with tie-breakers) for deterministic output.
csv.writer / csv.DictWriter produce correctly quoted output; format numbers yourself.

Going further: read the csv module docs on dialects and csv.Sniffer, and try decimal.Decimal in place of float in the region-totals exercise to see exact money arithmetic.

Exercises

Exercise 1 · quiz

Why not split on commas?

A line of a CSV file reads:

42,"Smith, Jane","said ""hi"""

How many fields does csv.reader see, and what is the third field's value?

5 fields; the third is Jane"
3 fields; the third is said "hi"
3 fields; the third is "said ""hi"""
4 fields; the third is said

Answer

3 fields; the third is said "hi"

In CSV, a field containing a comma, quote or newline is wrapped in double quotes, and a literal double quote inside it is doubled. The csv module undoes this: the fields are 42, Smith, Jane and said "hi". line.split(",") would give five broken pieces. That is the whole reason to use the module.

Exercise 2 · write code

Revenue per region

main.py receives the path of a sales CSV file as its only command-line argument. The file has a header row date,region,product,units,unit_price.

Print the total revenue (units * unit_price) for each region, sorted by region name, then a grand total, each with two decimals, in exactly this format:

East: 120.50
North: 72.00
West: 30.25
TOTAL: 222.75

(That is the result for the provided sales.csv.) If the file has a header but no data rows, print only TOTAL: 0.00. Use csv.DictReader and open the file with newline="".

File: main.py

Starting point

import sys

path = sys.argv[1]
with open(path) as f:
    for line in f:
        print(line.split(","))

Example with arguments sales.csv

expected output

East: 120.50
North: 72.00
West: 30.25
TOTAL: 222.75

Example with arguments empty.csv

expected output

TOTAL: 0.00

Hint 1

csv.DictReader(f) yields one dict per row, keyed by the header names; every value is a string.

Hint 2

Convert with int(row["units"]) and float(row["unit_price"]), and accumulate into a defaultdict(float).

Hint 3

Loop over sorted(totals) for a deterministic order. Region names are case-sensitive: south and South are different.

Reference solution

import csv
import sys
from collections import defaultdict


def region_totals(path):
    totals = defaultdict(float)
    with open(path, newline="", encoding="utf-8") as f:
        for row in csv.DictReader(f):
            totals[row["region"]] += int(row["units"]) * float(row["unit_price"])
    return totals


def main():
    totals = region_totals(sys.argv[1])
    for region in sorted(totals):
        print(f"{region}: {totals[region]:.2f}")
    print(f"TOTAL: {sum(totals.values()):.2f}")


if __name__ == "__main__":
    main()

Exercise 3 · write code

Top products, with ties

In solution.py, write top_products(path, n) that reads a sales CSV (same columns as before) and returns a list of (product, total_units) tuples for the n products with the most units sold.

Sort by total units descending, and break ties by product name ascending, so the result is always the same. total_units is an int. If there are fewer than n products, return them all.

File: solution.py

Starting point

import csv


def top_products(path, n):
    return []

Hint 1

Accumulate units per product in a Counter.

Hint 2

Counter.most_common breaks ties by insertion order, which depends on the file. Sort yourself instead.

Hint 3

sorted(items, key=lambda item: (-item[1], item[0])) sorts by count descending, then name ascending.

Reference solution

import csv
from collections import Counter


def top_products(path, n):
    units = Counter()
    with open(path, newline="", encoding="utf-8") as f:
        for row in csv.DictReader(f):
            units[row["product"]] += int(row["units"])
    ranked = sorted(units.items(), key=lambda item: (-item[1], item[0]))
    return ranked[:n]

Exercise 4 · write code

Write a cleaned CSV

In solution.py, write write_report(in_path, out_path) that reads a sales CSV and writes a new CSV to out_path with the header region,product,revenue, one row per input row, where:

region is stripped of surrounding spaces and title-cased (" north " becomes North),
product is copied unchanged (it may contain commas or quotes, the csv module must quote it),
revenue is units * unit_price formatted with two decimals ("40.00"),
rows whose units is 0 are skipped.

Return the number of rows written (excluding the header). Use csv.DictWriter and open the output with newline="".

File: solution.py

Starting point

import csv


def write_report(in_path, out_path):
    with open(out_path, "w") as out:
        out.write("region,product,revenue\n")
    return 0

Hint 1

Open both files in one with statement, separated by a comma.

Hint 2

writer = csv.DictWriter(dst, fieldnames=[...]), then writer.writeheader() and writer.writerow({...}) per row.

Hint 3

Do not build lines with string formatting: writerow adds the quotes that fields with commas or quotes need.

Reference solution

import csv


def write_report(in_path, out_path):
    count = 0
    with open(in_path, newline="", encoding="utf-8") as src, \
         open(out_path, "w", newline="", encoding="utf-8") as dst:
        writer = csv.DictWriter(dst, fieldnames=["region", "product", "revenue"])
        writer.writeheader()
        for row in csv.DictReader(src):
            units = int(row["units"])
            if units == 0:
                continue
            writer.writerow({
                "region": row["region"].strip().title(),
                "product": row["product"],
                "revenue": f"{units * float(row['unit_price']):.2f}",
            })
            count += 1
    return count