Paths: why your program cannot find a file that is right there

The failure

open("data/sales.csv")

FileNotFoundError: [Errno 2] No such file or directory: 'data/sales.csv'

The file exists. You can see it. The program still cannot find it, because a relative path is resolved against the current working directory — the folder your terminal was in when you started Python — and not against the folder the script lives in.

from pathlib import Path
print(Path.cwd())            # where relative paths start from
print(Path(__file__))        # where this script actually is

Run python3 tools/clean.py from the project root and the working directory is the root, so data/sales.csv resolves relative to the root. Run the same script from inside tools/ and it does not. Same code, same file, different answer.

The fix that always works

from pathlib import Path

HERE = Path(__file__).resolve().parent
DATA = HERE.parent / "data" / "sales.csv"

__file__ is the path of the module. .resolve() makes it absolute and follows symlinks. .parent walks up. Build every path in your program from that anchor and it stops mattering where anyone runs it from.

The one place __file__ is unavailable is an interactive session or a notebook cell, where Path.cwd() is the sensible substitute.

pathlib, and the slash operator

from pathlib import Path

p = Path("data") / "2026" / "sales.csv"
p.name          # 'sales.csv'
p.stem          # 'sales'
p.suffix        # '.csv'
p.parent        # PosixPath('data/2026')
p.exists()
p.is_file()
p.stat().st_size

The / operator joins path parts with the correct separator for the operating system. That is the point: "data" + "/" + name produces a path that is wrong on Windows, and os.path.join is correct but verbose. Code written with pathlib runs unchanged on Windows, macOS, Linux and a phone.

Reading and writing small files needs no open at all:

text = p.read_text(encoding="utf-8")
p.write_text(report, encoding="utf-8")
raw = p.read_bytes()

Creating folders, safely

out = HERE / "output" / "2026"
out.mkdir(parents=True, exist_ok=True)

parents=True creates the intermediate folders. exist_ok=True means a second run does not raise FileExistsError. Almost every script that writes output wants both, and forgetting them is the reason a job works once and fails on the retry.

Finding files

list(DATA_DIR.glob("*.csv"))            # this folder only
list(DATA_DIR.rglob("*.csv"))           # every subfolder too
sorted(DATA_DIR.glob("sales_*.csv"))

glob returns a generator, so wrap it in list or sorted when you want to count or order it. Sorting matters more than it looks: the order glob yields is whatever the file system gives, which differs between machines, and a pipeline that processes files in an unpredictable order produces unpredictable output.

The Windows detail

A Windows path in a Python string is a minefield of backslashes:

"C:\\Users\\asha\\data\\new.csv"       # doubled, correct but ugly
r"C:\Users\asha\data\new.csv"         # raw string, correct
Path("C:/Users/asha/data/new.csv")    # forward slashes work on Windows too

\n, \t and \U are escape sequences, so "C:\Users\new" contains a newline and is not the path you typed. Use pathlib and forward slashes and the problem does not arise.

Deleting, and being careful about it

p.unlink(missing_ok=True)       # delete a file
d.rmdir()                       # only works on an empty directory

There is deliberately no one-line recursive delete in pathlib. shutil.rmtree exists and removes a whole tree without confirmation. Before writing that call, ask whether the path could ever be empty or /, because rmtree(base / user_input) with an empty user_input deletes base. A --dry-run flag that prints what would be deleted, as the command-line lesson described, is worth the ten minutes.

Temporary files

import tempfile
with tempfile.TemporaryDirectory() as tmp:
    scratch = Path(tmp) / "working.csv"

The folder is deleted when the block ends, including if an exception is raised. This is better than writing to a fixed temporary folder by hand, and it works identically on Windows, where the Unix conventions do not apply.

Reading a file's details

p.stat().st_size          # bytes
p.stat().st_mtime         # last modified, as epoch seconds
p.is_dir(), p.is_file()
p.with_suffix(".json")    # same path, different extension
p.relative_to(HERE)       # for printing something short

st_mtime is what lets a job skip work it has already done: compare the source file's modification time against the output's and rebuild only when the source is newer. Ten lines of that turn a twenty-minute pipeline into a two-second one on the second run.

Writing safely

If a job is killed halfway through writing a file, you are left with a truncated file that looks valid. The standard defence is to write to a temporary name in the same folder, then rename:

tmp = out.with_suffix(".csv.tmp")
tmp.write_text(data, encoding="utf-8")
tmp.replace(out)

replace is atomic on the same file system, so readers see either the old complete file or the new complete file, never a half-written one.