Extract TODO and FIXME comments from Python

Before you begin

Python 3.12 or newer

What this script changesReads Python source and writes a new JSON result without changing or running analyzed code. Maximum 500 files and 2 MiB per file. Static findings are review aids; dynamic behavior and runtime imports are outside scope.

Set up your example

Save the complete code as python-todo-comments.py in a working folder.
Save the provided sample files in that same folder; run commands from that folder.

Sample files in this download:

sample.py

Download the ZIP, extract it, and open a terminal inside the extracted folder. The source file is python-todo-comments.py.

Run the script

python "python-todo-comments.py" --source "sample.py" --output "analysis.json"

If your system uses python3 instead of python, use that command for Python examples. Keep paths containing spaces in quotes. Required inputs and optional settings are described in the code and setup steps.

What to expect

Collect actionable comment markers without matching string literals. The fixture reports replace the demo greeting on its TODO comment line.

The complete script

#!/usr/bin/env python3
'Extract TODO and FIXME comments from Python'
import argparse
import json
from pathlib import Path

def write_json(path, value):
    data = json.dumps(value, indent=2, ensure_ascii=False, allow_nan=False) + "\n"
    with Path(path).open("x", encoding="utf-8", newline="") as handle:
        handle.write(data)

import ast
import tokenize
import io
import doctest
from collections import Counter, defaultdict

FUNCTIONS=(ast.FunctionDef,ast.AsyncFunctionDef)
SCOPES=(ast.FunctionDef,ast.AsyncFunctionDef,ast.ClassDef,ast.Lambda)
def functions(tree):
    return [n for n in ast.walk(tree) if isinstance(n,FUNCTIONS)]
def local_nodes(node):
    yield node
    for child in ast.iter_child_nodes(node):
        if not isinstance(child,SCOPES):
            yield from local_nodes(child)
def name(node):
    try: return ast.unparse(node)
    except (ValueError,TypeError): return type(node).__name__
def targets(node):
    return [n.id for n in ast.walk(node) if isinstance(n,ast.Name) and isinstance(n.ctx,ast.Store)]
def sources(value):
    root=Path(value)
    if root.is_file(): return [root]
    if not root.is_dir(): raise ValueError("--source must be a Python file or directory.")
    excluded={".git",".venv","venv","node_modules","__pycache__"}
    files=sorted(p for p in root.rglob("*.py") if p.is_file() and not any(x in excluded or x.startswith(".") for x in p.relative_to(root).parts))
    if len(files)>500: raise ValueError("At most 500 Python files per run.")
    return files


def analyze(tree,text):
    result=[]
    for token in tokenize.generate_tokens(io.StringIO(text).readline):
        if token.type==tokenize.COMMENT and any(marker in token.string.upper() for marker in ("TODO","FIXME")):
            result.append({"line":token.start[0],"column":token.start[1],"comment":token.string})
    return result

def main():
    parser = argparse.ArgumentParser(description='Extract TODO and FIXME comments from Python')
    parser.add_argument("--source",required=True,help="Python file or source directory")
    parser.add_argument("--output",required=True)
    args = parser.parse_args()
    reports=[]
    for path in sources(args.source):
        try:
            if path.stat().st_size>2*1024*1024: raise ValueError("Source exceeds 2 MiB.")
            with tokenize.open(path) as handle: text=handle.read()
            tree=ast.parse(text,filename=str(path))
            reports.append({"path":str(path),"result":analyze(tree,text)})
        except (OSError,SyntaxError,UnicodeError,ValueError) as exc:
            reports.append({"path":str(path),"error":str(exc)})
    write_json(args.output,reports)

if __name__ == "__main__":
    try:
        main()
    except (OSError, ValueError, KeyError, TypeError, ArithmeticError, IndexError, LookupError) as exc:
        raise SystemExit(str(exc))

Open source as plain text · Download source and sample files

How it works

The program parses source syntax without importing or executing it; per-file parse failures become error records.
The tokenizer identifies genuine comment tokens; TODO text inside ordinary strings is excluded.
The fixture reports replace the demo greeting on its TODO comment line.

Make it your own

--source

Use --source with a Python file or directory; directory mode skips common dependency and hidden folders.

Adapt the processing

Edit analyze's marker tuple to add HACK or a team-specific marker.

Change one setting at a time, run it again on sample data, and compare the output. The customization guide walks through this process.

If something goes wrong

A TODO in a docstring is absent.

This tool inventories comments only; docstrings are string syntax.

The destination already exists.

Choose a new --output path. The script deliberately refuses to replace an existing result.

Reference & next steps

These official references explain the underlying tools. The Scriptage example and guide are original instructional content. See verification notes for what has been checked and the limits of those checks.

Copies a template for your own notes. It does not send anything.