Extract Python docstring examples without running them

Before you begin

Python 3.12 or newer

What this script changesReads Python source and writes a new JSON result without changing or running analyzed code. Maximum 500 files and 2 MiB per file. Static findings are review aids; dynamic behavior and runtime imports are outside scope.

Set up your example

Save the complete code as python-doctest-extract.py in a working folder.
Save the provided sample files in that same folder; run commands from that folder.

Sample files in this download:

sample.py

Download the ZIP, extract it, and open a terminal inside the extracted folder. The source file is python-doctest-extract.py.

Run the script

python "python-doctest-extract.py" --source "sample.py" --output "analysis.json"

If your system uses python3 instead of python, use that command for Python examples. Keep paths containing spaces in quotes. Required inputs and optional settings are described in the code and setup steps.

What to expect

Export doctest prompts and expected output from docstrings. The fixture extracts greet('Ada') with expected output 'Hello Ada'.

The complete script

#!/usr/bin/env python3
'Extract Python docstring examples without running them'
import argparse
import json
from pathlib import Path

def write_json(path, value):
    data = json.dumps(value, indent=2, ensure_ascii=False, allow_nan=False) + "\n"
    with Path(path).open("x", encoding="utf-8", newline="") as handle:
        handle.write(data)

import ast
import tokenize
import io
import doctest
from collections import Counter, defaultdict

FUNCTIONS=(ast.FunctionDef,ast.AsyncFunctionDef)
SCOPES=(ast.FunctionDef,ast.AsyncFunctionDef,ast.ClassDef,ast.Lambda)
def functions(tree):
    return [n for n in ast.walk(tree) if isinstance(n,FUNCTIONS)]
def local_nodes(node):
    yield node
    for child in ast.iter_child_nodes(node):
        if not isinstance(child,SCOPES):
            yield from local_nodes(child)
def name(node):
    try: return ast.unparse(node)
    except (ValueError,TypeError): return type(node).__name__
def targets(node):
    return [n.id for n in ast.walk(node) if isinstance(n,ast.Name) and isinstance(n.ctx,ast.Store)]
def sources(value):
    root=Path(value)
    if root.is_file(): return [root]
    if not root.is_dir(): raise ValueError("--source must be a Python file or directory.")
    excluded={".git",".venv","venv","node_modules","__pycache__"}
    files=sorted(p for p in root.rglob("*.py") if p.is_file() and not any(x in excluded or x.startswith(".") for x in p.relative_to(root).parts))
    if len(files)>500: raise ValueError("At most 500 Python files per run.")
    return files


def analyze(tree,text):
    result=[]
    parser=doctest.DocTestParser()
    for n in [tree]+[n for n in ast.walk(tree) if isinstance(n,FUNCTIONS+(ast.ClassDef,))]:
        doc=ast.get_docstring(n)
        if doc:
            for example in parser.get_examples(doc):
                result.append({"owner":getattr(n,"name","<module>"),"definition_line":getattr(n,"lineno",1),"docstring_example_line":example.lineno+1,"source":example.source,"expected":example.want})
    return result

def main():
    parser = argparse.ArgumentParser(description='Extract Python docstring examples without running them')
    parser.add_argument("--source",required=True,help="Python file or source directory")
    parser.add_argument("--output",required=True)
    args = parser.parse_args()
    reports=[]
    for path in sources(args.source):
        try:
            if path.stat().st_size>2*1024*1024: raise ValueError("Source exceeds 2 MiB.")
            with tokenize.open(path) as handle: text=handle.read()
            tree=ast.parse(text,filename=str(path))
            reports.append({"path":str(path),"result":analyze(tree,text)})
        except (OSError,SyntaxError,UnicodeError,ValueError) as exc:
            reports.append({"path":str(path),"error":str(exc)})
    write_json(args.output,reports)

if __name__ == "__main__":
    try:
        main()
    except (OSError, ValueError, KeyError, TypeError, ArithmeticError, IndexError, LookupError) as exc:
        raise SystemExit(str(exc))

Open source as plain text · Download source and sample files

How it works

The program parses source syntax without importing or executing it; per-file parse failures become error records.
Doctest parsing recognizes >>> examples but never executes them; example line numbers are relative to cleaned docstrings.
The fixture extracts greet('Ada') with expected output 'Hello Ada'.

Make it your own

--source

Use --source with a Python file or directory; directory mode skips common dependency and hidden folders.

Adapt the processing

Edit analyze's result to group examples under each owner rather than a flat list.

Change one setting at a time, run it again on sample data, and compare the output. The customization guide walks through this process.

If something goes wrong

An example fails doctest parsing.

Check prompt indentation and expected-output formatting in the source docstring.

The destination already exists.

Choose a new --output path. The script deliberately refuses to replace an existing result.

Reference & next steps

These official references explain the underlying tools. The Scriptage example and guide are original instructional content. See verification notes for what has been checked and the limits of those checks.

Copies a template for your own notes. It does not send anything.