PythonBeginner · Windows, macOS, Linux; Python 3.12+

Pack size-limited ZIP volumes

Group source files into ZIPs by a maximum uncompressed payload budget.

Before you begin

  • Python 3.12+
What this script changesReads the chosen input and creates a new result. Existing outputs are refused. Reports can contain source names or text, so store them appropriately.

Set up your example

  1. Save the complete code as pack-size-limited-zips.py.
  2. Open a terminal in the folder containing the script.
  3. Create the supplied sample files at the shown relative paths.

Sample files in this download:

  • sample/notes.txt
  • sample/sub/report.md
  • sample/sub/copy.txt

Download the ZIP, extract it, and open a terminal inside the extracted folder. The source file is pack-size-limited-zips.py.

Run the script

Run command
python pack-size-limited-zips.py --input "sample" --output "packed" --max-bytes 12

If your system uses python3 instead of python, use that command for Python examples. Keep paths containing spaces in quotes. Required inputs and optional settings are described in the code and setup steps.

What to expect

The sample creates multiple ZIP volumes, each with at most 12 uncompressed payload bytes; ZIP headers add storage overhead.

The complete script

pack-size-limited-zips.py
import argparse, ast, collections, difflib, fnmatch, gzip, hashlib, io, json, math, os, random, re, shutil, statistics, string, sys, tarfile, textwrap, time, unicodedata, zipfile
from pathlib import Path
import xml.etree.ElementTree as ET

def write_text(path, content):
    path=Path(path)
    path.parent.mkdir(parents=True,exist_ok=True)
    with path.open('x',encoding='utf-8',newline='') as h: h.write(content)
    print('Created',path)

def fresh_dir(path):
    path=Path(path); path.mkdir(parents=True,exist_ok=False); return path

def files_under(root):
    root=Path(root)
    if not root.is_dir(): raise ValueError('Input must be a directory')
    return sorted((p for p in root.rglob('*') if p.is_file() and not p.is_symlink()),key=lambda p:p.as_posix().casefold())

def digest(path):
    h=hashlib.sha256()
    with Path(path).open('rb') as f:
        for b in iter(lambda:f.read(1048576),b''): h.update(b)
    return h.hexdigest()

def positive(n,name):
    if n<=0: raise ValueError(name+' must be positive')

def destination(root,name):
    name=name.replace('\\','/')
    if name.startswith('/') or any(p=='..' or ':' in p for p in name.split('/')): raise ValueError('Unsafe path: '+name)
    p=(root/name).resolve()
    if p!=root.resolve() and root.resolve() not in p.parents: raise ValueError('Path escapes destination')
    return p

def zip_members(z,limit=500000000):
    seen=set(); total=0; result=[]
    for i in z.infolist():
        if i.is_dir(): continue
        destination(Path.cwd(),i.filename)
        key=i.filename.replace('\\','/').casefold()
        if key in seen: raise ValueError('Duplicate/case-colliding member: '+i.filename)
        seen.add(key)
        if ((i.external_attr>>16)&0o170000)==0o120000: raise ValueError('Symlinks unsupported')
        total+=i.file_size
        if total>limit: raise ValueError('Expanded data exceeds limit')
        result.append(i)
    return result

def copy_member(z,i,root):
    p=destination(root,i.filename); p.parent.mkdir(parents=True,exist_ok=True)
    with z.open(i) as src,p.open('xb') as dst: shutil.copyfileobj(src,dst)

def read_input(a): return a.input.read_text(encoding='utf-8-sig')
NS={'w':'http://schemas.openxmlformats.org/wordprocessingml/2006/main','r':'http://schemas.openxmlformats.org/officeDocument/2006/relationships'}
def docx_root(path,member='word/document.xml'):
    with zipfile.ZipFile(path) as z:
        i=z.getinfo(member)
        if i.file_size>30000000: raise ValueError('Document XML exceeds 30 MB')
        return ET.fromstring(z.read(i))
def pdf_reader(path):
    try: from pypdf import PdfReader
    except ImportError as e: raise ValueError('Install dependency: python -m pip install pypdf') from e
    r=PdfReader(path)
    if r.is_encrypted: raise ValueError('Encrypted PDF unsupported')
    return r
def pdf_writer():
    try: from pypdf import PdfWriter
    except ImportError as e: raise ValueError('Install dependency: python -m pip install pypdf') from e
    return PdfWriter()
def save_pdf(w,p):
    Path(p).parent.mkdir(parents=True,exist_ok=True)
    with Path(p).open('xb') as h: w.write(h)
    print('Created',p)

def main():
    parser=argparse.ArgumentParser(description='Group source files into ZIPs by a maximum uncompressed payload budget.')
    parser.add_argument('--input',type=Path,required=True,help='Source file or folder')
    parser.add_argument('--output',type=Path,required=True,help='New result path; must not exist')
    parser.add_argument('--max-bytes',type=int,default=25000000)
    args=parser.parse_args()
    positive(args.max_bytes,'--max-bytes');paths=files_under(args.input);out=args.output.resolve();root=args.input.resolve()
    if out==root or root in out.parents: raise ValueError('Output must be outside source')
    if any(p.stat().st_size>args.max_bytes for p in paths): raise ValueError('A single file exceeds the budget')
    fresh_dir(out);groups=[];group=[];size=0
    for p in paths:
        n=p.stat().st_size
        if group and size+n>args.max_bytes: groups.append(group);group=[];size=0
        group.append(p);size+=n
    affected=groups+([group] if group else [])
    for n,group in enumerate(affected,1):
        with zipfile.ZipFile(out/f'volume-{n:03}.zip','x',compression=zipfile.ZIP_DEFLATED) as z:
            for p in group: z.write(p,p.relative_to(args.input).as_posix())

if __name__=='__main__':
    try: main()
    except (OSError,ValueError,KeyError,zipfile.BadZipFile,tarfile.TarError,ET.ParseError) as error:
        print('Error:',error,file=sys.stderr); sys.exit(2)

Open source as plain text · Download source and sample files

How it works

  1. Group source files into ZIPs by a maximum uncompressed payload budget.
  2. Customization in this script: Change --max-bytes to choose each ZIP volume’s maximum uncompressed payload budget.
  3. Review the output before using it elsewhere; existing output paths are refused.

Make it your own

Change the task behavior

Change --max-bytes to choose each ZIP volume’s maximum uncompressed payload budget.

Choose a source

Change --input in the run command to select your own source for pack size-limited zip volumes.

Choose a destination

Change --output to a new path. The write_text(), fresh_dir(), and save_pdf() helpers refuse existing outputs.

Change one setting at a time, run it again on sample data, and compare the output. The customization guide walks through this process.

If something goes wrong

The chosen source cannot be read.

Check the source path, quote paths with spaces, and confirm the file or folder is readable.

The output already exists.

Use a new --output path; existing results are deliberately preserved.

A task-specific argument is rejected.

Run python pack-size-limited-zips.py --help, then compare the options with the complete example command and supplied sample format.

Reference & next steps

These official references explain the underlying tools. The Scriptage example and guide are original instructional content. See verification notes for what has been checked and the limits of those checks.

Copies a template for your own notes. It does not send anything.