PythonBeginner · Windows, macOS, or Linux with Python 3.12+

Take a finite availability sample

Run a finite series of HTTP availability checks and save each outcome.

Before you begin

  • Python 3.12 or newer
What this script changesMakes requests only to supplied targets; run checks against services you own or have permission to inspect. No credentials are accepted. HTTP bodies default to 1 MiB and socket operations have a timeout; urllib's system DNS resolution can exceed the socket timeout. This is a one-shot tool, not a scheduled monitor.

Set up your example

  1. Save the complete code as http-availability-sample.py in a working folder.
  2. Save the provided sample files in that same folder; run commands from that folder.
  3. For the provided local HTTP examples, open another terminal in this folder and run: python -m http.server 8765 --bind 127.0.0.1. Keep it running while using the tool.

Sample files in this download:

  • urls.txt
  • sample.txt
  • data.json
  • page1.json
  • page2.json
  • robots.txt
  • ips.txt

Download the ZIP, extract it, and open a terminal inside the extracted folder. The source file is http-availability-sample.py.

Run the script

Run command
python "http-availability-sample.py" --url "http://127.0.0.1:8765/sample.txt" --attempts 3 --interval 1 --output "availability.json"

If your system uses python3 instead of python, use that command for Python examples. Keep paths containing spaces in quotes. Required inputs and optional settings are described in the code and setup steps.

What to expect

Run a finite series of HTTP availability checks and save each outcome. With the fixture server running, all three attempts report ok true.

The complete script

http-availability-sample.py
#!/usr/bin/env python3
'Take a finite availability sample'
import argparse
import json
from pathlib import Path

def write_json(path, value):
    data = json.dumps(value, indent=2, ensure_ascii=False, allow_nan=False) + "\n"
    with Path(path).open("x", encoding="utf-8", newline="") as handle:
        handle.write(data)

import urllib.request
import urllib.error
import urllib.parse
import urllib.robotparser
import socket
import ssl
import time
import re
import ipaddress
import queue
import threading
from datetime import datetime, timezone

def url(value):
    parts=urllib.parse.urlsplit(value)
    if parts.scheme not in {"http","https"} or not parts.hostname or parts.username is not None or parts.password is not None:
        raise ValueError("Use an http(s) URL without embedded credentials.")
    return value
def request(address,timeout,method="GET",headers=None,limit=1024*1024,opener=None):
    url(address)
    if not 0<timeout<=120: raise ValueError("Timeout must be 0..120 seconds, excluding zero.")
    req=urllib.request.Request(address,method=method,headers={"User-Agent":"ScriptageLearning/1.0",**(headers or {})})
    try: response=(opener.open(req,timeout=timeout) if opener else urllib.request.urlopen(req,timeout=timeout))
    except urllib.error.HTTPError as exc: response=exc
    with response:
        data=response.read(limit+1) if limit else b""
        if len(data)>limit: raise ValueError("Response exceeds the byte limit.")
        return response.status,response.geturl(),{k.lower():v for k,v in response.headers.items()},data
def checked(address,timeout,**kwargs):
    status,final,headers,data=request(address,timeout,**kwargs)
    if not 200<=status<300: raise ValueError("HTTP status "+str(status))
    return final,headers,data
def urls(path):
    result=[line.strip() for line in Path(path).read_text(encoding="utf-8").splitlines() if line.strip()]
    if len(result)>100: raise ValueError("At most 100 URLs per run.")
    return result
def resolve(host,port,timeout):
    result=queue.Queue()
    def work():
        try: result.put((True,socket.getaddrinfo(host,port,type=socket.SOCK_STREAM)))
        except OSError as exc: result.put((False,str(exc)))
    threading.Thread(target=work,daemon=True).start()
    try: ok,value=result.get(timeout=timeout)
    except queue.Empty: raise ValueError("DNS lookup timed out.")
    if not ok: raise ValueError(value)
    return value
def connect(host,port,timeout):
    last=None
    for family,kind,proto,_,address in resolve(host,port,timeout):
        connection=socket.socket(family,kind,proto)
        connection.settimeout(timeout)
        try:
            connection.connect(address)
            return connection
        except OSError as exc:
            last=exc
            connection.close()
    raise OSError(str(last))

def main():
    parser = argparse.ArgumentParser(description='Take a finite availability sample')
    parser.add_argument("--output",required=True,help="New output file")
    parser.add_argument("--timeout",type=float,default=10,help="Per-operation timeout in seconds")
    parser.add_argument("--url",required=True)
    parser.add_argument("--attempts",type=int,default=3)
    parser.add_argument("--interval",type=float,default=1)
    args = parser.parse_args()
    if not 0<args.timeout<=120: raise ValueError("Timeout must be greater than zero and at most 120.")
    if not 1<=args.attempts<=10 or not 0<=args.interval<=30: raise ValueError("Use 1..10 attempts and 0..30 seconds between attempts.")
    result=[]
    for i in range(args.attempts):
        try:
            status,_,_,_=request(args.url,args.timeout,method="HEAD",limit=0)
            result.append({"attempt":i+1,"ok":200<=status<400,"status":status,"at":datetime.now(timezone.utc).isoformat()})
        except (OSError,ValueError) as exc: result.append({"attempt":i+1,"ok":False,"error":str(exc)})
        if i+1<args.attempts: time.sleep(args.interval)
    write_json(args.output,result)

if __name__ == "__main__":
    try:
        main()
    except (OSError, ValueError, KeyError, TypeError, ArithmeticError, IndexError, LookupError) as exc:
        raise SystemExit(str(exc))

Open source as plain text · Download source and sample files

How it works

  1. The command names every destination explicitly and refuses an existing output.
  2. At most ten requests are made; connection errors become records and 2xx/3xx final statuses count as available.
  3. With the fixture server running, all three attempts report ok true.

Make it your own

--interval

Set --interval 5 for five seconds between requests.

Adapt the processing

Edit main's ok status range if a service expects one exact status.

Change one setting at a time, run it again on sample data, and compare the output. The customization guide walks through this process.

If something goes wrong

Every attempt fails.

Start the server or correct the URL; this script does not keep running after its finite sample.

The destination already exists.

Choose a new --output path. The script deliberately refuses to replace an existing result.

Reference & next steps

These official references explain the underlying tools. The Scriptage example and guide are original instructional content. See verification notes for what has been checked and the limits of those checks.

Copies a template for your own notes. It does not send anything.