Python Scripts Every Web Developer Should Keep Handy

You can build entire platforms in Python. You can also use it as a sharp pocket knife next to PHP and JavaScript. For freelancers and small agencies, that second use may be more valuable day to day: checking 200 URLs after a migration, cleaning a client CSV, resizing a folder of images, or skimming logs for 500s after deploy.

This guide is not “learn machine learning in a weekend”. It is a set of scripts and habits that earn their place on a web developer’s machine in 2026.

Why Python sits well next to PHP/JS work

• Readable syntax — easy to revisit in six months
• Excellent standard library for files, CSV, JSON, zip, HTTP
• Rich ecosystem (requests, httpx, beautifulsoup4, pillow)
• Great for one-off tools you do not want to deploy as a full app

Install a current Python 3, use virtual environments, and keep project dependencies tiny.

python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS/Linux
source .venv/bin/activate
pip install requests beautifulsoup4 pillow

1) Bulk URL status check after launch

import csv
import sys
import time
import requests

def check_urls(path, timeout=10):
    out = []
    with open(path, newline='', encoding='utf-8') as f:
        for row in csv.reader(f):
            if not row:
                continue
            url = row[0].strip()
            if not url or url.startswith('#'):
                continue
            try:
                r = requests.get(url, timeout=timeout, allow_redirects=True)
                out.append((url, r.status_code, r.url))
                print(r.status_code, url)
            except requests.RequestException as e:
                out.append((url, 'ERR', str(e)))
                print('ERR', url, e)
            time.sleep(0.15)  # be polite
    return out

if __name__ == '__main__':
    results = check_urls(sys.argv[1])
    with open('url-report.csv', 'w', newline='', encoding='utf-8') as f:
        w = csv.writer(f)
        w.writerow(['url', 'status', 'final_or_error'])
        w.writerows(results)

Feed it a CSV of critical templates: home, services, contact, blog posts, legal pages. Catch 404s before the client does.

2) Sitemap diff between staging and live

import sys
import xml.etree.ElementTree as ET
import requests

NS = {'sm': 'http://www.sitemaps.org/schemas/sitemap/0.9'}

def urls_from_sitemap(url):
    xml = requests.get(url, timeout=20).text
    root = ET.fromstring(xml)
    # urlset
    locs = [loc.text.strip() for loc in root.findall('sm:url/sm:loc', NS) if loc.text]
    if locs:
        return set(locs)
    # sitemap index — follow children lightly
    out = set()
    for loc in root.findall('sm:sitemap/sm:loc', NS):
        if loc.text:
            out |= urls_from_sitemap(loc.text.strip())
    return out

live = urls_from_sitemap(sys.argv[1])
stage = urls_from_sitemap(sys.argv[2])
print('Only on live:', len(live - stage))
for u in sorted(live - stage)[:50]:
    print('  ', u)
print('Only on staging:', len(stage - live))
for u in sorted(stage - live)[:50]:
    print('  ', u)

3) Clean a messy client CSV

import csv
from pathlib import Path

src = Path('leads-raw.csv')
dst = Path('leads-clean.csv')

with src.open(encoding='utf-8-sig', newline='') as f_in, dst.open('w', encoding='utf-8', newline='') as f_out:
    reader = csv.DictReader(f_in)
    fieldnames = ['name', 'email', 'phone']
    writer = csv.DictWriter(f_out, fieldnames=fieldnames)
    writer.writeheader()
    seen = set()
    for row in reader:
        email = (row.get('email') or row.get('Email') or '').strip().lower()
        if not email or '@' not in email or email in seen:
            continue
        seen.add(email)
        writer.writerow({
            'name': (row.get('name') or row.get('Name') or '').strip(),
            'email': email,
            'phone': (row.get('phone') or row.get('Phone') or '').strip(),
        })
print('Wrote', dst)

4) Batch rename and lightly process images

from pathlib import Path
from PIL import Image

folder = Path('incoming')
out = Path('web-ready')
out.mkdir(exist_ok=True)

for i, path in enumerate(sorted(folder.glob('*.jpg')), start=1):
    img = Image.open(path)
    img = img.convert('RGB')
    img.thumbnail((1600, 1600))
    target = out / f"gallery-{i:03d}.jpg"
    img.save(target, 'JPEG', quality=82, optimize=True)
    print(path.name, '->', target.name)

For production sites you may prefer dedicated image CDNs or build pipelines — but for a quick client handoff folder, this is gold.

5) Skim access logs for spikes

import re
from collections import Counter
from pathlib import Path

line_re = re.compile(r'"(\w+) ([^ ]+) HTTP/[^"]+" (\d{3})')
counter = Counter()
for line in Path('access.log').read_text(errors='ignore').splitlines():
    m = line_re.search(line)
    if not m:
        continue
    method, path, status = m.groups()
    if status.startswith('5'):
        counter[(status, path)] += 1

for (status, path), n in counter.most_common(20):
    print(n, status, path)

6) A tiny internal API with FastAPI (optional)

When a script becomes a tool the team hits daily, wrap it:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, HttpUrl
import requests

app = FastAPI()

class CheckIn(BaseModel):
    url: HttpUrl

@app.post('/check')
def check(body: CheckIn):
    try:
        r = requests.get(str(body.url), timeout=10)
    except requests.RequestException as e:
        raise HTTPException(status_code=502, detail=str(e))
    return {'status_code': r.status_code, 'final_url': r.url}

Lock it behind VPN or auth. Do not expose open relay URL checkers to the public internet.

Polite automation and legal common sense

• Respect robots.txt and site terms when scraping
• Throttle requests; identify your tool with a clear User-Agent for internal ops
• Never store passwords in scripts — use environment variables
• Do not exfiltrate personal data into random cloud notebooks

import os
API_KEY = os.environ.get('CLIENT_API_KEY')
if not API_KEY:
    raise SystemExit('Set CLIENT_API_KEY')

How to organise a personal “devtools” repo

• README.md with one-line purpose per script
• requirements.txt pinned lightly
• Sample input files (anonymised)
• No client secrets committed

If you have done a task twice, the third time is a candidate for a 30-line script.

FAQ

Should I rewrite my PHP site in Python?
Usually no. Use the best tool for delivery. Python shines for tooling and data; PHP/WordPress still shine for many business websites.

Is scraping Google or directories a good idea?
Often against terms and fragile. Prefer official APIs and first-party data.

What should I learn first?
Paths, virtualenv, requests, CSV, JSON, and reading error messages calmly.

Need automation around a launch or migration?

I help UK businesses ship websites and the boring-but-critical checks that keep launches calm.

Book a free enquiry →