inchoate/download_site.md

Last active October 10, 2024 13:13

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/inchoate/064435afeb2a68a5bac7c2646463682f.js"></script>
Save inchoate/064435afeb2a68a5bac7c2646463682f to your computer and use it in GitHub Desktop.

Simple Site Downloader

Raw

#!/bin/zsh

# Usage:
#     ./download_site.sh {URL}


url=$1
output_dir="./sites"

echo "Starting download for: ${url}"
wget --recursive \
  --directory-prefix=$output_dir \
  --level=2 \
  --convert-links \
  --no-parent \
  --adjust-extension \
  --wait=2 \
  --random-wait \
  ${url}
echo "Finished download for: ${url}"

# --span-hosts