Simple script to check if a password is in Troy Hunts Pwned Password List as fast as possible
https://haveibeenpwned.com/Passwords
First the file must be processed.
awk -FS="" 'BEGIN{FS="";OFS="\n"}{fn=$1$2$3; print > fn}' hashfile.txt
This will divide the 30GB file into multiple files based on the first 3 characters of each hash, this will reduce the amount of data to search through significantly.
#!/bin/bash
grep $(echo -n $1 | sha1sum | awk '{print $1}' | tr [a-z] [A-Z]) $(echo -n $1 | sha1sum | awk -FS="" 'BEGIN{FS="";OFS="\n"}{print $1$2$3}' | tr [a-z] [A-Z])
This script will search through the file that matches the first 3 characters of the sha1 of the password. Runs in around 0.1s