Skip to content

Instantly share code, notes, and snippets.

@blovett
Forked from kamaljoshi/check_cloudflare.rb
Created February 25, 2017 03:41
Show Gist options
  • Save blovett/1fb17d6c148dcdb331d99bd59654cc1c to your computer and use it in GitHub Desktop.
Save blovett/1fb17d6c148dcdb331d99bd59654cc1c to your computer and use it in GitHub Desktop.
This script collects domains from your Chrome history for the default profile and checks to see if they are being proxied through Cloudflare by checking presence of a header.
# Except sqlite3 all the libraries are standard that should be present with a Ruby installation.
# If you don't have sqlite3 installed. Use `gem install sqlite3`
require 'fileutils'
require 'sqlite3'
require 'uri'
require 'net/http'
require 'set'
require 'thread'
chrome_history_location = "#{ENV['HOME']}/Library/Application\ Support/Google/Chrome/Default/History"
temp_location = "/tmp/Chrome_history"
FileUtils.cp(chrome_history_location, temp_location)
sqlite_db = SQLite3::Database.new temp_location
chrome_history = sqlite_db.execute('SELECT DISTINCT(url) FROM urls;').flatten; nil
FileUtils.rm([temp_location])
domain_set = Set.new
cloudflare_set = Set.new
query_uris = Array.new
chrome_history.each do |url|
host = URI.parse(url).host rescue nil
query_uris += [URI::HTTPS.build({host: host}), URI::HTTP.build({host: host})] if !domain_set.include?(host) && !host.nil?
domain_set.add(host)
end; nil
uri_mutex, set_mutex, read_mutex = Mutex.new, Mutex.new, Mutex.new
(1..16).map do
Thread.new(query_uris, cloudflare_set) do |query_uris, cloudflare_set|
while !(uri = uri_mutex.synchronize { query_uris.pop }).nil?
cf_header_present = !Net::HTTP.get_response(uri)['cf-ray'].nil? rescue nil
read_mutex.synchronize{ print("#{query_uris.length} remaining\r") }
set_mutex.synchronize { cloudflare_set.add(uri.host) } if cf_header_present
end
end
end.each(&:join); nil
p cloudflare_set.to_a.sort
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment