Skip to content

Instantly share code, notes, and snippets.

@ian-kerins

ian-kerins/google_serp_scrapy_p1.py

Created April 12, 2021 13:54
Show Gist options
  • Save ian-kerins/f2398526e5791d37628f461ba4b9fed3 to your computer and use it in GitHub Desktop.
Save ian-kerins/f2398526e5791d37628f461ba4b9fed3 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
google_serp_scrapy_p1.py
from urllib.parse import urlparse
from urllib.parse import urlencode
def create_google_url(query, site=''):
google_dict = {'q': query, 'num': 100, }
if site:
web = urlparse(site).netloc
google_dict['as_sitesearch'] = web
return 'http://www.google.com/search?' + urlencode(google_dict)
return 'http://www.google.com/search?' + urlencode(google_dict)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment