Skip to content

Instantly share code, notes, and snippets.

@antitree

antitree/gist:768e1beb1f2d7d9ee9eb

Last active August 29, 2015 14:23
Download ZIP

Revisions

  1. antitree revised this gist Jun 29, 2015. 1 changed file with 8 additions and 4 deletions.
    12 changes: 8 additions & 4 deletions gistfile1.py
    View file
    Original file line number Diff line number Diff line change
    @@ -8,9 +8,13 @@
    print("Getting: %s" % test_url)
    r = session.get(test_url)
    s = bs.BeautifulSoup(r.text)
    result=s.find("meta",attrs={"http-equiv":"refresh"})
    wait,text = result["content"].split(';')
    results_url = text[4:]
    #result=s.find("meta",attrs={"http-equiv":"refresh"})
    results_url = (s('p')[1].a.get('href'))
    print("Getting: %s" % results_url)
    r = session.get(results_url)
    print(r.text)
    s = bs.BeautifulSoup(r.text)
    print("#" * 30)
    print("Results:")
    print("#" * 30)
    for p in (s('p')):
    print(p.text)
  2. antitree created this gist Jun 29, 2015.
    16 changes: 16 additions & 0 deletions gistfile1.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,16 @@
    import requesocks
    import BeautifulSoup as bs

    base_url = 'http://treebind.antitree.com:8000'
    test_url = base_url + '/test'
    session = requesocks.session()
    session.proxies = {'http': 'socks5://127.0.0.1:9050'}
    print("Getting: %s" % test_url)
    r = session.get(test_url)
    s = bs.BeautifulSoup(r.text)
    result=s.find("meta",attrs={"http-equiv":"refresh"})
    wait,text = result["content"].split(';')
    results_url = text[4:]
    print("Getting: %s" % results_url)
    r = session.get(results_url)
    print(r.text)