Skip to content

Instantly share code, notes, and snippets.

@amferraz
Forked from anonymous/gist:5142892
Last active December 14, 2015 20:19
Show Gist options
  • Select an option

  • Save amferraz/5143268 to your computer and use it in GitHub Desktop.

Select an option

Save amferraz/5143268 to your computer and use it in GitHub Desktop.
from scrapy.utils.project import get_project_settings
def stop_reactor():
reactor.stop() #Stops reactor to prevent script from hanging
if __name__ == '__main__':
#Handles engine_stopped to stop twisted reactor
dispatcher.connect(stop_reactor, signal=signals.engine_stopped)
spider = EstiloMASpider()
crawler = Crawler(get_project_settings()) #Loads the settings
crawler.configure()
crawler.crawl(spider)
crawler.start()
#Start log and twisted reactor
log.start()
reactor.run()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment