johnfink8

johnfink8

johnfink8

'''

ItemProcessor

Written by: John Fink

Takes a list of objects, and a function, and some optional arguments to 
that function.  Runs that function (with those optional arguments) against
each element of the input list, in parallel.  Results from each function
call can be retrieved in a list of tuples (along with the input) with

johnfink8

import urllib2,os,cookielib
from urlparse import urlparse
from BeautifulSoup import BeautifulSoup
from django.core.cache import cache
from django.http import HttpResponse
import multiprocessing
m=multiprocessing.Manager()
links=m.list()
hosts=m.Queue()

johnfink8

MIT License

Copyright (c) 2012 John Fink

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions: