Distil is the first easy and accurate way to defend websites against malicious bots. See it in action.
Jesse Noller: Rackspace Open Source Developer
The Future of Text Symposium
Join us for the 6th annual symposium in Silicon Valley on the 25th of August 2016!
python simple-requests
The goal of this library is to allow you to get the performance benefit of asynchronous requests, without needing to use any asynchronous coding paradigms. It is built on gevent and requests.
Usage
from simple_requests import Requests # Creates a session and thread pool requests = Requests() # Sends one simple request; the response is returned synchronously. login_response = requests.one('http://cat-videos.net/login?user=fanatic&password=c4tl0v3r') # Cookies are maintained in this instance of Requests, so subsequent requests # will still be logged-in. profile_urls = [ 'http://cat-videos.net/profile/mookie', 'http://cat-videos.net/profile/kenneth', 'http://cat-videos.net/profile/itchy' ] # Asynchronously send all the requests for profile pages for profile_response in requests.swarm(profile_urls): # Asynchronously send requests for each link found on the profile pages # These requests take precedence over those in the outer loop to minimize overall waiting # Order doesn't matter this time either, so turn that off for a slight performance gain for friends_response in requests.swarm(profile_response.links, maintainOrder = False): # Do something intelligent with the responses, like using # regex to parse the HTML (see http://stackoverflow.com/a/1732454 friends_response.html