brew install phantomjs <span class="c"><span class="hljs-comment"># or npm -g phantomjs </span></span> pip install selenium<span class="kn"><span class="hljs-keyword">from</span></span> <span class="nn">selenium</span> <span class="kn"><span class="hljs-keyword">import</span></span> <span class="n">webdriver</span> <span class="n">depot</span> <span class="o">=</span> <span class="n">DepotManager</span><span class="o">.</span><span class="n">get</span><span class="p">()</span> <span class="n">driver</span> <span class="o">=</span> <span class="n">webdriver</span><span class="o">.</span><span class="n">PhantomJS</span><span class="p">()</span> <span class="n">driver</span><span class="o">.</span><span class="n">set_window_size</span><span class="p">(</span><span class="mi"><span class="hljs-number">1024</span></span><span class="p">,</span> <span class="mi"><span class="hljs-number">768</span></span><span class="p">)</span> <span class="c"><span class="hljs-comment"># set the window size that you need </span></span> <span class="n">driver</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s"><span class="hljs-string">'https://github.com'</span></span><span class="p">)</span> <span class="n">driver</span><span class="o">.</span><span class="n">save_screenshot</span><span class="p">(</span><span class="s"><span class="hljs-string">'github.png'</span></span><span class="p">)</span>
Python Queue — A synchronized queue class
def worker(): while True: item = q.get() do_work(item) q.task_done() q = Queue() for i in range(num_worker_threads): t = Thread(target=worker) t.daemon = True t.start() for item in source(): q.put(item) q.join() # block until all tasks are done
The Basics of Python Multithreading and Queues
def do_stuff(q): while True: print q.get() q.task_done() q = Queue(maxsize=0) num_threads = 10 for i in range(num_threads): worker = Thread(target=do_stuff, args=(q,)) worker.setDaemon(True) worker.start()
Python Memoization: lru_cache
@functools.lru_cache(maxsize=128, typed=False)¶
Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.