def worker(): while True: item = q.get() do_work(item) q.task_done() q = Queue() for i in range(num_worker_threads): t = Thread(target=worker) t.daemon = True t.start() for item in source(): q.put(item) q.join() # block until all tasks are done
The Basics of Python Multithreading and Queues
def do_stuff(q): while True: print q.get() q.task_done() q = Queue(maxsize=0) num_threads = 10 for i in range(num_threads): worker = Thread(target=do_stuff, args=(q,)) worker.setDaemon(True) worker.start()
Python Memoization: lru_cache
@functools.lru_cache(maxsize=128, typed=False)¶
Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.
Python Collections Counter
A Counter is a container that tracks how many times equivalent values are added.