How to use Fabric in a development environment

Fabric is a Python (2.5 or higher) library and command-line tool for streamlining
the use of SSH for application deployment or systems administration tasks.

.. Instead of running something like pip install MyApp and getting whatever libraries
come along, you can create a requirements file that includes all dependencies.

Example of a requirements.txt file

BeautifulSoup==3.2.0
python-dateutil==1.4.1
django==1.3.0
django-debug-toolbar==0.8.5
django-tagging==0.3
Markdown==2.0.1

 

Python Mechanize: web browsing

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

  • mechanize.Browser and mechanize.UserAgentBase implement the interface of urllib2.OpenerDirector, so:
    • any URL can be opened, not just http:
    • mechanize.UserAgentBase offers easy dynamic configuration of user-agent features like protocol, cookie, redirection and robots.txt handling, without having to make a new OpenerDirector each time, e.g. by callingbuild_opener().
  • Easy HTML form filling.
  • Convenient link parsing and following.
  • Browser history (.back() and .reload() methods).
  • The Referer HTTP header is added properly (optional).
  • Automatic observance of robots.txt.

Parallelism: Counting Jelly Beans Example

Counting Jelly Beans

Imagine you are presented with a large glass jar full of assorted jelly beans, and asked to count how many there are. Assuming you are able to count beans at an average rate of five per second, it would take you a little over ten minutes to determine that this particular jar contains 3,027 jelly beans.

If four of your friends offer to help with the task, you could choose from a number of potential strategies, but let’s consider one that closely mirrors the sort of strategy that SQL Server would adopt. You seat your friends around a table with the jar at its centre, and a single scoop to remove beans from the jar. You ask them to help themselves to a scoop of beans whenever they need more to count. Each friend is also given a pen and a piece of paper, to keep a running total of the number of beans they have counted so far.

Once a person finishes counting and finds the jar empty, they pass their individual bean count total to you. As you collect each subtotal, you add it to a grand total. When you have received a subtotal from each of your friends, the task is complete. With four people counting beans simultaneously the whole task is completed in around two and a half minutes – a four-fold improvement over counting them all yourself. Of course, four people still worked for a totalof ten minutes (plus the few seconds it took you to add the last subtotal to the grand total).

This particular task is well-suited to parallel working because each person is able to work concurrently andindependently. The desired result is obtained much more quickly, without doing much more work overall