Calculating Interest Rates with Excel

Here’s an article I wrote for Microfinance Transparency in February 24, 2010:

Chuck Waterfield and Alexandra Fiorillo, MFTransparency’s CEO and VP respectively, have been doing many presentations about how interest rates can be calculated using our excel tool, but we haven’t yet featured a story on our blog about our data collection process and our corresponding excel tool. Although technical, interest rate calculations are really at the heart of MFTransparency’s mission and calculating accurate interest rates is vital to providing transparent pricing data. So today, I would like to give you a brief demonstration of the IRR and XIRR Excel functions, as a way to provide background for the techniques we’ve used to automate interest rate calculations on our website.

For those of you less familiar with excel, this spreadsheet software offers numerous formulas allowing quick and easy calculations within each spreadsheet. As it is particularly geared towards financial use, there are ready-made formulas specifically meant for calculating interest rates. The most basic (but still powerful) calculation is the internal rate of return.

IRR() : Internal Rate of Return

The internal rate of return formula is capable of taking a cash flow and returning the per-period interest rate. It assumes equal lengths of time between each amount in the cash flow. Let’s first start with a sample spreadsheet of loan payments, and use the IRR function to calculate the interest rate.

Screenshot: Excel IRR function

You can see that by applying the IRR formula, we get an “Internal Rate of Return” for the loan. This IRR can then be multiplied by the number of periods in a year to get the APR. Annual Percentage Rate is the standardized format most commonly used in the United States.

  • APR = IRR * n, where n is the number of payments per year.
  • 24.09% = 0.0200757 * 12

The EIR takes into account the effect of compound interest and can be calculated using the formula. This is the standardized interest rate often reported in European countries:

  • EIR = ((1+IRR)^n) -1)
  • 26.94% = ((1+ 0.0200757)^12)-1)

The IRR function is sufficient when there are equal (or near equal) periods between repayments, but what about when repayments occur irregularly? Prior to Excel 2007, there was no easy solution… but thankfully the wizards at microsoft have now provided us with a solution:

XIIR(): accounts for actual payment dates

As mentioned above, the XIRR function is useful for loans with irregular repayment schedules, and is only available in newer versions of Excel (2007) and in recent versions of Open Office.

Excel 2007 XIIR function

Notice that the XIRR function takes into account the payment dates in addition to the payment amounts. It actually provides us with the EIR (so annualized interest rate WITH compounding) for the cash flow in question. We won’t get into the math behind this, but suffice it to say that this formula is powerful, and a significant step up in allowing accurate calculations of interest rates. If you download the sample spreadsheet and play with the numbers, you can see the effect that an early first payment and a short month (February) have on the XIIR result.

  • better EIR = Excel XIIR()
  • XIRR() = 32.16%

So XIRR is a more accurate way to calculate the interest rate because it takes into account both actual payment dates and the effect of compound interest.

These tools are what allow MFTransparency to calculate accurate interest rates that are comparable between MFIs, despite different/irregular repayment schedules, additional fees, etc. So while they may seem mundane, they are actually the crux of transparent pricing!

In a future post, I’ll explain how software programmers can efficiently add advanced Excel 2007-like XIRR calculations to their software programs.

Further Information

For those of you who would like to know more about interest rate calculation, I encourage you to check out the following:

The next post in this series, “Calculating Interest Rates Using Newton’s Method” is a more advanced version of this article that explains the algorithm behind the XIIR formula, and how this technique can be applied in programming languages like C#, python, or Java.

P.S. For investors, Excel’s XIRR feature can also be used to calculate a Personal Investment Rate of Return, which is more relevant than the values that appear in a fund’s prospectus because it takes into account the investor’s actual purchase history.

Calculating interest rates using newton’s method

Here’s a post I wrote for Microfinance Transparency on March 5, 2010

In a previous blog post, I described how to use a spreadsheet like Microsoft Excel to calculate interest rates. Again, interest rate calculations are at the core of MFTransparency’s ability to provide accurate data that can be compared across various products offered by numerous MFIs. In the last post we looked at Excel’s IRR and XIRR functions and concluded that XIRR is more accurate because it takes into account the actual payment dates of the loan and thus allows us to calculate annualized interest rates even with irregular repayment schedules.

But for the more technical among us, I realize that even this may not be sufficient. Today I’m going to demonstrate how to write a computer program that is as accurate as Excel 2007’s XIRR function. This article is likely to be of less broad interest, but it provides transparency into how we will calculate interest rates for future data collection trips; and it may be useful for MFIs that wish to automate interest rate calculations for a larger data set than can be handled with Excel.

Let’s start with the EIR formula and describe two techniques.

  • EIR = cf*(1+rate)^n

cf = cashflow, n=number of periods/year

Simple Guess and Check

The first technique is to make a guess about the interest rate and then run the numbers through the EIR formula to see how close you are. You then iterate, guessing somewhere in the middle of your previous guesses, or widening your area by doubling. The advantage of this technique is that it is simple and it gets you the right answer eventually (or at least fairly accurately given enough guesses).

Newton’s method

A more advanced way to solve the EIR formula is to use Calculus. It’s still a “guessing” technique, but it is much more efficient and elegant.

It’s easier to visualize this technique if we draw a graph and plot an initial guess, with the goal of finding the point on the graph where it crosses the x axis.

We start by making an initial guess and then figuring out what the “tangent” line at that point would be. This is the same thing as the derivative of the EIR calculation:

  • EIR = cf*(1+rate)^n
  • f’rate = cf*n*(rate+1)^(n-1) = pink tangent line is the derivative

cf = cashflow, n=number of periods/year

We can then figure out where the tangent line intersects the x axis and use that to make a much more accurate second guess.

The speed advantage the Newton-Raphson method has over a simple guess and check technique is quite remarkable. It is common to be able to achieve a result that matches Excel to within 8 decimal places in 5-10 iterations.

Implementing Newton-Raphson

Fortunately for me, I didn’t have to implement the entire Newton-Raphson algorithm myself because the programming language I use already has a library to do this. I just give it the EIR function and the derivative function and it does the rest.

For the programmers out there, here are a few links to implementations in various languages:

  • C#: uses bisection rather than Newton’s method
  • Java
  • Python

Related Information:

For those of you who would like to know more, please explore the following links:

 

Next Article:

Calculating Effective Interest Rates Using Cashflow Discounting

Here’s a post I wrote for Microfinance Transparency on October 26, 2010:

In a previous post, I described the technique that computer programs like Microsoft Excel use to calculate the XIRR (effective interest rate) as a very smart version of “guess and check.” The post on Newton’s Method described how the “guessing” part works, but it did not describe how the computer is able to finally verify when it has the correct EIR figure — the “check” part.

In today’s post, I’m going to describe the process that a computer program uses to generate a discounted cashflow, a method of calculating the value of a cashflow that uses the time-value of money. By adding up the discounted cashflows we are able to determine whether we have the correct EIR.

I’m going to start with a sample loan of $4,825.00 that was disbursed on the 28th of the month but is paid back on the 16th of the month every month for about a year. The fact that the disbursement date is not exactly one month prior to the first repayment means that a simple IRR formula can not be used for accurate results, and the calculation must take all the actual dates into account. The exact details are at right.

How to Check an EIR

The first step in the process is to come up with a guess. For my example, I’ll start with a guess of 30 percent. The next step is then to take that 30% rate and plug it into the discounting formula for each date of the loan:

Discounted cashflow = cashflow * (rate +1)^(-days/365)

This gives us the discounted value of the cashflow for each date period. We then add up all the discounted cashflows to produce a total. This total should be 0 if the rate is correct, indicating that discounted cashflow is equivalent to the nominal cashflow at the specified rate. If it is not 0, we guess again, until we get closer and closer to 0. (For our purposes we decided that 8 decimal places of accuracy is good enough!) Below, you will see that Newton’s method is able to guess the correct EIR in only 5 guesses!

About the Table of Iterations

The tables below list a series of payments, sorted by date. The second column is a calculation of how many days between the payment and original disbursement. The “cashflow” column refers to the total value of all disbursements and payments on that date. The “Discounted” column indicates the value of the cashflow after it is discounted by the discounting formula; and the final column illustrates how to plug the numbers into the discounting formula.

Iteration 1: 30%

  # DATE          # DAYS  CASHFLOW      DISCOUNTED    Formula: cf * (rate + 1)^(-days/365)
   --------------------------------------------------------------------------------------------
   0 2010-06-28      days  -4825.00  -4825.00000000 = -4825.00 * ((0.3000000000 + 1)^(-0 /365))
   1 2010-07-16   18 days     48.00     47.38295189 =    48.00 * ((0.3000000000 + 1)^(-18 /365))
   2 2010-08-16   49 days    492.00    474.97264481 =   492.00 * ((0.3000000000 + 1)^(-49 /365))
   3 2010-09-16   80 days    492.00    464.50588150 =   492.00 * ((0.3000000000 + 1)^(-80 /365))
   4 2010-10-16  110 days    492.00    454.59641896 =   492.00 * ((0.3000000000 + 1)^(-110 /365))
   5 2010-11-16  141 days    492.00    444.57867758 =   492.00 * ((0.3000000000 + 1)^(-141 /365))
   6 2010-12-16  171 days    492.00    435.09432889 =   492.00 * ((0.3000000000 + 1)^(-171 /365))
   7 2011-01-16  202 days    492.00    425.50634649 =   492.00 * ((0.3000000000 + 1)^(-202 /365))
   8 2011-02-16  233 days    492.00    416.12965025 =   492.00 * ((0.3000000000 + 1)^(-233 /365))
   9 2011-03-16  261 days    492.00    407.83810625 =   492.00 * ((0.3000000000 + 1)^(-261 /365))
  10 2011-04-16  292 days    492.00    398.85075724 =   492.00 * ((0.3000000000 + 1)^(-292 /365))
  11 2011-05-16  322 days    492.00    390.34193788 =   492.00 * ((0.3000000000 + 1)^(-322 /365))
  12 2011-06-16  353 days    492.00    381.74014425 =   492.00 * ((0.3000000000 + 1)^(-353 /365))
  13 2011-07-16  383 days    488.00    370.55898273 =   488.00 * ((0.3000000000 + 1)^(-383 /365))
                             total:    287.09682872

Iteration 2: 42.8 %

  # DATE          # DAYS  CASHFLOW      DISCOUNTED    Formula: cf * (rate + 1)^(-days/365)
   --------------------------------------------------------------------------------------------
   0 2010-06-28      days  -4825.00  -4825.00000000 = -4825.00 * ((0.4289338537 + 1)^(-0 /365))
   1 2010-07-16   18 days     48.00     47.16249843 =    48.00 * ((0.4289338537 + 1)^(-18 /365))
   2 2010-08-16   49 days    492.00    468.98100804 =   492.00 * ((0.4289338537 + 1)^(-49 /365))
   3 2010-09-16   80 days    492.00    454.97741867 =   492.00 * ((0.4289338537 + 1)^(-80 /365))
   4 2010-10-16  110 days    492.00    441.82381340 =   492.00 * ((0.4289338537 + 1)^(-110 /365))
   5 2010-11-16  141 days    492.00    428.63112723 =   492.00 * ((0.4289338537 + 1)^(-141 /365))
   6 2010-12-16  171 days    492.00    416.23920530 =   492.00 * ((0.4289338537 + 1)^(-171 /365))
   7 2011-01-16  202 days    492.00    403.81046552 =   492.00 * ((0.4289338537 + 1)^(-202 /365))
   8 2011-02-16  233 days    492.00    391.75284304 =   492.00 * ((0.4289338537 + 1)^(-233 /365))
   9 2011-03-16  261 days    492.00    381.17184633 =   492.00 * ((0.4289338537 + 1)^(-261 /365))
  10 2011-04-16  292 days    492.00    369.79020416 =   492.00 * ((0.4289338537 + 1)^(-292 /365))
  11 2011-05-16  322 days    492.00    359.09940022 =   492.00 * ((0.4289338537 + 1)^(-322 /365))
  12 2011-06-16  353 days    492.00    348.37683266 =   492.00 * ((0.4289338537 + 1)^(-353 /365))
  13 2011-07-16  383 days    488.00    335.55465113 =   488.00 * ((0.4289338537 + 1)^(-383 /365))
                             total:     22.37131413

Iteration 3: 44.07%

  # DATE          # DAYS  CASHFLOW      DISCOUNTED    Formula: cf * (rate + 1)^(-days/365)
   --------------------------------------------------------------------------------------------
   0 2010-06-28      days  -4825.00  -4825.00000000 = -4825.00 * ((0.4407450200 + 1)^(-0 /365))
   1 2010-07-16   18 days     48.00     47.14335675 =    48.00 * ((0.4407450200 + 1)^(-18 /365))
   2 2010-08-16   49 days    492.00    468.46303106 =   492.00 * ((0.4407450200 + 1)^(-49 /365))
   3 2010-09-16   80 days    492.00    454.15728021 =   492.00 * ((0.4407450200 + 1)^(-80 /365))
   4 2010-10-16  110 days    492.00    440.72909533 =   492.00 * ((0.4407450200 + 1)^(-110 /365))
   5 2010-11-16  141 days    492.00    427.27027316 =   492.00 * ((0.4407450200 + 1)^(-141 /365))
   6 2010-12-16  171 days    492.00    414.63706332 =   492.00 * ((0.4407450200 + 1)^(-171 /365))
   7 2011-01-16  202 days    492.00    401.97502997 =   492.00 * ((0.4407450200 + 1)^(-202 /365))
   8 2011-02-16  233 days    492.00    389.69966512 =   492.00 * ((0.4407450200 + 1)^(-233 /365))
   9 2011-03-16  261 days    492.00    378.93475953 =   492.00 * ((0.4407450200 + 1)^(-261 /365))
  10 2011-04-16  292 days    492.00    367.36298994 =   492.00 * ((0.4407450200 + 1)^(-292 /365))
  11 2011-05-16  322 days    492.00    356.50107412 =   492.00 * ((0.4407450200 + 1)^(-322 /365))
  12 2011-06-16  353 days    492.00    345.61437612 =   492.00 * ((0.4407450200 + 1)^(-353 /365))
  13 2011-07-16  383 days    488.00    332.66871441 =   488.00 * ((0.4407450200 + 1)^(-383 /365))
                             total:      0.15670905

Iteration 4: 44.0828927%

   # DATE          # DAYS  CASHFLOW      DISCOUNTED    Formula: cf * (rate + 1)^(-days/365)
   --------------------------------------------------------------------------------------------
   0 2010-06-28      days  -4825.00  -4825.00000000 = -4825.00 * ((0.4408289273 + 1)^(-0 /365))
   1 2010-07-16   18 days     48.00     47.14322135 =    48.00 * ((0.4408289273 + 1)^(-18 /365))
   2 2010-08-16   49 days    492.00    468.45936857 =   492.00 * ((0.4408289273 + 1)^(-49 /365))
   3 2010-09-16   80 days    492.00    454.15148325 =   492.00 * ((0.4408289273 + 1)^(-80 /365))
   4 2010-10-16  110 days    492.00    440.72136020 =   492.00 * ((0.4408289273 + 1)^(-110 /365))
   5 2010-11-16  141 days    492.00    427.26066094 =   492.00 * ((0.4408289273 + 1)^(-141 /365))
   6 2010-12-16  171 days    492.00    414.62575064 =   492.00 * ((0.4408289273 + 1)^(-171 /365))
   7 2011-01-16  202 days    492.00    401.96207459 =   492.00 * ((0.4408289273 + 1)^(-202 /365))
   8 2011-02-16  233 days    492.00    389.68517791 =   492.00 * ((0.4408289273 + 1)^(-233 /365))
   9 2011-03-16  261 days    492.00    378.91897968 =   492.00 * ((0.4408289273 + 1)^(-261 /365))
  10 2011-04-16  292 days    492.00    367.34587501 =   492.00 * ((0.4408289273 + 1)^(-292 /365))
  11 2011-05-16  322 days    492.00    356.48275888 =   492.00 * ((0.4408289273 + 1)^(-322 /365))
  12 2011-06-16  353 days    492.00    345.59491081 =   492.00 * ((0.4408289273 + 1)^(-353 /365))
  13 2011-07-16  383 days    488.00    332.64838595 =   488.00 * ((0.4408289273 + 1)^(-383 /365))
                             total:      0.00000779

Iteration 5: 44.0828931%

   # DATE          # DAYS  CASHFLOW      DISCOUNTED    Formula: cf * (rate + 1)^(-days/365)
   --------------------------------------------------------------------------------------------
   0 2010-06-28      days  -4825.00  -4825.00000000 = -4825.00 * ((0.4408289314 + 1)^(-0 /365))
   1 2010-07-16   18 days     48.00     47.14322135 =    48.00 * ((0.4408289314 + 1)^(-18 /365))
   2 2010-08-16   49 days    492.00    468.45936839 =   492.00 * ((0.4408289314 + 1)^(-49 /365))
   3 2010-09-16   80 days    492.00    454.15148296 =   492.00 * ((0.4408289314 + 1)^(-80 /365))
   4 2010-10-16  110 days    492.00    440.72135982 =   492.00 * ((0.4408289314 + 1)^(-110 /365))
   5 2010-11-16  141 days    492.00    427.26066046 =   492.00 * ((0.4408289314 + 1)^(-141 /365))
   6 2010-12-16  171 days    492.00    414.62575008 =   492.00 * ((0.4408289314 + 1)^(-171 /365))
   7 2011-01-16  202 days    492.00    401.96207394 =   492.00 * ((0.4408289314 + 1)^(-202 /365))
   8 2011-02-16  233 days    492.00    389.68517719 =   492.00 * ((0.4408289314 + 1)^(-233 /365))
   9 2011-03-16  261 days    492.00    378.91897890 =   492.00 * ((0.4408289314 + 1)^(-261 /365))
  10 2011-04-16  292 days    492.00    367.34587416 =   492.00 * ((0.4408289314 + 1)^(-292 /365))
  11 2011-05-16  322 days    492.00    356.48275797 =   492.00 * ((0.4408289314 + 1)^(-322 /365))
  12 2011-06-16  353 days    492.00    345.59490985 =   492.00 * ((0.4408289314 + 1)^(-353 /365))
  13 2011-07-16  383 days    488.00    332.64838494 =   488.00 * ((0.4408289314 + 1)^(-383 /365))
                             total:      0.00000000

Summary

As you can see, the process of generating the exact EIR, is fairly simple:

  1. formulate a guess
  2. check the guess using the discounting formula
  3. repeat until the discounted cashflow equals zero

Knowing this procedure allows us to calculate any EIR, for any loan, no matter what the breakdown of fees, and no matter how irregularly spaced the payment dates.

If you are designing your own EIR calculator, you don’t have to use Newton’s method. Any “guess and check” technique will work, even if it requires additional guesses. The key part is to discount your cashflow using the formula:

discounted cashflow = cashflow * (rate + 1)^(-days/365)

A Note about APR

This technique is used to calculate the EIR very exactly. The APR formula is simpler and does not perform this type of cashflow discounting. APR is technically less accurate than EIR, but in many cases it is still a useful way of communicating the price of a loan. For more information on the IRR formula used to calculate the APR, see “Calculating Interest Rates with Excel“.

Further Information

 

 

How an Engineering Student’s Question Prevented a NYC Skyscraper from Falling Down

Citigroup Center

Engineering crisis of 1978 (from Wikipedia)

Due to material changes during construction, the building as initially completed was structurally unsound. William LeMessurier‘s original design for the chevron load braces used welded joints. To save money, Bethlehem Steel changed the plans in 1974 to use bolted joints, which was accepted by LeMessurier’s office but not known to the engineer himself.[22] Furthermore, according to The New Yorker, LeMessurier originally only needed to calculate wind loads from perpendicular winds under the building code; in typical buildings, loads from quartering winds at the corners would be less.[22][159] In June 1978, after an inquiry from Princeton University engineering student Diane Hartley, LeMessurier recalculated the wind loads on the building with quartering winds.[159][f] He found that, for four of the eight tiers of chevrons, such winds would create a 40 percent increase in wind loads and a 160 percent increase in load at the bolted joints.[22]

Citicorp Center’s use of bolted joints and the increased loads from quartering winds would not have caused concern if these issues had been isolated from each other. However, the combination of the two findings prompted LeMessurier to run tests on the structural safety.[103] The original welded-joint design could withstand the load from straight-on and quartering winds, but a 75-mile-per-hour (121 km/h) hurricane force quartering wind would exceed the strength of the bolted-joint chevrons.[99] With the tuned mass damper active, LeMessurier estimated that a wind capable of toppling the building would occur on average once every 55 years.[162][161] If the tuned mass damper could not function due to a power outage, a wind strong enough to cause the building’s collapse would occur once every 16 years on average.[162] LeMessurier also discovered that his firm had used New York City’s truss safety factor of 1:1 instead of the column safety factor of 1:2.[99]

LeMessurier debated how to address the issue before ultimately contacting Stubbins’s lawyer. LeMessurier then contacted Citicorp’s lawyers, the latter of which hired Leslie E. Robertson as an expert adviser.[163] Citicorp accepted LeMessurier’s proposal to weld steel plates over the bolted joints, and Karl Koch Erecting was hired for the welding process.[104] Very few people were made aware of the issue, besides Citicorp leadership, mayor Ed Koch, acting buildings commissioner Irving E. Minkin, and the head of the welder’s union.[99][104] Starting in August 1978, construction crews installed the welded panels at night. Officials made no public mention of any possible structural issues, and the city’s three major newspapers had gone on strike.[88][104] The work continued despite the threat of Hurricane Ella several weeks after the repairs started.[99][164] Repairs were completed in October 1978, before the media resumed publishing. LeMessurier claimed a wind strong enough to topple the building would only occur once every 700 years.[88][165] Stubbins and LeMessurier covered all of the repair costs, which were estimated to be several million dollars.[165] Since no structural failure occurred, the work was only publicized in a lengthy article in The New Yorker in 1995.[88][160]

 

William LeMessurier-The Fifty-Nine-Story Crisis: A Lesson in Professional Behavior

 

Citicorp Center | NYC skyscraper saved by a student’s question


The Citicorp Center repair is a classic engineering case study of how mistakes must be avoided in engineering and construction of public works. A skyscraper in New York City needed a unique structural system. While reviewing the design a student (named Diane Hartly) asked a question that made the engineer realize that a mistake had been made. There is a daring race to make the repairs for the building collapses. The video gives the details and then discusses how the engineer handled the situation.

 

Princeton Engineering Student

Diane Hartley

Diane Hartley

 

Principal at Hartley LLC
Washington, District of Columbia, United States  Contact info

I now have Fiber Internet to the Home through (Windstream Internet)

This afternoon, a Windstream tech install fiber in my basement.  (thin yellow cable)

Fiber Optic to Ethernet Internet Connection

 

The white Adtran box converts fiber to Ethernet for the modem

The black cord is power.

The Green cord is telephone cable (Voice)

The thick yellow cable is Ethernet (LAN) to modem

The thin yellow connection is fiber (Fiber)

 

Modem

Windstream DSL Modem

I was able to keep my old modem, which takes Ethernet from the Adtran fiber to ethernet converter.

 

Pricing

  Previous DSL Current Fiber Fiber Upgrade Fiber Upgrade Fiber Upgrade
Download 50 mbps 200 mbps 400 mbps 500 mbps 1000 mbps
Upload 2 mbps 200 mbps 400 mbps 500 mbps 1000 mbps
Intro Price (monthly) $37 $37 $27 $47 $57
12 month price $37 $55 $55 $75 $85

Check Pricing & Availability in your Neighborhood

 

Phone

I still have the ability to call 911 on my phones (even though I dropped phone service).

The tech told me that the 911 call is routed over fiber.

 

Fiber into my Basement

Fiber Cable in home

The tech told me that fiber comes in a roll and really curls into a coil as it spreads out.

The black curling cable is fiber.

The other two cables coming from the ceiling are Ethernet and telephone.

How a lack of legitimacy undermines “Broken Windows Policing”


In a 2011 New Yorker talk, Malcolm Gladwell described the central role “legitimacy” plays in motivating people.  Previously, political theorists had focused on “deterrence theory” that treats people as rational actors who decide whether to follow the law based upon a weighing of the pros and cons of compliance.

Protesting Illegitimate Authority

Gladwell cites NYU Professor Tom Tyler’s work on “legitimacy”, and argues that people will fight to the death and even go on hunger strikes against an authority they feel is illegitimate, despite overwhelming penalties that deterrence theorists assume would be effective.

Gladwell identifies 3 factors in establishing legitimacy:

  1. Does the authority grant one standing and listen to one’s petitions?
  2. Is authority administered with neutrality or is there one set of rules for one group and a different system for others?
  3. Is the system trustworthy — does it follow well-defined rules that are sensible and are not subject to arbitrary change?

 

How Reform Efforts go Wrong:

Society celebrates civil rights leaders such as Martin Luther King and police reformers such as Jack Maple, but society often fails to recognize how easily reform can falter if succeeding leaders do not follow in the original spirit.

In this article I originally wanted  to also include an evaluation of the role “legitimacy” plays in the right’s perception of the anti-racism movement, but this post is already quite long so this post will focus solely on policing.

In the following post, I summarize a number of sources that suggest that while the policing reforms of the 1980s were innovative and effective, the system mutated into an irrational system of “broken-windows policing” that disproportionately harasses and alienates minorities.

When one asks why do minorities not comply with police, I believe that generally an important factor is American policing’s weak legitimacy.

 

How Policing Reform turned into “broken windows” policing

There is an excellent 2-part podcast series by Reply-all that chronicles how desperate the crime problem was in New York City in the 1980s and how one policeman rescued the city and transformed policing on a nation-wide basis. Unfortunately his reforms were warped into the system that is today referred to as “broken windows” policing.

 

Please listen to these two podcasts because they are a major influence on my thinking about broken-windows policing.

  1. The Crime Machine, Part I (please listen)
  2. The Crime Machine, Part II (please listen)

 

The Reply-all podcast reports that in the 1980s, when New York City was in crisis, the New York City police only cared about crime that affected white people or rich people.

Police wouldn’t even investigate a theft that was less than $10,000 ($32,000 in today’s dollars).  It was during this crisis that a transit-cop named Jack Maple created an innovative COMSTAT database that allowed police to identify the most prolific criminals and treat every crime seriously.

 

Leadership cares more about image than problem solving

A big part of the episode is about people’s efforts to fight systemic dysfunction in a world where leadership cares more about looking good than actually addressing problems.

 

There are many organizations whose leadership care more about public relations or resume-building than actually dealing with real problems.  Maybe you’ve had a CEO or supervisor who manages their organization with too much of an eye to how they look in the press or how an act contributes to their resume.  I suspect the same problem occurs with the police and elected city leadership.  In this environment it’s more important to have a lot of superficial statistics to placate the establishment and “earn” a promotion rather than dealing with difficult problems.  When too many people follow this path, it becomes difficult for the average person to do the right thing.

 

Jack Maple was fighting a dysfunctional police system when he created COMSTAT but the introduction of the database was not handled in a healthy way, and the domineering and brutal New York police culture perpetuated itself in the way the database came to be used.

 

Rather than being used as a tool to help the police become better partners with the community and identify the most prolific criminals, COMSTAT came to be used as a tool of the police leadership and mayor to generate good-sounding statistics.

 

Making Leadership Look Good at Minorities Expense

After Jack Maple (the database’s original inventor) retired, police were ordered to rack up stats in minority neighborhoods to make quota so that the mayor and police leadership could brag about how tough they were on crime.  After a while, police leadership painted themselves into a corner and they needed to generate even more “activity” statistics (tickets) in minority neighborhoods and also to under-report actual crime so that the city could still advertise itself as an increasingly safe place for tourists and those wanting to buy property.  After a series of years in which crime was understated, it became difficult for honest police to accurately report reality.

 

In minority neighborhoods, under Rudolph Giuliani’s  “broken windows” policing, police were given quotas that caused them to sweep up whole minority demographic groups, forcing the police to create laughably weak pretexts for their tickets.  In theory, the summons were supposed to be related to a nearby crime, but this was often not the case. Rather than focus on significant crime, police would be forced to cite scores of people on bogus charges such as  “blocking pedestrian traffic” because that was the only way they could meet their quotas.  Another favorite catchall for police was to cite people for “furtive movements“, despite the fact the police officers who cited this reason most were unable to define what the term means.  Police would pull down “suspects” pants and underware to search them for drugs and a culture of silence prevented accountability.

 

In the earlier era, Jack Maple had complained that the police only cared about the white and wealthy.  In the post-Jack Maple era, where the crime rate was lower nationwide, the police and politicians still only cared about the white and wealthy and looking good, but instead of ignoring minority neighborhoods or actually engaging in a partnership with the communities, in the new era they focused on building an image of toughness and  by creating an increasing amount of minority ‘activity” (tickets)  rather than fighting actual crime.  They would actually downgrade real crime while simultaneously increasing low-level harassment of minorities.

Making Broken Windows Race Neutral?

If one were to argue the new quota system was applied in a race-neutral manner, one would have to show that police adopted broken-windows policing in white neighborhoods as well.  And as the second episode describes: If the police make the mistake of issuing a citation to a white architect for riding his bike on the sidewalk, they’d find their boss would tell them to back off because such people have lawyers and connections.
A relevant question is whether this is solely a class thing — whether black architects who ride on the sidewalk get cited more often than white architects.  I suspect the answer is yes.  Upper-class black people have to dress and act much higher class than white people to get the same class benefit.  Riding a bike or walking is not an activity which separates you from the masses so you’re apt to be targeted, either by racist citizens reporting you for “riding while black” or by the police themselves.  I expect that a black man would situationally be able to achieve the same class benefit I have as a white man if they drove a luxury car that costs many times more than my Corolla and and if they dress 50% more formally.  But if they choose to go out for a run in shorts, their class benefit melts to because a runner can’t bring along their class signifiers.  It is my understanding that running in the dark while black is not the same experience I have when I go out for a run in the dark.

 

If you pay attention to the second episode, you’ll notice that they mention that this “broken windows” policing wasn’t confined to just New York City.   New York City as seen as a model city and it exported the model nationwide.  Fergusson Missouri had a huge number of summons per household and Sandra Bland (Link #3: Malcolm Gladwell) had a large number of unpaid fines for police citations.

 

Sandra Bland: Arrested for Resisting Arrest

You can dismiss this case as an extreme case of one bad apple, but Malcolm Gladwell says the arresting officer should not be written off as an anomaly.  In fact, the arresting officer is a model of how current policing philosophy and training intends police to operate.
After a difficult period of her life in Illinois, Bland moved to Texas to start a new life as a student in a small town in Texas.  The minute she left the University parking lot, a police officer manufactured a bogus charge of failing to signal as she pulled over to the side of the road in response to his aggressive driving.
This incident only got attention because she committed suicide after being jailed for 3 days.
I aknowlege Bland’s suicide is an anomily, but this officer’s behavior is not.

 

Dashcam video of Sandra Bland’s arrest

  1. The Texas police don’t seem to have have the nerve to include the video immediately before the stop where the police officer sets a trap to use a pretext for pulling her over.
  2. The police officer never responded to Bland’s request when asked for a reason why he was arresting Bland.  He was either too zoned-in in demanding she comply or he knew he couldn’t justify charging her with failure to signal.
  3. The officer later gave the rationalization that she was arrested for resisting arrest (a catch-22).

Sandra Bland’s Phone Video of her Own Arrest

 

Broken windows policing is an entire movement whose police harassment falls disproportionately on minorities.  If you listen to the Joe Rogan episode you’ll hear Malcolm Gladwell describe how broken windows policing trains police to pull over hundreds of motorists for “bullshit reasons” with the hope of “hitting the jackpot” of finding one person with heroin in their trunk.

 

 

How Representative are Video Accounts?

I hear that some of the much publicized police incidents (such as the Michael Brown incident) are misrepresented in the media.  That may be true.

 

I also hear questions about how typical police mistreatment is of blacks that does not result in death.   If you believe Amber Ruffin’s claim, every black person she knows has stories about run-ins with the police.  Ruffin shares her experience that police are quick to pull a gun on a black woman and that the police will change their tone in an instant when they realize they are being watched by someone who is white.

I would also guess that there is a sizeable fraction of the police that I would characterize as “authoritarian” who demand compliance, rather than earning legitimacy.  They do not treat black citizens with the dignity they show to whites and yet they insist that even questionable orders be obeyed.

 

So to reiterate NY Professor Tom Tyler’s framework of legitimacy as applied to broken-windows policing:
  1. Standing: Do police give citizens a fair hearing and articulate legitimate reasons that support their actions rather than declaring that they have the authority to order citizens around and force them into compliance?
  2. Neutrality: Do police conduct themselves in a way that persuades citizens that there is a standard of law that applies equally to the police themselves and to all other people?
  3. Trustworthiness: Is the law arbitrary or likely to change?

 

One can argue about whether the intention of broken windows policing is racist, but I would argue that is better to first deal with the fact that American policing (like many other American institutions) has legitimacy problems.

White Americans who have a long history of condoning rebellion should not expect minorities to treat them as meek and polite Canadians would.  Rather, White Americans should expect legitimate resistance to contemporary policing philosophy.  The challenge is to channel legitimate frustration and anger into productive dialog, reconciliation, and reform.

 

The amazing woman responsible for the Pfizer and Moderna Vaccines


Here’s the amazing story of grit and perseverance about the woman who did the pioneering work for the Pfizer and Moderna vaccine.
Katalin Kariko light corrected.jpeg
Katalin Karikó, born in Hungary
Temple University & University of Pennsylvania

Timeline: Development of Vaccine by Katalin Karikó:

(from Wired Magazine article)

 

  1. 1955: Katalin Karikó was born in Hungary
  2. 1976: She hears about ideas of using mRNA to target viruses while an undergrad at the University of Szegedin Hungary.
  3. She completes her Ph.D.
  4. 1985: As an immigrant from Hungary, Katalin Karikó immigrated to the US to do research at Temple University.
  5. After a dispute with her boss, Temple University tried to have her deported.
  6. She switched to the University of Pennsylvania, but her research was not considered promising because there were significant challenges in getting the immune system to accept the mRNA that the vaccine uses.
  7. The mid-1990s — She failed to get funding for her work at the University of Pennsylvania and was forced to choose between stopping work on her mRNA research or be demoted from a track to be a full professor.
  8. She chose to be demoted and continue her research.
  9. UPenn’s ultimatum was posed just after she had been diagnosed with cancer.
  10. She persisted and was able to get her research funded with the help of an established immunology professor — Drew Weissman — who she met at the photocopier.
  11. In the early 2000s: she read a study that gave her an idea of how to avoid the adverse immune system reaction that prevented mRNA from being used in vaccines.
  12. 2005: Karikó and Weissman published a study suggesting that there may be a way to avoid the immune reaction.
  13. After publishing their research and patenting it, they received no invitations to talk about their work.
  14. But Derrick Rossi, a postdoc at Stanford University noticed their research and created a company called Moderna in 2010 to commercialize the technology.
  15. Karikó and Weissman licensed their technology to a small German company called BioNTech, after five years of trying and failing.  (BioNTech was founded by a Turkish immigrant named Ugur Sahin)
  16. 2013: UPenn refused to reinstate Katalin Karikó as a full professor after demoting her in 1995. She told them she was leaving to go to BioNTech: ”When I told them I was leaving, they laughed at me and said, ‘BioNTech doesn’t even have a website.’”
  17. 2017: Moderna (founded by the Derrick Rossi Stanford postdoc) used the technology Karikó pioneered to develop a Zika virus.
  18. 2018: The German company Karikó and Weissman licensed their technology to partnered with Pfizer to develop an influenza vaccine.
  19. April 2020: Derrick Rossi’s Moderna received $483 million (£360m) from the US Biomedical Advanced Research and Development Authority to fast-track its Covid-19 vaccine program
  20. Pfizer developed their mRNA vaccine using Karikó and Weissman research, but without government funding.

Read More:

1) Read full “Wired” article: