stats

People Don’t Actually Want Equality: They want fairness

Frankfurt suggests that if people take the time to reflect, they’ll realize that inequality isn’t really what’s bothering them.

People might be troubled by what they see as unjust causes of economic inequality, a perfectly reasonable concern given how much your income and wealth are determined by accidents of birth, including how much money your parents had, your sex, and the color of your skin. We are troubled as well by potential consequences of economic inequality. We may think it corrodes democracy, or increases crime, or diminishes overall happiness. Most of all, people worry about poverty—not that some have less, but rather “that those with less have too little.”

.. A world in which everyone suffered from horrible poverty would be a perfectly equal one, he says, but few would prefer that to the world in which we now live. Therefore, “equality” can’t be what we really value.

.. My favorite example here is from the comedian Louis C.K., where he describes how his five-year-old’s toy broke and she demanded that he break her sister’s toy, which would make things equal. “And I did. I was like crying. And I look at her. She’s got this creepy smile on her face.”

.. The psychologists Alex Shaw and Kristina Olson told children between the ages of six and eight about two boys, Dan and Mark, who had cleaned up their room and were to be rewarded with erasers—but there were five of them, so an even split was impossible. Children overwhelmingly reported that the experimenter should throw away the fifth eraser rather than establish an unequal division.

.. In other words, they were fine with inequality, so long as it was fair.

.. He argues that these egalitarian structures emerge because nobody wants to get screwed. Individuals in these societies end up roughly equal because everyone is struggling to ensure that nobody gets too much power over him or her.

.. Boehm writes, “Individuals who otherwise would be subordinated are clever enough to form a large and united political coalition. … Because the united subordinates are constantly putting down the more assertive alpha types in their midst, egalitarianism is in effect a bizarre type of political hierarchy: The weak combine forces to actively dominate the strong.”

This analysis helps us explain why such huge power differentials exist in the world right now, where it’s far harder for the weak to team up to dominate the strong.

.. They found that Americans are very wrong about just how unequal their country is—they think that the bottom 40 percent has 9 percent of the wealth and the top 20 percent has 59 percent, while the actual proportions are 0.3 percent and 84 percent.

.. the vast majority of Americans prefer a distribution of wealth more equal than what exists in Sweden

Analyzing S3 and CloudFront Access Logs with AWS RedShift

Log data is an interesting case for RedShift. In our environment as mentioned previously we have so much log data from our CloudFront and S3 usage that nobody could conceivably work with those datasets using standard text tools such as grep or tail. Many people load their access logs into databases, but we have not found this to be feasible using MySQL or PostgreSQL due to the fact that ad-hoc queries run against sets with billions of rows can take hours. Once imported into RedShift the same queries take minutes at the most.

.. For our simple example though, we’ll just load one month of logs from just one of our CloudFront distributions:
COPY cf_logentries
 FROM 's3://cloudfront-logs/E1DHT7QI9H0ZOB.2014-04-'
 CREDENTIALS 'aws_access_key_id=;aws_secret_access_key='
 DELIMITER '\t' MAXERROR 200 FILLRECORD IGNOREHEADER 2 gzip;

.. With CloudFront you really should care about your cache hit ratio - maybe it's obvious, but the load on your origin systems decrease as your content becomes easier to cache. This query will look at the most used URLs and give you a cache hit ratio:

How to Make a Box-and-Whisker Plot in SQL

Flowing Data’s Nathan Yau does a great job of introducing box plots. As he notes, box plots typically graph six data points:

The lowest value, excluding outliers

The first quartile (this is the 25th percentile, or median of all the numbers below the median)

The median value (equivalent to the 50th percentile)

The third quartile (this is the 75th percentile, or median of all the numbers above the median)

The highest value, excluding outliers

Outliers

Understanding the Historic Divergence Between Productivity and a Typical Worker’s Pay

Since 1973, hourly compensation of the vast majority of American workers has not risen in line with economy-wide productivity. In fact, hourly compensation has almost stopped rising at all. Net productivity grew 72.2 percent between 1973 and 2014. Yet inflation-adjusted hourly compensation of the median worker rose just 8.7 percent, or 0.20 percent annually, over this same period, with essentially all of the growth occurring between 1995 and 2002. Another measure of the pay of the typical worker, real hourly compensation of production, nonsupervisory workers, who make up 80 percent of the workforce, also shows pay stagnation for most of the period since 1973, rising 9.2 percent between 1973 and 2014. Again, the lion’s share of this growth occurred between 1995 and 2002.