AWS Public Datasets

AWS hosts a variety of public datasets that anyone can access for free.

Previously, large datasets such as satellite imagery or genomic data have required hours or days to locate, download, customize, and analyze. When data is made publicly available on AWS, anyone can analyze any volume of data without needing to download or store it themselves. These datasets can be analyzed using AWS compute and data analytics products, including Amazon EC2, Amazon Athena, AWS Lambda and Amazon EMR.

Stop Misquoting Donald Knuth!

Donald Knuth once wrote these unfortunate words:

” We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.”

This statement is often quoted in an attempt to de-emphasize code performance in favor of other factors which are considered more important:

  • Ease of use
  • Rapid development
  • Clarity and readability
  • Reusability
  • Risk of accidental bugs
  • Extensibility
  • Elegance
  • Object-Oriented Purity

.. The irony about Knuth’s quote is that the people who most often quote it would be horrified by its original context. Knuth made this statement in an article in which he argued in favor of the careful use of go-to statements for micro-optimization in performance-intensive loops

 

.. The improvement in speed from Example 2 to Example 2a is only about 12%, and many people would pronounce that insignificant. The conventional wisdom shared by many of today’s software engineers calls for ignoring efficiency in the small; but I believe this is simply an overreaction to the abuses they see being practiced by penny-wise- and-pound-foolish programmers, who can’t debug or maintain their “optimized” programs. In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering. Of course I wouldn’t bother making such optimizations on a one-shot job, but when it’s a question of preparing quality programs, I don’t want to restrict myself to tools that deny me such efficiencies.

 

..  As software engineers, we have largely forgotten about our obligation to create quality programs where efficiency is concerned. We have built for ourselves a culture in which constant factors are the compiler’s problem and the hardware’s problem, in which our productivity, comfort, and profit are more important than the effective utilization of the user’s resources, and the quality of their experience under load.

It’s up to us to kill false information. Good luck.

Individuals bear much of the blame for fake news. The study found that false rumors travel the Internet much more rapidly and widely than facts. These untruths get their velocity and reach not from celebrity influencers but from ordinary citizens sharing among their networks.

Evidently, we humans have a strong preference for novelty and sensationalism over scrupulous reality.

.. “Falsehood diffused significantly farther, faster, deeper, and more broadly than the truth in all categories of information,” the MIT scientists concluded after examining more than 125,000 stories shared by more than 3 million Twitter users. The most viral lies, they found, involved “false political news.”

.. Politics is tribal. It is a way of organizing conflict.

.. We are inclined to credit anything we hear from our allies and to believe the worst of our foes. In politics we see information as potential ammunition; we evaluate it for its potency and lethality rather than its strict veracity.

.. the Internet smokes out our self-deceptions and shows us as we really are.

Gambling and porn flourish on the Internet. Reasoned civil discourse, not so much.

.. This is a profound blow to idealists of the marketplace of ideas. From Adam Smith to Friedrich Hayek to James Surowiecki, the author of “The Wisdom of Crowds,” wise thinkers have emphasized the positive economic effects of dispersed power. A great many people, free to pursue the wisdom of their experiences and the perspectives from their vantage points, will arrive — as if moved by an invisible hand — at better results than any single mind or central planning bureaucracy could achieve.

.. But it turns out that the crowd is wise only when it is asking the right questions. A crowd determined to get the best value on flat-screen televisions will soon discover the proper price; but a crowd swept up by tulips or cryptocurrency may find itself pricing euphoria instead of value.

 What we see from Twitter and other platforms clearly signals that too many people are asking the wrong questions
.. our ability to spread our careless and malign thinking is brand-new. Of all the digital-age jargon, perhaps none is more apt than “going viral,” because the contagion of bad information is a matter of individuals passing germs from host to host with geometric speed. Only disciplined digital hygiene can halt the epidemic.