openpolitics.com

But as anyone who’s worked with tech people knows, this intellectual background can also lead to arrogance. People who excel at software design become convinced that they have a unique ability to understand any kind of system at all, from first principles, without prior training, thanks to their superior powers of analysis. Success in the artificially constructed world of software design promotes a dangerous confidence.

.. Approaching the world as a software problem is a category error that has led us into some terrible habits of mind.

.. First, programmers are trained to seek maximal and global solutions. Why solve a specific problem in one place when you can fix the general problem for everybody, and for all time? We don’t think of this as hubris, but as a laudable economy of effort.

.. Second, treating the world as a software project gives us a rationale for being selfish.

.. Third, treating the world as software promotes fantasies of control. And the best kind of control is control without responsibility.

.. Instead of relying on algorithms, which we can be accused of manipulating for our benefit, we have turned to machine learning, an ingenious way of disclaiming responsibility for anything. Machine learning is like money laundering for bias. It’s a clean, mathematical apparatus that gives the status quo the aura of logical inevitability.

.. Of course, people obsessed with control have to eventually confront the fact of their own extinction. The response of the tech world to death has been enthusiastic. We are going to fix it. Google Ventures, for example, is seriously funding research into immortality. Their head VC will call you a “deathist” for pointing out that this is delusional.

.. Elon Musk’s apparently sincere belief that we’re living in a simulation. For a computer programmer, that’s the ultimate loss of control. Instead of writing the software, you are the software.

.. Just like industrialized manufacturing changed the relationship between labor and capital, surveillance capitalism is changing the relationship between private citizens and the entities doing the tracking. Our old ideas about individual privacy and consent no longer hold in a world where personal data is harvested on an industrial scale.

Those who benefit from the death of privacy attempt to frame our subjugation in terms of freedom, just like early factory owners talked about the sanctity of contract law. They insisted that a worker should have the right to agree to anything, from sixteen-hour days to unsafe working conditions, as if factory owners and workers were on an equal footing.

Companies that perform surveillance are attempting the same mental trick. They assert that we freely share our data in return for valuable services. But opting out of surveillance capitalism is like opting out of electricity, or cooked foods—you are free to do it in theory. In practice, it will upend your life.

.. The customs service announced yesterday it wants to start asking people for their social media profiles.

.. We’re used to talking about the private and public sector in the real economy, but in the surveillance economy this boundary doesn’t exist. Much of the day-to-day work of surveillance is done by telecommunications firms, which have a close relationship with government. The techniques and software of surveillance are freely shared between practitioners on both sides. All of the major players in the surveillance economy cooperate with their own country’s intelligence agencies, and are spied on (very effectively) by all the others.

.. Or consider the other candidate running for President, the one we consider the sane alternative, who has been a longtime promoter of a system of extrajudicial murder that uses blanket surveillance of cell phone traffic, email, and social media to create lists of people to be tracked and killed with autonomous aircraft.

.. That this toolchain for eliminating enemies of the state is only allowed to operate in poor, remote places is a comfort to those of us who live elsewhere, but you can imagine scenarios where a mass panic would broaden its scope.

.. We tend to imagine dystopian scenarios as one where a repressive government uses technology against its people. But what scares me in these scenarios is that each one would have broad social support, possibly majority support.

.. Those who run the surveillance apparatus understand its capabilities in a way the average citizen does not. My greatest fear is seeing the full might of the surveillance apparatus unleashed against a despised minority, in a democratic country.

.. I am very suspicious of attempts to change the world that can’t first work on a local scale. If after decades we can’t improve quality of life in places where the tech élite actually lives, why would we possibly make life better anywhere else?

.. We should not listen to people who promise to make Mars safe for human habitation, until we have seen them make Oakland safe for human habitation. We should be skeptical of promises to revolutionize transportation from people who can’t fix BART, or have never taken BART.

.. Techies will complain that trivial problems of life in the Bay Area are hard because they involve politics. But they should involve politics. Politics is the thing we do to keep ourselves from murdering each other. In a world where everyone uses computers and software, we need to exercise democratic control over that software.

.. The goal should be not to make the apparatus of surveillance politically accountable (though that is a great goal), but to dismantle it.

.. There is also prior art in attempts at achieving immortality, limitless wealth, and Galactic domination. We even know what happens if you try to keep dossiers on an entire country.