The Most Productive People Think This More Often: Charles Duhigg

Productivity is not something that has an objective standard; productivity is what people define as being helpful to them.

.. We are living through this economic revolution that most economists agree will be as profound as the Agrarian Revolution or the Industrial Revolution in terms of how we live, and work, and how our social lives function. And what happens is during each economic revolution is that the definition of productivity is actually what’s being revolutionized.

.. Before the Industrial Revolution the most important unit of productivity was the hour. How did you spend your hour? And the best way to be productive before the Industrial Revolution was to simply own land. If you owned land and had access to cheap labor, you had a whole bunch of other people’s hours to apply to that land. So you were super productive and you were super successful and rich, even if you were sitting around drinking gin and tonics all day.

.. The Industrial Revolution changed all that. All the sudden, simply spending hours on a task does not matter as much as how much smarts you bring to that task. Because if you can invent a machine that can work as fast as five humans, hence this machine is doing the task more efficiently, then it doesn’t matter if you have more hours.

.. And right now, we’re living through this economic revolution, whether you call it the “knowledge revolution” or the “technology revolution” something is happening right now that is changing the definition of productivity.

.. What I do think is unique to this age is the number of ways that we can be distracted, or the different channels for “busyness” have now exploded. That means it’s easier to feel overwhelmed and feel busy.

.. Email was the original asynchronous coordinating device. The benefit of email is that you can send a message to someone and they can reply to it when they have a minute, as opposed to being available when you call.

.. What Google found, and this is backed by science, is that there is this group norm known as psychological safety. Psychological safety is this idea that you can bring your full self to work, and you can be you, and the group will hear you being you and not bristle against it.

.. ostentatious listening. Simply listening to someone often isn’t enough. You have to show them you’re listening by doing things like picking up on nonverbal cues, repeating what they said to you, complimenting their idea, or taking their idea and building on it.

.. So all this said, what is the *true secret* to productivity?

A huge part of it is exposing yourself to the knowledge of alternative ways of thinking that are possible.

.. They’re more productive because they develop mental habits, what psychologists sometimes call “cognitive mental routines” that push them to think more deeply about the choices they’re making. The best way to learn how to think better is to expose yourself to more ideas, and more specifically, lessons that explain to you how to think differently and expose you to the habits around how to think differently on a regular basis.

.. The literal secret to productivity is someone saying, “Oh, I never thought about it that way.”tiny_twitter_bird.pngTweet this

 

multiprocessing, gevent, requests and friends

hi guys, I’ve been working on a project for large scaling high profile scraping, i got around 2-3k(should be in the future around the 100k) urls under the same host.

i took the amount of urls, split it by number of process, each part of urls went to new process with gevent pool. the results are good but i want better.

I’m using multiprocessing, requests.Session(), and gevent pool.

code structure: http://pastebin.com/Xu7Xy41i

the parser is lxml, which i found the fastest. requests.Session() support requests for same host multiprocessing + gevent.pool for multiprocessing async work

  • i believe the ssl handshake slow things up, maybe there is a good solution for fast handshake, or avoid multiple handshakes.
  • I’m up for any solution to get better performance.
  • i thought about maybe keep amount of sockets open and get a queue of urls for each socket.

Cheers

Eventlet vs. Asyncio

Explicit and implicit concurrency in Python.

.. With asyncio, an event loop runs and is in charge of switching between various coroutines. The main difference between asyncio and eventlet is that switching is performed explicitly. I have to use yield from in my coroutine if I wish to indicate that I’m ready to be switched out.