When browsers make a request, they include HTTP headers for the server to decide what to send back (Is this a mobile client? Can it handle compressed content? Does it need a certain language?).
That’s great for direct access, but modern networks use intermediate caches and CDNs. And there’s the problem: how does the cache use headers to decide what to send back? How can it replicate the server’s decision-making logic?
Vary to the rescue. The Vary header describes what information “uniquely” identifies a request — caches should only be used if the incoming request matches the Vary information in the cache.
Nginx
<span class="hljs-attribute">gzip_vary</span> <span class="hljs-literal">on</span>
What Powers Instagram: Hundreds of Instances, Dozens of Technologies
We run Django on Amazon High-CPU Extra-Large machines
.. We’ve found that our particular work-load is very CPU-bound rather than memory-bound, so the High-CPU Extra-Large instance type provides the right balance of memory and CPU.
.. We use http://gunicorn.org/ as our WSGI server; we used to use mod_wsgi and Apache, but found Gunicorn was much easier to configure, and less CPU-intensive. To run commands on many instances at once (like deploying code), we use Fabric, which recently added a useful parallel mode so that deploys take a matter of seconds.
.. Most of our data (users, photo metadata, tags, etc) lives in PostgreSQL; we’ve previously written about how we shard across our different Postgres instances. Our main shard cluster involves 12 Quadruple Extra-Large memory instances (and twelve replicas in a different zone.)
.. The photos themselves go straight to Amazon S3, which currently stores several terabytes of photo data for us. We use Amazon CloudFront as our CDN, which helps with image load times from users around the world (like in Japan, our second most-popular country).