Load spikes and excessive memory usage in mod_python

Specifically, the number defined for the maximum clients should be dropped comensurate with the amount of memory available to run it and how big the application gets.
.. The solution here is not to create only a minimal number of servers when Apache starts, but create closer to what would be the maximum number of processes you would expect to require to handle the load. That way the processes always exist and are ready to handle requests and you will not end up in a situation where Apache needs to suddenly create a huge number of processes.
.. First off, don’t run PHP on the same web server. That way you can run worker MPM instead of prefork MPM. This immediately means you drop down drastically the number of processes you require because each process will then be multithreaded rather than single threaded and can handle many concurrent requests.
.. The important thing to note here is that although the maximum number of clients is still 150, each process has 25 threads. Thus, the maximum number of processes that could be created is 6. For that 30MB process that means you only need 180MB in the worst case scenario rather than the 4GB required with the default MPM settings for prefork.
.. Keep that in mind and one has to question how wise the advice in the Django documentation is that states “you should use Apache’s prefork MPM, as opposed to the worker MPM” when using mod_python.
.. With Django 1.0 now believed to be multithread safe, which was in part why prefork was recommended previously, that advice should perhaps be revisited, or it made obvious that one would need to consider tuning your Apache MPM settings if you intend using prefork MPM.

PostgreSQL Scalability: Towards Millions TPS

PostgreSQL scalability on multicore and multisocket machines became a subject of optimization long time ago once such machines became widely used. This blog post shows brief history of vertical scalability improvements between versions 8.0 and 8.4. PostgreSQL 9.2 had very noticeable scalability improvement. Thanks to fast path locking and other optimizations it becomes possible to achieve more than 350 000 TPS in select-only pgbench test. The latest stable release PostgreSQL 9.5 also contain significant scalability advancements including LWLock improvement which allows achieving about 400 000 TPS in select-only pgbench test.

 

.. It’s HP ProLiant DL580 Gen9 with 3 TB of RAM.

.. It was 100% CPU on each core. Waiting on bus is still reported as CPU busy. Thus, we can’t say it’s not memory bandwidth limited.
I expect slightly higher numbers with clients on network, because pgbench consumes some resources itself. But those numbers could be achieved with higher number of clients due to network latency.

WP Offload S3

WP Offload™ S3 copies files from your WordPress Media Library to Amazon S3 and rewrites URLs to serve the files from S3, CloudFront, or another CDN. And with the Assets addon, it can identify assets (CSS, JS, images, etc) used by your site, copy them to S3, and serve them from S3, CloudFront, or another CDN.