Fluentd is an advanced open-source log collector developed at Treasure Data, Inc (see previous post). Because Fluentd handles logs as semi-structured data streams, the ideal database should have strong support for semi-structured data. There are several databases that meet this criterion, but we believe MongoDB is the market leader.
What is Facebook’s architecture?
300 TB of data is stored in Memcached processes
High concurrency counters without sharding
Our solution has three major components:
- A ‘permanent’ count, stored in the datastore.
- A ‘current’ count, stored in memcache.
- A task queue task that updates the datastore with the total from memcache.
CDN Repsonse Time
Below you can find a few measurements taken yesterday over a 24 hour period. These measurements represent a sub-set data collected from the top 10% of contributing networks in the selected country. Much more data is available within our free benchmarking community. Join for free.