Get to know to your problem first:

• Read scalability problem symptoms:
  • Pages load slowly or just timeout.
  • Some pages load, some don't.
  • 503 errors
  • Database metrics show high CPU or read IO.
• Write scalability problem symptoms:
  • Replicate databases have hard time keeping up with the master.
  • SQL metrics show that update operations take longer to execute.
  • Database metrics show high write IO but lesser CPU

Heroku is a great place to start. You might require full control of application layer in your app later.

• Ruby on Rails
• app server = unicorn
• web server = nginx
• deployment tool = capistrano
• server configuration = chef

Use memcached or redis

• Use memory storages for web sessions.
• Use memory storages to cache HTML and serialized objects.
• Transient non-business critical data.

Use background jobs e.g. Sidekiq. Background processing for email sending and other long running tasks.

Use CDN like CloudFront or fastly.com

Multiple loadbalancers require DNS round robin and shorter TTL.

Acceptable request response time 80ms. This means how much time Ruby, database, caches and web services should take. For APIs and backend services, target 12ms or lower. Ruby is usually CPU bound, others are IO bound.

Add more caching.

• Anything that can be cached, should be as caches are cheap.
• Use caches_page in controller actions that don't require authentication and are pretty similar to all users e.g. marketing pages. Small amount of customization can be added with AJAX.
• Use caches_action in controller actions and and send all personalization through AJAX. Action caching means that entire output is cached, but requests still go through filters like before action authentication check.
• Use write_fragment / read_fragment to cache reusable widgets inside your views.
• Compositor gem allows caching JSON API responses. Cache partial objects, multi_get and merge.
• IdentityCache gem allows caching ActiveRecord models.
• If you use separate subdomain for your JSON like api.example.com so you can use CDN to cache JSON responses outside of your server. CloudFront and fastly are both good.

Database

All SQL statements that take longer than 100ms are worth to optimize. Remove it, cache it or fix the query. Can you add new indexes?

PostgreSQL has setting to log slow queries.

SELECT query, calls, total_time, rows
FRaOM pg_stat_statements
ORDER BY total_time DESC
LIMIT 10;

Set database shared_buffers to 25% of RAM, but up to 12GB.

Makara gem allows sending reads to replica databases. Note that replicates are not synced right away, sometimes you will need to do a force fetch from the master after trying reading replica.

• Keep track of how much your replicates are behind.
• Background workers can use specialized replica to improve file system cache hits.

Move write-heavy tables outside of the main database. Use another database, a no-SQL database, S3, CloudWatch or

Schemas:

• Split wide tables into several 1-1 tables to reduce update impact.
• Operates faster and vacuuming is faster.

Users =>
    Users       id, email, created_at
    UserCounts  user_id, follower_count
    UserLogins  user_id, encrypted_password, last_sign_in

Sources

• 12-Step Program for Scaling Web Applications on PostgreSQL