Single server setup
A journey of a thousand miles begins with a single step, and building a complex system is no different. To start with something simple, everything is running on a single server, including web app, database, cache, etc:
- Web application: it uses a combination of server-side languages (Java, Python, etc.) to handle business logic, storage, etc., and client-side languages (HTML and JavaScript) for presentation.
- Mobile application: HTTP protocol is the communication protocol between the mobile app and the web server. JavaScript Object Notation (JSON) is commonly used API response format to transfer data due to its simplicity.
Database
With the growth of the user base, one server is not enough, and we need multiple servers: one for web/mobile traffic, the other for the database. Separating web/mobile traffic (web tier) and database (data tier) servers allows them to be scaled independently:
Which database to choose?
Relational vs non-relational database
Load balancer
A load balancer distributes incoming traffic among web servers that are defined in a load-balanced set.
Users connect to the public IP of the load balancer directly. With this setup, clients cannot reach web servers directly. For better security, private IPs are used for communication between servers.
After a load balancer and a second web server are added, we successfully address failover issue and improved the availability of the web tier.
- If server 1 goes offline, all the traffic will be routed to server 2. This prevents the website from going offline. We will also add a new healthy web server to the server pool to balance the load.
- If the website traffic grows rapidly, and two servers are not enough to handle the traffic, the load balancer can handle this problem gracefully. You only need to add more servers to the web server pool, and the load balancer automatically starts to send requests to them.
Database replication
Avantages of database replication:
- Better performance: In the master-slave model, all writes and updates happen in master nodes; whereas, read operations are distributed across slave nodes. This model improves performance because it allows more queries to be processed in parallel.
- Reliability: If one of your database servers is destroyed by a natural disaster, data is still preserved. You do not need to worry about data loss because data is replicated across multiple locations.
- High availability: By replicating data across different locations, your website remains in operation even if a database is offline as you can access data stored in another database server.
Now, we intend to improve request response time. This can be done by adding a cache layer and shifting static content (JavaScript/CSS/image/video files) to the content delivery network (CDN).
Cache
A cache is a temporary storage area that stores the result of expensive responses or frequently accessed data in memory so that subsequent requests are served more quickly.
Caching Strategies and How to Choose the Right One
Consideration for using cache
- Expiration policy
- Consistency
- Mitigating failures
- Eviction policy
Content delivery network
A CDN is a network of geographically dispersed servers used to cache and deliver static content like images, videos, CSS, JavaScript files, etc.
Consideration for using CDN
- Cost
- Setting an appropriate cache expiry
- CDN fallback
- Invalidating files
Integrating Cache and CDN
Stateless web tier
Now we intend to scale the web tier horizontally. For this, we need to move state (e.g., user session data) out of the web tier. A good practice is to store session data in the persistent storage which allows each web server in the cluster to concurrently access state data. This way, servers can be freely added or removed without having to worry how to deal with any specific client requests. Any request to be routed to any server. This mechanism is called stateless web tier.
The shared data store could be a relational database, Memcached/Redis, NoSQL, etc. The NoSQL data store is chosen as it is easy to scale.
Data centers
Your website grows rapidly and attracts a significant number of users internationally. Supporting multiple data centers is crucial to improve availability and provide a better user experience across wider geographical areas:
In the event of any significant data center outage, we direct all traffic to a healthy data center.
Several technical challenges must be resolved to achieve multi-data center setup:
- Traffic redirection
- Data synchronization: In failover cases, traffic might be routed to a data center where data is unavailable. A common strategy is to replicate data across multiple data centers.
- Active-Active for Multi-Regional Resiliency
- Test and deployment
- With multi-data center setup, it is important to provide automated deployment tools to keep services consistent through all the data centers
Message queue
To further scale our system, we need to decouple different components of the system so they can be scaled independently. Messaging queue is a key strategy employed by many real-world distributed systems to solve this problem.
Logging, metrics, and automation
Database scaliing
Horizontal scaliing
Sharding separates large databases into smaller and more easily managed partitions called shards. Each shard shares the same schema, though the actual data on each shard is unique to the shard.
User data is allocated to a database server based on user IDs. Anytime you access data, a hash function is used to find the corresponding shard:
A sharding key allows you to retrieve and modify data efficiently by routing database queries to the correct database. When choosing a sharding key, one of the most important criteria is to choose a key that can evenly distributed data.
Millions of users and beyond
