Definitions:
Consistency: consistency means
all clients see the same data at the same time no matter which node they
connect to.
Availability: availability
means any client which requests data gets a response even if some of the nodes
are down.
Partition Tolerance: a partition
indicates a communication break between two nodes. Partition tolerance means
the system continues to operate despite network partitions.
CAP theorem:
CAP
theorem states it is impossible for a distributed system to simultaneously
provide more than two of these three guarantees: consistency, availability, and
partition tolerance.
In
distributed systems, data is usually replicated multiple times. Assume data are
replicated on three replica nodes, n1, n2 and n3 as shown below.In the ideal
world, network partition never occurs. Data written to n1 is automatically
replicated to n2 and n3. Both consistency and availability are achieved.
In real world distributed systems, partitions cannot be avoided, and when a partition occurs, we must choose between consistency and availability.
In
above Figure, n3 goes down and cannot communicate with n1 and n2. If clients
write data to n1 or n2, data cannot be propagated to n3. If data is written to
n3 but not propagated to n1 and n2 yet, n1 and n2 would have stale data.
consistency over availability
If
we choose consistency over availability (CP system), we must block all write
operations to n1 and n2 to avoid data inconsistency among these three servers,
which makes the system unavailable. Bank systems usually have extremely high
consistent requirements. For example, it is crucial for a bank system to display
the most up-to-date balance info. If inconsistency occurs due to a network
partition, the bank system returns an error before the inconsistency is
resolved.
Availability over consistency
The
system keeps accepting reads, even though it might return stale data. For
writes, n1 and n2 will keep accepting writes, and data will be synced to n3
when the network partition is resolved. Choosing the right CAP guarantees that
fit your use case is an important step in building a distributed key-value
store.
