Fault tolerance
What happens when a node, a disk or the network between machines fails.
TeselaDB keeps serving when a minority of the replicas of each range goes down. What changes when a node goes down is who leads each range and where its replicas live.
How it works
n2 goes down
│
├─▶ ranges n2 led: the replicas on n1 and n3 elect a new leader
├─▶ drivers: retry the operation in flight on another node
└─▶ balancer: recreates the missing replicas on n1 and n3, if n2 does not come backWhen a node goes down, the ranges it led elect a new leader among the remaining replicas. The ranges that only had a follower replica on it go on as before. If the node does not come back, the balancer repairs the missing replicas on other nodes. If the network cuts a node off from the others, that node stops accepting writes in the ranges where it does not have the majority.
What it means for your application
Your application sees, at most, transient errors that the driver retries. With w: "majority", a confirmed write is not lost when a minority goes down.
Limits
If a range loses a majority of its replicas at once, it stops accepting writes until they come back. Repairs go on even if you stop the balancer.