Nodes

Add nodes to the cluster and retire the ones you no longer need without stopping the service.

Every node runs the same binary. The cluster spreads replicas and leaders across them, so adding or removing one needs no manual data movement.

Requirements

  • Access to teseladb admin on a node of the cluster.
  • To add one, a machine with the binary installed and port 27100 open towards the other nodes.

List the nodes

Shell
teseladb admin nodes

Add a node

Shell
teseladb admin token create --ttl 1h
teseladb start --dbpath /var/lib/teseladb --join node-01:27100 --enroll-token <token>

The first command runs on any node of the cluster and creates a single-use enrolment token; the second starts the binary on the new machine with that token. The new node gets its certificate signed by the cluster CA and starts receiving range replicas.

Drain a node

Shell
teseladb admin node drain <node_id>

You cannot drain the last active node. Drain the node before retiring its machine. The node goes to Draining and the balancer takes away its replicas and its leaderships and, last, its vote in Group 0; once it has no replica left, it closes its client port and the drivers go to another node.

Remove a node

Shell
teseladb admin node remove <node_id>

Remove the node once it has no replicas and no vote in Group 0. If it still has replicas, the command fails, says how many are left and asks you to finish the drain first.

Rotate a node's certificate

Shell
teseladb admin cert rotate

Without restarting, the node generates a new key and certificate signed by the cluster CA and starts using them.

Check

teseladb admin nodes shows the new node as active and, after a while, with replicas. A drained node shows up as Draining until it has no replicas left. As soon as you remove it, it leaves the list.

Common problems

  • node drain fails on the last active node, because a cluster needs at least one node serving.
  • node remove fails and says how many replicas the node has left. Wait for the drain to finish and try again.

See also