Key metrics to watch
What to look at in each node's metrics to know the cluster is healthy.
Watch a few metrics of each node to know the cluster is healthy before your users notice.
Requirements
- Each node's
/metricsin your metrics system, as in Metrics.
What to watch
| Signal | What it indicates |
|---|---|
teseladb_up at 0 or missing | The node is not running or does not answer. |
/healthz with 503 | The node is running but not serving. |
teseladb_range_leader very uneven across nodes | One node leads many more ranges than the others. |
teseladb_range_applied_index stuck on a replica | That replica does not keep up with its range. |
| Errors by code in the REST API | Requests that fail, per route. |
Check
With a node stopped on purpose in a test cluster, teseladb_up of that node stops arriving and your alert fires.
Common problems
- An alert fires on every planned restart. Silence it during maintenance.