Counters without hotspots

TeselaDB keeps documents and index entries sorted by their key and spreads them across ranges. A counter that every request increments is a single document, in a single range and with a single leader. An index on a creation date sends all its new entries to the end of its interval.

What you will build

A page view counter spread across eight documents, read with an aggregation, and an index on the date of the events with its entries spread across hash buckets.

Requirements

• mongosh and a user with readWrite on the shop database.
• A TeselaDB cluster with several nodes, so the ranges spread across them.

1. Spread the counter

Each increment picks one of the page's eight documents at random. That way the writes go to different keys and do not collide on a single one:

const SHARDS = 8;
for (let i = 0; i < 100; i++) {
  const shard = Math.floor(Math.random() * SHARDS);
  db.pageviews.updateOne({ _id: { page: "home", shard } }, { $inc: { count: 1 } }, { upsert: true });
}

2. Spread the index on the date

With teselaBuckets, the index entries are spread across 16 hash buckets. The planner merges the buckets when it reads, so queries by date do not change:

db.events.createIndex({ createdAt: 1 }, { teselaBuckets: 16 });

Try it

The page total is the sum of its eight documents. An aggregation that matches the page and adds up the count field of each of them returns the 100 increments written by the loop of step 1:

db.pageviews.aggregate([
  { $match: { "_id.page": "home" } },
  { $group: { _id: "$_id.page", views: { $sum: "$count" } } },
]);

[ { _id: 'home', views: 100 } ]

Under real load, teseladb admin ranges --collection shop.pageviews shows the counter documents in several ranges once the collection grows and splits.

Clean up

db.pageviews.drop();
db.events.drop();

Next steps

• Keys and hotspots, with why a key that always grows concentrates the load.
• Find hotspots, to see which ranges take the most writes.