Change streams

Follow the changes of a collection, a database or the whole cluster, in commit order.

MongoDB change streams work on a collection, a database or the whole cluster. Events come out in commit order, with transactions together, with no losses or duplicates even when ranges split or merge. The same holds when a range changes leader.

Requirements

A user with the find and changeStream privileges on what you watch, be it a collection, a database or the whole cluster.

Open the stream

JavaScript

const stream = db.orders.watch([{ $match: { operationType: { $in: ["insert", "update"] } } }]);
for await (const change of stream) {
  console.log(change.operationType, change.documentKey);
}

Resume

Every event carries a resume token. If your application goes down, open the stream again with resumeAfter or startAfter and carry on where it stopped. You can resume while the change is still in the history the cluster keeps: 24 hours by default, which you change with the teselaHistoryRetention parameter.

JavaScript

db.adminCommand({ setClusterParameter: { teselaHistoryRetention: { seconds: 172800 } } });

Pre-images and post-images

Turn on changeStreamPreAndPostImages on the collection and ask for fullDocumentBeforeChange or fullDocument: "updateLookup" as in MongoDB. changeStreamOptions.preAndPostImages.expireAfterSeconds limits how long they are kept.

DDL and large events

Events that create or drop collections and indexes, and those that change them, come out with showExpandedEvents, and an event larger than 16 MiB is split with $changeStreamSplitLargeEvent.

Check

With the stream open, insert a document into the collection from another session. The loop then prints an insert event.

Common problems

You cannot resume because the token is older than the history the cluster keeps, 24 hours by default. Raise teselaHistoryRetention if your processes can be stopped for longer.