Reader

8 min read

8 min read

8 min read

8 min read

8 min read

8 min read

8 min read

8 min read

8 min read

A reader represents the subscriber side of a DataStorm publish–subscribe application. You create readers to receive data published by writers.

Readers receive data samples from a specific topic. Each reader is associated with a single topic, but multiple readers can be created for the same topic. The reader’s Key, Value, and UpdateTag template parameters must match the corresponding types of the topic from which it is created.

When a reader is created, DataStorm notifies connected peers. Writers whose configuration matches the reader (topic, keys/filters) will attach so they can send samples to that reader.

You can optionally assign a name to a reader at creation time. Writers see this name in their connected-readers notifications (getConnectedReaders and onConnectedReaders).

Readers keep a history (queue) of unread samples. When samples are retrieved (e.g., with getNextUnread or getAllUnread), they are removed from the queue.

  • Single-key reader Reads samples for a single key.
  • Multi-key reader Reads samples for a set of predefined keys.
  • Any-key reader Reads samples for any key (no keys provided at creation).
  • Filtered-key reader Reads samples for keys matching a key filter (name + criteria value).

You create readers using the corresponding reader class constructors or via makeXxxReader helper functions. Using the makeXxxReader helpers lets the compiler deduce template parameters (Key, Value, UpdateTag) from the topic.

  • Single-key: writers with the same key attach.
  • Multi-key: writers attach if at least one of the reader’s keys matches.
  • Any-key: all writers for the topic attach (since any key can be received).
  • Filtered-key: writers attach if their key matches the reader’s key filter (name + criteria).
C++
// Constructor
Topic<string, float> temperatures{node, "temperatures"};
SingleKeyReader<string, float> reader{
temperatures,
"floor1/kitchen",
"kitchen-reader"};

Or

C++
// Helper (template params deduced)
Topic<string, float> temperatures{node, "temperatures"};
auto reader = makeSingleKeyReader(temperatures, "floor1/kitchen", "kitchen-reader");
C++
// Constructor
Topic<string, float> temperatures{node, "temperatures"};
MultiKeyReader<string, float> reader{
temperatures,
{"floor1/kitchen", "floor1/living-room"},
"first-floor-reader"};

Or

C++
// Helper (template params deduced)
Topic<string, float> temperatures{node, "temperatures"};
auto reader = makeMultiKeyReader(
temperatures,
{"floor1/kitchen", "floor1/living-room"},
"first-floor-reader");
C++
// Constructor: empty key set
Topic<string, float> temperatures{node, "temperatures"};
MultiKeyReader<string, float> reader{temperatures, {}, "temperature-reader"};

Or

C++
// Helper (template params deduced)
Topic<string, float> temperatures{node, "temperatures"};
auto reader = makeAnyKeyReader(temperatures, "temperature-reader");
C++
// Constructor
Topic<string, float> temperatures{node, "temperatures"};
FilteredKeyReader<string, float> reader{
temperatures,
Filter<string>("startsWith", "floor1/"),
"temperature-reader"};

Or

C++
// Helper (template params deduced)
Topic<string, float> temperatures{node, "temperatures"};
auto reader = makeFilteredKeyReader(
temperatures,
Filter<string>("startsWith", "floor1/"),
"temperature-reader");

All reader types support sample filtering. A reader specifies the sample filter (name + criteria), but the filter is executed by the writer. This minimizes network usage: only samples matching the filter are sent. Unlike key filters (which test only keys), sample filters can inspect the entire sample.

  • The filter name must match a sample filter registered on the publisher’s topic.
  • The criteria type must match the type used to register the filter; otherwise, the writer cannot unmarshal the criteria.

Example: receive temperatures lower than 18 °C:

C++
Topic<string, float> temperatures{node, "temperatures"};
auto reader = makeSingleKeyReader(
temperatures,
"floor1/kitchen",
Filter<float>("lower-than", 18.0f),
"kitchen-reader");

Readers provide methods to retrieve unread samples:

  • getNextUnread — retrieves the next unread sample (removes it from the queue), waiting if none is queued.
  • getAllUnread — retrieves all unread samples (removes them from the queue).
  • waitForUnread — blocks until the given number of unread samples are available.
  • hasUnread — checks if there are unread samples in the queue.

You can register callbacks with onSamples(init, queue), which replaces any callbacks registered before. init receives the samples in the unread queue at the time you register the callbacks, if there are any; queue receives each sample the reader receives afterwards. Either function can be empty. The callbacks do not remove samples from the unread queue; a reader that uses only callbacks typically sets sampleCount to 0.

Callbacks are executed by the node’s callback executor. By default, the executor uses a dedicated thread; you can supply a custom executor via NodeOptions::customExecutor when constructing the node.

Reader behavior is configurable via DataStorm::ReaderConfig. You can set:

Precedence: lower levels override higher ones (per-reader config > topic defaults > global properties). DataStorm takes each option you leave unset from the next level.

Specifies the maximum number of samples kept in the unread queue. When the queue is full, the oldest samples are discarded. A negative value, the default, sets no limit; 0 keeps no samples. Note that with the default clearHistory policy, OnAll, the queue holds at most one sample.

Specifies how long a sample stays in the unread queue, in milliseconds. DataStorm removes samples older than this duration from the queue. 0, the default, or a negative value sets no age limit.

Controls when the reader’s unread sample queue is cleared, based on sample events (ClearHistoryPolicy). DataStorm clears the queue before it queues the sample that triggers the clearing, so the queue then holds only this sample. Default: OnAll.

  • OnAdd: Clears the unread queue on receiving an Add sample.
  • OnRemove: Clears the unread queue on receiving a Remove sample.
  • OnAll: Clears the queue on receiving any sample.
  • OnAllExceptPartialUpdate: Clears the queue on receiving any sample except a PartialUpdate.
  • Never: Never clears the queue automatically.

Whether to discard samples on receipt of new samples (DiscardPolicy). Default: None.

  • None: Never discard.
  • SendTime: Discard if the new sample’s timestamp is at or before the latest timestamp the reader has accepted.
  • Priority: Keep only the samples from the writers with the highest priority among the writers connected for the sample's key.
  • Use onConnectedKeys(initCallback, updateCallback) to register callbacks that monitor connected keys, replacing any callbacks registered before:

    • DataStorm calls initCallback once with the initial set of connected keys.
    • DataStorm calls updateCallback whenever a key is connected or disconnected.
  • Use onConnectedWriters(initCallback, updateCallback) to register callbacks that monitor connected writers, replacing any callbacks registered before:

    • DataStorm calls initCallback once with the initial set of connected writers.
    • DataStorm calls updateCallback whenever a writer connects or disconnects.