Ice Docs

IceStorm Persistent Data

4 min read

4 min read

4 min read

4 min read

4 min read

4 min read

4 min read

4 min read

4 min read

IceStorm stores various information in a LMDB database, unless it's in transient mode. This section describes the data stored by IceStorm and database-related constraints.

IceStorm stores the following data in its LMDB database:

  • topics (topic name)
  • subscriptions (subscriber proxy, quality of service for this subscription)
  • topic links (proxy to the target topic, link cost)

A LMDB database consists of one or more persistent key-value maps, and the size of the keys in these maps is limited to 511 bytes.

IceStorm uses a single persistent map with application-provided data. This map has the following key:

Slice
struct SubscriberRecordKey
{
   // The identity of the topic, for example IceStorm/<topic-name>
   Ice::Identity topic;


   // The identity of the subscriber. An empty id corresponds to a topic-only entry.
   Ice::Identity id;
}

This key is streamed into a sequence of bytes using the Ice encoding.

If you attempt to create a topic or register a subscriber with a topic and the resulting SubscriberRecordKey's encoded representation is too large for the LMDB database, IceStorm will throw an Ice::UnknownException.

A LMDB database has a maximum size, known as its map size. The IceStorm database can store up to IceStorm.LMDB.MapSize megabytes of data in its database; any attempt to store more data will fail with an Ice::UnknownException. If you exceed this limit, increase IceStorm.LMDB.MapSize and restart IceStorm.

If you don't set IceStorm.LMDB.MapSize, or set it to 0, IceStorm uses a map size of 10 MB on Windows, and 100 MB on Linux and macOS.

On Windows, LMDB immediately allocates a file with the given map size, while on Linux and macOS LMDB uses sparse files and the allocated data file starts small and grows as needed, until it reaches the configured limit.

You should consider making regular backups of your IceStorm database. We recommend using one of the following tools to perform backups while IceStorm is running: