Non-persistent messaging

This cookbook provides:

Overview

By default, Pulsar persistently stores all unacknowledged messages on multiple bookies (storage nodes). Data for messages on persistent topics can thus survive broker restarts and subscriber failover.

Pulsar also, however, supports non-persistent topics, which are topics on which messages are never persisted to disk and live only in memory. When using non-persistent delivery, killing a Pulsar broker or disconnecting a subscriber to a topic means that all in-transit messages are lost on that (non-persistent) topic, meaning that clients may see message loss.

Non-persistent topics have names of this form (note the non-persistent in the name):

  1. non-persistent://tenant/namespace/topic

For more high-level information about non-persistent topics, see the Concepts and Architecture documentation.

Use

To use non-persistent topics, you need to enable them in your Pulsar broker configuration and differentiate them by name when interacting with them. This pulsar-client produce command, for example, would produce one message on a non-persistent topic in a standalone cluster:

  1. bin/pulsar-client produce non-persistent://public/default/example-np-topic \
  2.   --num-produce 1 \
  3.   --messages "This message will be stored only in memory"

For a more thorough guide to non-persistent topics from an administrative perspective, see the Non-persistent topics guide.

Enable

To enable non-persistent topics in a Pulsar broker, the enableNonPersistentTopics must be set to true. This is the default, so you won’t need to take any action to enable non-persistent messaging.

Configuration for standalone mode

If you’re running Pulsar in standalone mode, the same configurable parameters are available but in the standalone.conf configuration file.

If you’d like to enable only non-persistent topics in a broker, you can set the enablePersistentTopics parameter to false and the enableNonPersistentTopics parameter to true.

Manage with cli

Non-persistent topics can be managed using the pulsar-admin non-persistent command-line interface. With that interface, you can perform actions like create a partitioned non-persistent topic, get stats for a non-persistent topic, list non-persistent topics under a namespace, and more.

Use with Pulsar clients

You shouldn’t need to make any changes to your Pulsar clients to use non-persistent messaging beyond making sure that you use proper topic names with non-persistent as the topic type.