A consumer is a stateful view of a stream. It acts as interface for clients to consume a subset of messages stored in a stream and will keep track of which messages were delivered and acknowledged by clients.
Unlike with core NATS which provides an at most once delivery guarantee of a message, a consumer can provide an at least once delivery guarantee. This is achieved by the combination of published messages being persisted to the stream as well as the consumer tracking delivery and acknowledgement of each individual message as clients receive and process them. JetStream consumers support multiple kinds of acknowledgements and multiple acknowledgement policies. They will take care of automatically re-deliver un-acked (or 'nacked') messages up to a user specified maximum number of delivery attempts (there is an advisory being emitted when a message reaches this limit).
Consumers can be push-based where messages will be delivered to a specified subject or pull-based which allows clients to request batches of messages on demand. The choice of what kind of consumer to use depends on the use-case but typically in the case of a client application that needs to get their own individual replay of messages from a stream you would use an 'ordered push consumer'. If there is a need to process messages and easily scale horizontally, you would use a 'pull consumer'.
In addition to the choice of being push or pull, a consumer can also be ephemeral or durable. A consumer is considered durable when an explicit name is set on the
Durablefield when creating the consumer, otherwise it is considered ephemeral. Durables and ephemeral behave exactly the same except that an ephemeral will be automatically cleaned up (deleted) after a period of inactivity, specifically when there are no subscriptions bound to the consumer. By default, durables will remain even when there are periods of inactivity (unless
InactiveThresholdis set explicitly).
Below are the set of consumer configuration options that can be defined. The
Versioncolumn indicates the version of the server the option was introduced. The
Editablecolumn indicates the option can be edited after the consumer is created.
The policies choices include:
AckExplicit- The default policy. It means that each individual message must be acknowledged. It is recommended to use this mode, as it provides the most reliability and functionality.
AckNone- You do not have to ack any messages, the server will assume ack on delivery.
AckAll- If you receive a series of messages, you only have to ack the last one you received. All the previous messages received are automatically acknowledged at the same time.
If an ack is required but is not received within the
AckWaitwindow, the message will be redelivered.
The policies choices include:
DeliverAll- The default policy. The consumer will start receiving from the earliest available message.
DeliverLast- When first consuming messages, the consumer will start receiving messages with the last message added to the stream, or the last message in the stream that matches the consumer's filter subject if defined.
DeliverLastPerSubject- When first consuming messages, start with the latest one for each filtered subject currently in the stream.
DeliverNew- When first consuming messages, the consumer will only start receiving messages that were created after the consumer was created.
DeliverByStartSequence- When first consuming messages, start at the first message having the sequence number or the next one available. The consumer is required to specify
OptStartSeqwhich defines the sequence number.
DeliverByStartTime- When first consuming messages, start with messages on or after this time. The consumer is required to specify
OptStartTimewhich defines this start time.
MaxAckPendingcapability provides one-to-many flow control and applies to both push and pull consumers. For push consumers,
MaxAckPendingis the only form of flow control. However, for pull consumers because the delivery of the messages to the client application is client-driven (hence the 'pull') rather than server initiated (hence the 'push') there is an implicit one-to-one flow control with the subscribers (the maximum batch size of the Fetch calls). Therefore you should remember to set it to an appropriately high value (e.g. the default value of 1000), as it can otherwise place a limit on the horizontal scalability of the processing of the stream in high throughput situations.