Core NATS is the foundational functionality in a NATS system. It operates on a publish-subscribe model using subject/topic-based addressing. This model offers two significant advantages: location independence and a default many-to-many (M:N) communication pattern. These fundamental concepts enable powerful and innovative solutions for common development patterns, such as microservices, without requiring additional technologies like load balancers, API gateways, or DNS configuration.

NATS systems can be enhanced with JetStream, which adds persistence capabilities. While Core NATS provides best-effort, at-most-once message delivery, JetStream introduces at-least-once and exactly-once semantics.

The official NATS documentation

NATS is a simple, secure and high performance open source data layer for cloud native applications, IoT messaging, and microservices architectures.

We feel that it should be the backbone of your communication between services. It doesn't matter what language, protocol, or platform you are using; NATS is the best way to connect your services.

10,000 foot view

• Publish and subscribe to messages at millions of messages per second. At most once delivery.

• Supports fan-in/out delivery patterns

• Request/reply

• Every major language is supported

• Persistence via JetStream

  • at least once delivery or exactly once delivery

  • work queues

  • stream processing

  • data replication

  • data retention

  • data deduplication

  • Higher order data structures

    • Key/Value with watchers, versioning, and TTL

    • Object storage with versioning

• Security

  • TLS

  • JWT-based zero trust security

• Clustering

  • High availability

  • Fault tolerance

  • Auto-discovery

• Protocols supported

  • TCP

  • MQTT

  • WebSockets

All of this in a single binary that is easy to deploy and manage. No external dependencies, just drop it in and add a configuration file to point to other NATS servers and you are ready to go. In fact, you can even embed NATS in your application (for Go users)!

Guided tour

1. In general we recommend trying to solve your problems first using Core NATS.

2. If you need to share state between services, take a look at the KV or Object Store in JetStream.

3. When you need lower level access to persistence streams, move on to using JetStream directly for more advanced messaging patterns.

4. Learn about deployment strategies

5. Secure your deployments with zero trust security

Contribute

NATS is Open Source as is this documentation. Please let us know if you have updates and/or suggestions for these docs. You can also create a Pull Request using the Edit on GitHub link on each page.

Additional questions?

Feel free to chat with us on Slack slack.nats.io.

Thank you from the entire NATS Team of Maintainers for your interest in NATS!

NATS supports several kinds of connectivity directly to the NATS servers.

• Plain NATS connections

• TLS encrypted NATS connections

• WebSocket NATS connections

• MQTT client connections

There is also a number of adapters available to bridge traffic to and from other messaging systems

• Kafka Bridge

• JMS which can also be used to bridge MQ and RabbitMQ, since they both offer a JMS interface

In general, applications can receive messages asynchronously or synchronously. Receiving messages with NATS can be library dependent.

Some languages, like Go or Java, provide synchronous and asynchronous APIs, while others may only support one type of subscription.

In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. When an application is done with a subscription it unsubscribes telling the server to stop sending messages.

A client will receive a message for each matching subscription, so if a connection has multiple subscriptions using identical or overlapping subjects (say foo and >) the same message will be sent to the client multiple times.

Note: The client API (asynchronous) subscribe call can return before the subscription is actually fully established at the nats-server. Call Flush() on the connection right after you call subscribe if you need to synchronize with the subscription being ready at the server level.

Consider this architecture

While it is an incomplete architecture it does show a number of key points:

• Many related subjects are stored in a Stream

• Consumers can have different modes of operation and receive just subsets of the messages

• Multiple Acknowledgement modes are supported

A new order arrives on ORDERS.received, gets sent to the NEW Consumer who, on success, will create a new message on ORDERS.processed. The ORDERS.processed message again enters the Stream where a DISPATCH Consumer receives it and once processed it will create an ORDERS.completed message which will again enter the Stream. These operations are all pull based meaning they are work queues and can scale horizontally. All require acknowledged delivery ensuring no order is missed.

All messages are delivered to a MONITOR Consumer without any acknowledgement and using Pub/Sub semantics - they are pushed to the monitor.

As messages are acknowledged to the NEW and DISPATCH Consumers, a percentage of them are Sampled and messages indicating redelivery counts, ack delays and more, are delivered to the monitoring system.

Example Configuration

Additional documentation introduces the nats utility and how you can use it to create, monitor, and manage streams and consumers, but for completeness and reference this is how you'd create the ORDERS scenario. We'll configure a 1 year retention for order related messages:

nats stream add ORDERS --subjects "ORDERS.*" --ack --max-msgs=-1 --max-bytes=-1 --max-age=1y --storage file --retention limits --max-msg-size=-1 --discard=old
nats consumer add ORDERS NEW --filter ORDERS.received --ack explicit --pull --deliver all --max-deliver=-1 --sample 100
nats consumer add ORDERS DISPATCH --filter ORDERS.processed --ack explicit --pull --deliver all --max-deliver=-1 --sample 100
nats consumer add ORDERS MONITOR --filter '' --ack none --target monitor.ORDERS --deliver last --replay instant

In order for a NATS client application to connect to the NATS service, and then subscribe or publish messages to subjects, it needs to be able to be configured with the details of how to connect to the NATS service infrastructure and of how to authenticate with it.

NATS URL

1. A 'NATS URL' is a string (in a URL format) that specifies the IP address and port where the NATS server(s) can be reached, and what kind of connection to establish:

   • TLS encrypted only TCP connection (i.e. NATS URLs starting with tls://...)

   • TLS encrypted if the server is configured for it or plain un-encrypted TCP connection otherwise (i.e. NATS URLs starting with nats://...)

   • Websocket connection (i.e. NATS URLs starting with ws://...)

Connecting to clusters

Note that when connecting to a NATS service infrastructure with clusters there is more than one URL and the application should allow for more than one URL to be specified in its NATS connect call (typically you pass a comma separated list of URLs as the URL, e.g. "nats://server1:port1,nats://server2:port2").

When connecting to a cluster it is best to provide the complete set of 'seed' URLs for the cluster.

Authentication details

1. If required: authentication details for the application to identify itself with the NATS server(s). NATS supports multiple authentication schemes:

   • (which can be passed as part of the NATS URL)

   • (where the application is configured with the location of 'credentials file' containing the JWT and private Nkey)

   • (where the application is configured with a Token string)

   • (where the client is configured to use a client TLS certificate and the servers are configured to map the TLS client certificates to users defined in the server configuration)

   • (where the client is configured with a Seed and User NKeys)

Runtime configuration

Your application should expose a way to be configured at run time with the NATS URL(s) to use. If you want to use a secure infrastructure, the application must provide for the definition of either the credentials file (.creds) to use, or the means to encode the token, or Nkey, in the URL(s).

Connection Options

Besides the connectivity and security details, there are numerous options for a NATS connection ranging from to to setting in your application.

See Also

WebSocket and NATS

NATS WebSockets and React

Publish-Subscribe

NATS implements a publish-subscribe message distribution model for one-to-many communication. A publisher sends a message on a subject and any active subscriber listening on that subject receives the message. Subscribers can also register interest in wildcard subjects that work a bit like a regular expression (but only a bit). This one-to-many pattern is sometimes called a fan-out.

Messages

Messages are composed of:

1. A subject.

2. A payload in the form of a byte array.

3. Any number of header fields.

4. An optional 'reply' address field.

Messages have a maximum size (which is set in the server configuration with max_payload). The size is set to 1 MB by default, but can be increased up to 64 MB if needed (though we recommend keeping the max message size to something more reasonable like 8 MB).

NATS supports messaging. In this tutorial you explore how to exchange point-to-point messages using NATS.

Prerequisites

If you have not already done so, you need to the nats CLI Tool and optionally, the nats-server on your machine.

Walkthrough

Start two terminal sessions. These will be used to run the NATS request and reply clients.

In one terminal, run the reply client listener

You should see the message: Listening on [help.please]

This means that the NATS receiver client is listening for request messages on the "help.please" subject. In NATS, the receiver is a subscriber.

In the other terminal, run the request client

The NATS requestor client makes a request by sending the message "I need help!" on the “help.please” subject.

The NATS receiver client receives the message, formulates the reply ("OK, I CAN HELP!!!"), and sends it to the inbox of the requester.

NATS is a client/server system in the fact that you have 'NATS client applications' (applications using one of the NATS client libraries) that connect to 'NATS servers' that provide the NATS service. The NATS servers work together to provide a NATS service infrastructure to their client applications.

NATS is extremely flexible and scalable and allows the service infrastructure to be as small as a single process running locally on your local machine and as large as an 'Internet of NATS' of Leaf Nodes, and Leaf Node clusters all interconnected in a secure way over a global shared NATS super-cluster.

Regardless of the size and complexity of the NATS service infrastructure being used, the only configuration needed by the client applications being the location (NATS URLs) of one or more NATS servers and depending on the required security, their credentials.

Note that if your application is written in Golang then you even have the option of embedding the NATS server functionality into the application itself (however you need to then configure your application instances with nats-server configuration information).

You do not actually need to run your NATS service infrastructure, instead you can instead make use of a public NATS infrastructure offered by a NATS Service Provider such as , think of Synadia Cloud as being an 'Internet of NATS' (literally an "InterNATS") and of Synadia as being an "InterNATS Service Provider".

The Evolution of your NATS service infrastructure

You will typically start by running a single instance of nats-server on your local development machine, and have your applications connect to it while you do your application development and local testing.

Next you will probably want to start testing and running those applications and servers in a VPC, or a region or in some on-prem location, so you will deploy either single NATS server or clusters of NATS servers in your VPCs/regions/on-prem/etc... locations and in each location have the applications connect their local nats-server or nats-server cluster. You can then connect those local nats-servers or local nats-server clusters together by making them leaf nodes connecting to a 'backbone' cluster or super-cluster, or by connecting them directly together via gateway connections.

If you have many client applications (e.g., applications deployed on end-user devices all over the Internet, or for example many IoT devices) or many servers in a lot of locations you will then scale your NATS service infrastructure by deploying clusters of NATS servers in multiple locations and multiple cloud providers and VPCs. You will then need to connect those clusters into a global super-cluster and then devise a scheme to intelligently direct your client applications to the right 'closest' NATS server cluster.

Running your own NATS service infrastructure

You can deploy and run your own NATS service infrastructure of nats-server instances, composed of servers, clusters of servers, super-cluster and leaf node NATS servers.

Virtualization and containerization considerations

If using Kubernetes we recommend you use the .

Software applications and services need to exchange data. NATS is an infrastructure that allows such data exchange, segmented in the form of messages. We call this a "message oriented middleware".

With NATS, application developers can:

• Effortlessly build distributed and scalable client-server applications.

• Store and distribute data in realtime in a general manner. This can flexibly be achieved across various environments, languages, cloud providers and on-premises systems.

NATS Client Applications

Developers use one of the NATS client libraries in their application code to allow them to publish, subscribe, request and reply between instances of the application or between completely separate applications. Those applications are generally referred to as 'client applications' or sometimes just as 'clients' throughout this manual (since from the point of view of the NATS server, they are clients).

NATS Service Infrastructure

The NATS services are provided by one or more NATS server processes that are configured to interconnect with each other and provide a NATS service infrastructure. The NATS service infrastructure can scale from a single NATS server process running on an end device (the nats-server process is less than 20 MB in size!) all the way to a public global super-cluster of many clusters spanning all major cloud providers and all regions of the world such as Synadia's NGS.

Connecting NATS Client applications to the NATS servers

To connect a NATS client application with a NATS service, and then subscribe or publish messages to subjects, it only needs to be configured with:

1. URL: A . This is a string (in a URL format) that specifies the IP address and port where the NATS server(s) can be reached, and what kind of connection to establish (plain TCP, TLS, or Websocket).

2. Authentication (if needed): details for the application to identify itself with the NATS server(s). NATS supports multiple authentication schemes (username/password, decentralized JWT, token, TLS certificates and Nkey with challenge).

Simple messaging design

NATS makes it easy for applications to communicate by sending and receiving messages. These messages are addressed and identified by subject strings, and do not depend on network location.

Data is encoded and framed as a message and sent by a publisher. The message is received, decoded, and processed by one or more subscribers.

With this simple design, NATS lets programs share common message-handling code, isolate resources and interdependencies, and scale by easily handling an increase in message volume, whether those are service requests or stream data.

NATS Quality of service (QoS)

NATS offers multiple qualities of service, depending on whether the application uses just the Core NATS functionality or also leverages the added functionalities enabled by NATS JetStream (JetStream is built into nats-server but may not be enabled on all service infrastructures).

• At most once QoS: Core NATS offers an at most once quality of service. If a subscriber is not listening on the subject (no subject match), or is not active when the message is sent, the message is not received. This is the same level of guarantee that TCP/IP provides. Core NATS is a fire-and-forget messaging system. It will only hold messages in memory and will never write messages directly to disk.

• At-least / exactly once QoS: If you need higher qualities of service (at least once and exactly once), or functionalities such as persistent streaming, de-coupled flow control, and Key/Value Store, you can use , which is built in to the NATS server (but needs to be enabled). Of course, you can also always build additional reliability into your client applications yourself with proven and scalable reference designs such as acks and sequence numbers.

NATS sends and receives messages using a protocol that includes a target subject, an optional reply subject and an array of bytes. Some libraries may provide helpers to convert other data formats to and from bytes, but the NATS server will treat all messages as opaque byte arrays.

All of the NATS clients are designed to make sending a message simple. For example, to send the string “All is Well” to the “updates” subject as a UTF-8 string of bytes you would do:

JetStream, the persistence layer of NATS, not only allows for the higher qualities of service and features associated with 'streaming', but it also enables some functionalities not found in messaging systems.

One such feature is the Object store functionality, which allows client applications to create buckets (corresponding to streams) that can store a set of files. Files are stored and transmitted in chunks, allowing files of arbitrary size to be transferred safely over the NATS infrastructure.

Note: Object store is not a distributed storage system. All files in a bucket will need to fit on the target file system.

Basic Capabilities

The Object Store implements a chunking mechanism, allowing you to for example store and retrieve files (i.e. the object) of any size by associating them with a path or file name as the key.

• add a bucket to hold the files.

• put Add a file to the bucket

• get Retrieve the file and store it to a designated location

• del Delete a file

Advanced Capabilities

• watch Subscribe to changes in the bucket. Will receive notifications on successful put and del operations.

We have provided Walkthroughs for you to try NATS (and JetStream) on your own. In order to follow along with the walkthroughs, you could choose one of these options:

• The nats CLI tool must be installed, and a local NATS server must be installed (or you can use a remote server you have access to).

• You can use Synadia's NGS.

• You could even use the demo server from where you installed NATS. This is accessible via nats://demo.nats.io (this is a NATS connection URL; not a browser URL. You pass it to a NATS client application).

Installing the nats CLI Tool

Please refer to the installation section in the readme.

Installing the NATS server locally (if needed)

If you are going to run a server locally you need to first install it and start it. Please refer to the nats server installation doc

Alternatively if you already know how to use NATS on a remote server, you only need to pass the server URL to nats using the -s option or preferably create a context using nats context add, to specify the server URL(s) and credentials file containing your user JWT.

Start the NATS server (if needed)

To start a simple demonstration server locally, simply run:

nats-server

(or nats-server -m 8222 if you want to enable the HTTP monitoring functionality)

When the server starts successfully, you will see the following messages:

[14524] 2021/10/25 22:53:53.525530 [INF] Starting nats-server
[14524] 2021/10/25 22:53:53.525640 [INF]   Version:  2.6.1
[14524] 2021/10/25 22:53:53.525643 [INF]   Git:      [not set]
[14524] 2021/10/25 22:53:53.525647 [INF]   Name:     NDAUZCA4GR3FPBX4IFLBS4VLAETC5Y4PJQCF6APTYXXUZ3KAPBYXLACC
[14524] 2021/10/25 22:53:53.525650 [INF]   ID:       NDAUZCA4GR3FPBX4IFLBS4VLAETC5Y4PJQCF6APTYXXUZ3KAPBYXLACC
[14524] 2021/10/25 22:53:53.526392 [INF] Starting http monitor on 0.0.0.0:8222
[14524] 2021/10/25 22:53:53.526445 [INF] Listening for client connections on 0.0.0.0:4222
[14524] 2021/10/25 22:53:53.526684 [INF] Server is ready

The NATS server listens for client connections on TCP Port 4222.

What is NATS? NATS is a connective technology that powers modern distributed systems. A connective technology is responsible for addressing, discovery and exchanging of messages that drive the common patterns in distributed systems; asking and answering questions, aka services/microservices, and making and processing statements, or stream processing.

Challenges faced by modern distributed systems Modern distributed systems are defined by an ever increasing number of hyper-connected moving parts and the additional data they generate. They employ both services and streams to drive business value. They are also being defined by location independence and mobility, and not just for things we would typically recognize as front end technologies. Today’s systems and the backend processes, microservices and stream processing are being asked to be location independent and mobile as well, all while being secure.

These modern systems present challenges to technologies that have been used to connect mobile front ends to fairly static backends. These incumbent technologies typically manage addressing and discovery via hostname (DNS) or IP and port, utilize a 1:1 communication pattern, and have multiple different security patterns for authentication and authorization. Although not perfect, incumbent technologies have been good enough in many situations, but times are changing quickly. As microservices, functions, and stream processing are being asked to move to the edge, these technologies and the assumptions they make are being challenged.

What makes the NATS connective technology unique for these modern systems?

Effortless M:N connectivity: NATS manages addressing and discovery based on subjects and not hostname and ports. Defaulting to M:N communications, which is a superset of 1:1, meaning it can do 1:1 but can also do so much more. If you have a 1:1 system that is successful in development, ask how many other moving parts are required for production to work around the assumption of 1:1? Things like load balancers, log systems, and network security models, as well as proxies and sidecars. If your production system requires all of these things just to get around the fact that the connective technology being used, e.g. HTTP or gRPC, is 1:1, it’s time to give NATS.io a look.

Deploy anywhere: NATS can be deployed nearly anywhere; on bare metal, in a VM, as a container, inside K8S, on a device, or whichever environment you choose. NATS runs well within deployment frameworks or without.

Secure: Similarly, NATS is secure by default and makes no requirements on network perimeter security models. When you start considering mobilizing your backend microservices and stream processors, many times the biggest roadblock is security.

Scalable, Future-Proof Deployments

NATS infrastructure and clients communicate all topology changes in real-time. This means that NATS clients do not need to change when NATS deployments change. Having to change clients with deployments would be like having to reboot your phone every time your cell provider added or changed a cell tower. This sounds ridiculous of course, but think about how many systems today have their front ends tied so closely to the backend, that any change requires a complete front end reboot or at least a reconfiguration. NATS clients and applications need no such change when backend servers are added and removed and changed. Even DNS is only used to bootstrap first contact, after that, NATS handles endpoint locations transparently.

Hybrid Deployments

Another advantage to utilizing a NATS is that it allows a hybrid mix of SaaS/Utility computing with separately owned and operated systems. Meaning you can have a shared NATS service with core microservices, streams and stream processing be extended by groups or individuals who have a need to run their own NATS infrastructure. You are not forced to choose one or the other.

Adaptability

Today’s systems will fall short with new demands. As modern systems continue to evolve and utilize more components and process more data, supporting patterns beyond 1:1 communications, with addressing and discovery tied to DNS is critical. Foundational technologies like NATS promise the most return on investment. Incumbent technologies will not work as modern systems unify cloud, Edge, IoT and beyond. NATS does.

Use Cases

NATS can run anywhere, from large servers and cloud instances, through edge gateways and even IoT devices. Use cases for NATS include:

• Cloud Messaging

  • Services (microservices, service mesh)

  • Event/Data Streaming (observability, analytics, ML/AI)

• Command and Control

  • IoT and Edge

  • Telemetry / Sensor Data / Command and Control

• Augmenting or Replacing Legacy Messaging Systems

You can use nsc to administer multiple operators. Operators can be thought of as the owners of nats-servers, and fall into two categories: local and managed. The key difference, pardon the pun, is that managed operators are ones which you don't have the nkey for. An example of a managed operator is the Synadia's NGS.

Accounts, as represented by their JWTs, are signed by the operator. Some operators may use local copies of JWTs (i.e. using the memory resolver), but most should use the NATS account resolver built-in to 'nats-server' to manage their JWTs. Synadia uses a custom server for their JWTs that works similarly to the open-sourced account server.

There are a few special commands when dealing with server based operators:

• Account JWTs can be pushed to the server using nsc push

• Account JWTs can be pulled from a server using nsc pull

For managed operators this push/pull behavior is built into nsc. Each time you edit your account JWT nsc will push the change to a managed operator's server and pull the signed response. If this fails the JWT on disk may not match the value on the server. You can always push or pull the account again without editing it. Note - push only works if the operator JWT was configured with an account server URL.

The managed operator will not only sign your account JWT with its key, but may also edit the JWT to include limits to constrain your access to their NATS servers. Some operators may also add demonstration or standard imports. Generally you can remove these, although the operator gets the final call on all Account edits. As with any deployment, the managed operator doesn't track user JWTs.

To start using a managed operator you need to tell nsc about it. There are a couple ways to do this. First you can manually tell nsc to download the operator JWT using the add operator command:

nsc add operator -i

The operator JWT (or details) should be provided to you by the operator. The second way to add a managed operator is with the init command:

nsc init -o synadia -n MyFirstAccount

You can use the name of an existing operator, or a well known one (currently only "synadia").

Once you add a managed operator you can add accounts to it normally, with the caveat that new accounts are pushed and pulled as described above.

Defining "Well Known Operators"

To define a well known operator, you would tell nsc about an operator that you want people in your environment to use by name with a simple environment variable of the form nsc_<operator name>_operator the value of this environment variable should be the URL for getting the operator JWT. For example:

export nsc_zoom_operator=https://account-server-host/jwt/v1/operator

will tell nsc that there is a well known operator named zoom with its JWT at https://account-server-host/jwt/v1/operator. With this definition you can now use the -u flag with the name "zoom" to add the operator to an nsc store directory.

The operator JWT should have its account JWT server property set to point to the appropriate URL. For our example this would be:

nsc edit operator -u https://account-server-host/jwt/v1

You can also set one or more service urls. These allow the nsc tool actions like pub and sub to work. For example:

nsc edit operator -n nats://localhost:4222
nsc tool pub hello world

NATS has a lot of security features:

• Connections can be encrypted with TLS

• Client connections can be authenticated in many ways:

  • Token Authentication

  • Username/Password credentials

  • TLS Certificate

  • NKEY with Challenge

  • Decentralized JWT Authentication/Authorization

  • You can also integrate NATS with your existing authentication/authorization system or create your own custom authentication using the Auth callout

• Authenticated clients are identified as users and have a set of authorizations

You can use accounts for multi-tenancy: each account has its own independent 'subject namespace' and you control the import/export of both streams of messages and services between accounts, and any number of users that client applications can be authenticated as. The subjects or subject wildcards that a user is allowed to publish and/or subscribe to can be controlled either through server configuration or as part of signed JWTs.

JWT authentication/authorization administration is decentralized because each account private key holder can manage their users and their authorizations on their own, without the need for any configuration change on the NATS servers by minting their own JWTs and distributing them to the users. There is no need for the NATS server to ever store any user private keys as they only need to validate the signature chain of trust contained in the user JWT presented by the client application to validate that they have the proper public key for that user.

The JetStream persistence layer of NATS also provides encryption at rest.

nk is a command line tool that generates nkeys. NKeys are a highly secure public-key signature system based on Ed25519.

With NKeys the server can verify identity without ever storing secrets on the server. The authentication system works by requiring a connecting client to provide its public key and digitally sign a challenge with its private key. The server generates a random challenge with every connection request, making it immune to playback attacks. The generated signature is validated a public key, thus proving the identity of the client. If the public key validation succeeds, authentication succeeds.

NKey is an awesome replacement for token authentication, because a connecting client will have to prove it controls the private key for the authorized public key.

Installing nk

To get started with NKeys, you’ll need the nk tool from https://github.com/nats-io/nkeys/tree/master/nk repository. If you have go installed, enter the following at a command prompt:

go install github.com/nats-io/nkeys/nk@latest

Generating NKeys and Configuring the Server

To generate a User NKEY:

nk -gen user -pubout

SUACSSL3UAHUDXKFSNVUZRF5UHPMWZ6BFDTJ7M6USDXIEDNPPQYYYCU3VY
UDXU4RCSJNZOIQHZNWXHXORDPRTGNJAHAHFRGZNEEJCPQTT2M7NLCNF4

The first output line starts with the letter S for Seed. The second letter U stands for User. Seeds are private keys; you should treat them as secrets and guard them with care.

The second line starts with the letter U for User, and is a public key which can be safely shared.

To use nkey authentication, add a user, and set the nkey property to the public key of the user you want to authenticate. You are only required to use the public key and no other properties are required. Here is a snippet of configuration for the nats-server:

authorization: {
  users: [
    { nkey: UDXU4RCSJNZOIQHZNWXHXORDPRTGNJAHAHFRGZNEEJCPQTT2M7NLCNF4 }
  ]
}

To complete the end-to-end configuration and use an nkey, the client is configured to use the seed, which is the private key.

NATS supports two types of revocations. Both of these are stored in the Account JWT, so that the nats-server can see the revocations and apply them.

Users are revoked by public key and time. Access to an export, called an activation, can be revoked for a specific account at a specific time. The use of time here can be confusing, but is designed to support the primary uses of revocation.

When a user or activation is revoked at time T, it means that any user JWT or activation token created before that time is invalid. If a new user JWT or new activation token is created after T it can be used. This allows an account owner to revoke a user and renew their access at the same time.

Let's look at an example. Suppose you created a user JWT with access to the subject "billing". Later you decide you don't want that user to have access to "billing". Revoke the user, say at noon on May 1st 2019, and create a new user JWT without access to "billing". The user can no longer log in with the old JWT because it is revoked, but they can log in with the new JWT because it was created after noon May 1st 2019.

nsc provides a number of commands to create, remove or list revocations:

nsc revocations -h

Manage revocation for users and activations from an account

Usage:
  nsc revocations [command]

Available Commands:
  add-user          Revoke a user
  add_activation    Revoke an accounts access to an export
  delete-user       Remove a user revocation
  delete_activation Remove an account revocation from an export
  list-users        List users revoked in an account
  list_activations  List account revocations for an export

Flags:
  -h, --help   help for revocations

Global Flags:
  -i, --interactive          ask questions for various settings
  -K, --private-key string   private key

Use "nsc revocations [command] --help" for more information about a command.

Both add commands take the flag --at which defaults to 0, for now, which can be used to set the unix timestamp as described above. By default revocations are at the current time, but you can set them in the past for situations where you know when a problem occurred and was fixed.

Deleting a revocation is permanent and can allow an old activation or user JWT to be valid again. Therefore delete should only be used if you are sure the tokens in question have expired.

Pushing the changes to the nats servers

If your nats servers are configured to use the built-in NATS resolver, remember that you need to 'push' any account changes you may have done (locally) using nsc revocations to the servers for those changes to take effect.

i.e. nsc push -i or nsc push -a B -u nats://localhost

If there are any clients currently connected with as a user that gets added to the revocations, their connections will be immediately terminated as soon as you 'push' your revocations to a nats server.

JetStream, the persistence layer of NATS, not only allows for the higher qualities of service and features associated with 'streaming', but it also enables some functionalities not found in messaging systems.

One such feature is the Key/Value store functionality, which allows client applications to create buckets and use them as immediately (as opposed to eventually) consistent, persistent associative arrays (or maps).

Do note, while we do guarantee immediate consistency when it comes to monotonic writes and monotonic reads. We don't guarantee read your writes at this time, as reads through direct get requests may be served by followers or mirrors. More consistent results can be achieved by sending get requests to the underlying stream leader of the Key/Value store.

• Walkthrough

• Details

Managing a Key Value store

1. Create a bucket, which corresponds to a stream in the underlying storage. Define KV/Stream limits as appropriate

2. Use the operation below.

Map style operations

You can use KV buckets to perform the typical operations you would expect from an immediately consistent key/value store:

• put: associate a value with a key

• get: retrieve the value associated with a key

• delete: clear any value associated with a key

• purge: clear all the values associated with all keys

• keys: get a copy of all of the keys (with a value or operation associated with it)

Atomic operations used for locking and concurrency control

• create: associate the value with a key only if there is currently no value associated with that key (i.e. compare to null and set)

• update: compare and set (aka compare and swap) the value for a key

Limiting size, TTL etc.

You can set limits for your buckets, such as:

• the maximum size of the bucket

• the maximum size for any single value

• a TTL: how long the store will keep values for

Treating the Key Value store as a message stream

Finally, you can even do things that typically can not be done with a Key/Value Store:

• watch: watch for changes happening for a key, which is similar to subscribing (in the publish/subscribe sense) to the key: the watcher receives updates due to put or delete operations on the key pushed to it in real-time as they happen

• watch all: watch for all the changes happening on all the keys in the bucket

• history: retrieve a history of the values (and delete operations) associated with each key over time (by default the history of buckets is set to 1, meaning that only the latest value/operation is stored)

Notes

The key conforms to the same naming restriction as a NATS subject, i.e. it can be a dot-separated list of tokens (which means that you can then use wildcards to match hierarchies of keys when watching a bucket), and can only contain valid characters. The value can be any byte array

Recently we have agreed upon an initial specification for a services protocol so that we can add first-class services support to NATS clients and support this in our tooling. This services protocol is an agreement between clients and tooling and doesn't require any special functionality from the NATS server or JetStream.

To check if the NATS client in your favorite language supports the new services API, make sure you check the docs and GitHub repository for that client. The services API is relatively new and not all clients may support it yet.

To see the services API in action in different languages, take a look at the NATS By Example samples.

Concepts

There are a few high level concepts in the services API worth understanding before you start developing your own services.

Service

The service is the highest level abstraction and refers to a group of logically related functionality. Services are required to have names and versions that conform to the semver rules. Services are discoverable within a NATS system.

Endpoint

A service endpoint is the entity with which clients interact. You can think of an endpoint as a single operation within a service. All services must have at least 1 endpoint.

Group

A group is a collection of endpoints. These are optional and can provide a logical association between endpoints as well as an optional common subject prefix for all endpoints.

Service Operations

The services API supports 3 operations for discoverability and observability. While the NATS client will take care of responding on these subjects, it is still the developer's responsibility to respond to requests made the service's actual endpoints.

• PING - Requests made on the $SRV.PING.> subject gather replies from running services. This facilitates service listing by tooling.

• STATS - Requests made on the $SRV.STATS.> subject query statistics from services. Available stats include total requests, total errors, and total processing time.

• INFO - Requests made on the $SRV.INFO.> subject obtain the service definition and metadata, including groups, endpoints, etc.

Using NATS from client application

The most common form of connecting to the NATS messaging system will be through an application built with any of the 40+ client libraries available for NATS.

The client application will connect to an instance of the NATS server, be it a single server, a cluster of servers or even a global super-cluster such as Synadia Cloud, sending and receiving messages via a range of subscribers contracts. If the application is written in GoLang the NATS server can even be embedded into a Go application.

Client APIs will also allow access to almost all server configuration tasks when using an account with sufficient permissions.

Command Line Tooling

Besides using the client API to manage NATS servers, the NATS ecosystem also has many tools to interact with other applications and services over NATS and streams, support server configuration, enhance monitoring or tune performance such as:

• General interaction and management

  • nats - The nats Command Line Tool is the easiest way to interact with, test and manage NATS and JetStream from a terminal or from scripts. It's list of features are ever growing, so please download the latest version.

• Security

  • nk - Generate NKeys for use with JSon Web Tokens (JWT) used with nsc

  • nsc - Configure Operators, Accounts, Users and permission offline to later push them to a production server. This is the preferred tools to create security configuration unless you are using Synadia Control Plane

  • nats account server - (legacy, replaced by the built-in NATS resolver) a custom security server. NAS can still be used as a reference implementation for you tailor-made security integration.

• Monitoring

  • nats top - Monitor NATS Servers

  • prometheus-nats-exporter - Export NATS server metrics to Prometheus and a Grafana dashboard.

• Benchmarking

  • see nats bench subcommand of the nats tool

NATS supports a form of load balancing using queue groups. Subscribers register a queue group name. A single subscriber in the group is randomly selected to receive the message.

Walkthrough prerequisites

If you have not already done so, you need to install the nats CLI Tool and optionally the nats-server on your machine.

1. Start the first member of the queue group

The nats reply instances don't just subscribe to the subject but also automatically join a queue group ("NATS-RPLY-22" by default)

nats reply foo "service instance A Reply# {{Count}}"

2. Start a second member of the queue-group

In a new window

nats reply foo "service instance B Reply# {{Count}}"

3. Start a third member of the queue-group

In a new window

nats reply foo "service instance C Reply# {{Count}}"

4. Publish a NATS message

nats request foo "Simple request"

5. Verify message publication and receipt

You should see that only one of the my-queue group subscribers receives the message and replies it, and you can also see which one of the available queue-group subscribers processed the request from the reply message received (i.e. service instance A, B or C)

6. Publish another message

nats request foo "Another simple request"

You should see that a different queue group subscriber receives the message this time, chosen at random among the 3 queue group members.

You can also send any number of requests back-to-back. From the received messages, you'll see the distribution of those requests amongst the members of the queue-group. For example: nats request foo --count 10 "Request {{Count}}"

7. Stop/start queue-group members

You can at any time start yet another service instance, or kill one and see how the queue-group automatically takes care of adding/removing those instances from the group.

See Also

Queue groups using the NATS CLI

When subscribers register themselves to receive messages from a publisher, the 1:N fan-out pattern of messaging ensures that any message sent by a publisher, reaches all subscribers that have registered. NATS provides an additional feature named "queue", which allows subscribers to register themselves as part of a queue. Subscribers that are part of a queue, form the "queue group".

How queue groups function

As an example, consider message delivery occurring in the 1:N pattern to all subscribers based on the subject name (delivery happens even to subscribers that are not part of a queue group). If a subscriber is registered based on a queue name, it will always receive messages it is subscribed to, based on the subject name. However, if more subscribers are added to the same queue name, they become a queue group, and only one randomly chosen subscriber of the queue group will consume a message each time a message is received by the queue group. Such distributed queues are a built-in load balancing feature that NATS provides.

Advantages

• Ensures application fault tolerance

• Workload processing can be scaled up or down

• Scale your consumers up or down without duplicate messages

• No extra configuration required

• Queue groups are defined by the application and their queue subscribers, rather than the server configuration

Queue group names follow the same naming rules as subjects. Foremost, they are case sensitive and cannot contain whitespace. Consider structuring queue groups hierarchically using a period .. Some server functionalities like queue permissions can use wildcard matching on them.

Queue subscribers are ideal for scaling services. Scale up is as simple as running another application, scale down is terminating the application with a signal that drains the in flight requests. This flexibility and lack of any configuration changes makes NATS an excellent service communication technology that can work with all platform technologies.

No responder

When a request is made to a service (request/reply) and the NATS Server knows there are no services available (since there are no client applications currently subscribing to the subject in a queue-group) the server will send a “no-responders” protocol message back to the requesting client which will break from blocking API calls. This allows applications to react immediately. This further enables building a highly responsive system at scale, even in the face of application failures and network partitions.

Stream as a queue

With JetStream a stream can also be used as a queue by setting the retention policy to WorkQueuePolicy and leveraging pull consumers to get easy horizontal scalability of the processing (or using an explicit ack push consumer with a queue group of subscribers).

Queuing geo-affinity

When connecting to a globally distributed NATS super-cluster, there is an automatic service geo-affinity due to the fact that a service request message will only be routed to another cluster (i.e. another region) if there are no listeners on the cluster available to handle the request locally.

Tutorial

Try NATS queue subscriptions on your own, using a live server by walking through the queueing walkthrough.

Some libraries also provide a special way to connect to a default url, which is generally nats://localhost:4222:

const nc = await connect();
// Do something with the connection
doSomething();
// When done close it
await nc.close();

nc = NATS()
await nc.connect()

# Do something with the connection

await nc.close()

// dotnet add package NATS.Net
using NATS.Net;

await using var client = new NatsClient();

// It's optional to call ConnectAsync()
// as it will be called when needed automatically
await client.ConnectAsync();

require 'nats/client'

NATS.start do |nc|
   # Do something with the connection

   # Close the connection
   nc.close
end

natsConnection      *conn = NULL;
natsStatus          s;

s = natsConnection_ConnectTo(&conn, NATS_DEFAULT_URL);
if (s != NATS_OK)
  // handle error

// Destroy connection, no-op if conn is NULL.
natsConnection_Destroy(conn);

Request-Reply is a common pattern in modern distributed systems. A request is sent, and the application either waits on the response with a certain timeout, or receives a response asynchronously.

The increased complexity of modern systems necessitates features like location transparency, scale-up and scale-down, observability (measuring a system's state based on the data it generates) and more. In order to implement this feature-set, various other technologies needed to incorporate additional components, sidecars (processes or services that support the primary application) and proxies. NATS on the other hand, implemented Request-Reply much more easily.

NATS makes Request-Reply simple and powerful

• NATS supports the Request-Reply pattern using its core communication mechanism — publish and subscribe. A request is published on a given subject using a reply subject. Responders listen on that subject and send responses to the reply subject. Reply subjects are called "inbox". These are unique subjects that are dynamically directed back to the requester, regardless of the location of either party.

• Multiple NATS responders can form dynamic queue groups. Therefore, it's not necessary to manually add or remove subscribers from the group for them to start or stop being distributed messages. It’s done automatically. This allows responders to scale up or down as per demand.

• NATS applications "drain before exiting" (processing buffered messages before closing the connection). This allows the applications to scale down without dropping requests.

• Since NATS is based on publish-subscribe, observability is as simple as running another application that can view requests and responses to measure latency, watch for anomalies, direct scalability and more.

• The power of NATS even allows multiple responses, where the first response is utilized and the system efficiently discards the additional ones. This allows for a sophisticated pattern to have multiple responders, reduce response latency and jitter.

The pattern

Try NATS request-reply on your own, using a live server by walking through the request-reply walkthrough.

No responders

When a request is sent to a subject that has no subscribers, it can be convenient to know about it right away. For this use-case, a NATS client can opt-into no_responder messages. This requires a server and client that support headers. When enabled, a request sent to a subject with no subscribers will immediately receive a reply that has no body, and a 503 status.

Most clients will represent this case by raising or returning an error. For example:

m, err := nc.Request("foo", nil, time.Second);
# err == nats.ErrNoResponders

From a single process to a global super-cluster with leaf node servers, you can always adapt your NATS service deployment to your needs. From servers and VPCs in many clouds, to partially connected small edge devices and everything in between, you can always easily extend and scale your NATS service as your needs grow.

A single server

The simplest version of a NATS service infrastructure is a single nats-server process. The nats-server binary is highly optimized, very lightweight and extremely efficient in its resources' usage.

Client applications establish a connection to the URL of that nats-server process (e.g. "nats://localhost").

A cluster of servers

If you need a fault-tolerant NATS service or if you need to scale your service capacity, you can cluster a set of nats-server processes together in a cluster.

Client applications establish and maintain a connection to (one of) the nats server URL(s) composing the cluster (e.g. "nats://server1","nats://server2",...).

A super-cluster

You can go further than a single cluster and have disaster recovery and get global deployments (e.g. on multiple locations or regions, multiple VPCs or multiple Cloud providers) by deploying multiple clusters and connecting them together via gateway connections (which are interest pruned).

Client applications establish a connection to (one of) the nats server URL(s) of one of the clusters (e.g. "nats://us-west-1.company.com","nats://us-west-2.company.com",...).

With Leaf Nodes

You can easily 'extend' the NATS service provided by a cluster or super-cluster by deploying 'locally' one or more leaf node nats servers that proxy and route traffic between their client applications and the NATS service infrastructure. The context of 'locality' in this case is not just physical: it could mean a location, an edge device or a single development machine, but it could also service a VPC, a group of server processes for a specific application or different accounts, or even a business unit. Leaf node NATS servers can be configured to connect to their cluster over a WebSocket connection (rather than TLS or plain TCP).

Leaf nodes appear to the cluster as a single account connection. Leaf nodes can provide continuous NATS service for their clients, even while being temporarily disconnected from the cluster(s). You can even enable JetStream on the leaf nodes in order to create local streams that are mirrored (mirroring is store and forward and therefore can recover from connectivity outages) to global streams in the upstream cluster(s).

Client applications are configured with the URLs of their 'local' leaf node server(s) and establish a connection to (one of) the leaf node server(s) (e.g. "nats://leaf-node-1","nats://leaf-node-2",...).

See Also

NATS Clusters

NATS Super-clusters

NATS Leaf Nodes

NATS Service Geo-affinity in Queues

NATS is a messaging system . Subscribers listening on a subject receive messages published on that subject. If the subscriber is not actively listening on the subject, the message is not received. Subscribers can use the wildcard tokens such as * and > to match a single token or to match the tail of a subject.

NATS Pub/Sub Walkthrough

This simple walkthrough demonstrates some ways in which subscribers listen on subjects, and publishers send messages on specific subjects.

Walkthrough prerequisites

If you have not already done so, you need to the nats CLI Tool and optionally the nats-server on your machine.

1. Create Subscriber 1

In a shell or command prompt session, start a client subscriber program.

Here, <subject> is a subject to listen on. It helps to use unique and well thought-through subject strings because you need to ensure that messages reach the correct subscribers even when wildcards are used.

For example:

You should see the message: Listening on [msg.test]

2. Create a Publisher and publish a message

In another shell or command prompt, create a NATS publisher and send a message.

Where <subject> is the subject name and <message> is the text to publish.

For example:

3. Verify message publication and receipt

You'll notice that the publisher sends the message and prints: Published [msg.test] : 'NATS MESSAGE'.

The subscriber receives the message and prints: [#1] Received on [msg.test]: 'NATS MESSAGE'.

If the receiver does not get the message, you'll need to check if you are using the same subject name for the publisher and the subscriber.

4. Try publishing another message

You'll notice that the subscriber receives the message. Note that a message count is incremented each time your subscribing client receives a message on that subject.

5. Create Subscriber 2

In a new shell or command prompt, start a new NATS subscriber.

6. Publish another message using the publisher client

Verify that both subscribing clients receive the message.

7. Create Subscriber 3

In a new shell or command prompt session, create a new subscriber that listens on a different subject.

8. Publish another message

Subscriber 1 and Subscriber 2 receive the message, but Subscriber 3 does not. Why? Because Subscriber 3 is not listening on the message subject used by the publisher.

9. Alter Subscriber 3 to use a wildcard

Change the last subscriber to listen on msg.* and run it:

Note: NATS supports the use of wildcard characters for message subscribers only. You cannot publish a message using a wildcard subject.

10. Publish another message

This time, all three subscribing clients should receive the message.

Do try out a few more variations of substrings and wildcards to test your understanding.

See Also

Publish-subscribe pattern with the NATS CLI

When a server goes down, there is a possible anti-pattern called the Thundering Herd where all of the clients try to reconnect immediately, thus creating a denial of service attack. In order to prevent this, most NATS client libraries randomize the servers they attempt to connect to. This setting has no effect if only a single server is used, but in the case of a cluster, randomization, or shuffling, will ensure that no one server bears the brunt of the client reconnect attempts.

However, if you want to disable the randomization process for connect and reconnect, so that servers are always checked in the same order, you can do that in most libraries with a connection option:

Connections can be assigned a name which will appear in some of the server monitoring data. This name is not required, but is highly recommended as a friendly connection name will help in monitoring, error reporting, debugging, and testing.

The Core NATS client libraries try as much as possible to be fire and forget, and you should use JetStream functionalities to get higher qualities of service that can deal with Core NATS messages being dropped due to the server connection being interrupted. That said, one of the features that may be included in the library you are using is the ability to buffer outgoing messages when the connection is down.

During a short reconnect, the client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages once reconnected. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client and an error will be returned.

Be aware, while the message appears to be sent to the application it is possible that it is never sent because the connection is never remade. Your applications should use patterns like acknowledgements or use the JetStream publish call to ensure delivery.

For clients that support this feature, you are able to configure the size of this buffer with bytes, messages or both.

As mentioned throughout this document, each client library may behave slightly differently. Please check the documentation for the library you are using.

NATS 2.2 is the largest feature release since version 2.0. The 2.2 release provides highly scalable, highly performant, secure and easy-to-use next generation streaming in the form of JetStream, allows remote access via websockets, has simplified NATS account management, native MQTT support, and further enables NATS toward our goal of securely democratizing streams and services for the hyperconnected world we live in.

Next Generation Streaming

JetStream is the next generation streaming platform for NATS, highly resilient, highly available, and easy to use. We’ve spent a long time listening to our community, learning from our experiences, looking at the needs of today, and thinking deeply about the needs of tomorrow. We built JetStream to address these needs.

JetStream:

• is easy to deploy and manage, built into the NATS server

• simplifies and accelerates development

• supports wildcard subjects

• supports at least once delivery and exactly once within a window

• is horizontally scalable at runtime with no interruptions

• persists data via streams and delivers or replays via consumers

• supports multiple patterns to consume data on the same stream

• supports push and pull modes when consuming messages

• is account aware

• allows for detailed granularity of security, by stream, by consumer, by function

Get started with JetStream.

Security and Simplified Account Management

Account management just became much easier. This version of NATS has a built-in account management system, eliminating the need to set up an account manager when not using the memory account resolver. With automated default system account generation, and the ability to preload accounts, simply enable a set of servers in your deployment to be account resolvers or account resolver caches, and they will handle public account information provided to the NATS system through the NATS nsc tooling. Have enterprise-scale account management up and running in minutes.

CIDR Block Account Restrictions

By specifying a CIDR block restriction for a user, policy can be applied to limit connections from clients within a certain range or set of IP addresses. Use this as another layer of security atop user credentials to better secure your distributed system. Ensure your applications can only connect from within a specific cloud, enterprise, geographic location, virtual or physical network.

Time-Based Account Restrictions

Scoped to the user, you can now specify a specific block of time during the day when applications can connect. For example, permit certain users or applications to access the system during specified business hours, or protect business operations during the busiest parts of the day from batch driven back-office applications that could adversely impact the system when run at the wrong time.

Default User Permissions

Now you can specify default user permissions within an account. This significantly reduces efforts around policy, reduces chances for error in permissioning, and simplifies the provisioning of user credentials.

WebSockets

Connect mobile and web applications to any NATS server using WebSockets. Built to more easily traverse firewalls and load balancers, NATS WebSocket support provides even more flexibility to NATS deployments and makes it easier to communicate to the edge and endpoints. This is currently supported in NATS server leaf nodes, nats.ts, nats.deno, and the nats.js clients.

Native MQTT Support

With the Adaptive Edge architecture and the ease with which NATS can extend a cloud deployment to the edge, it makes perfect sense to leverage existing investments in IoT deployments. It’s expensive to update devices and large edge deployments. Our goal is to enable the hyperconnected world, so we added first-class support for MQTT 3.1.1 directly into the NATS Server.

Seamlessly integrate existing IoT deployments using MQTT 3.1.1 with a cloud-native NATS deployment. Add a leaf node that is MQTT enabled and instantly send and receive messages to your MQTT applications and devices from a NATS deployment whether it be edge, single-cloud, multi-cloud, on-premise, or any combination thereof.

Build Better Systems

We’ve added a variety of features to allow you to build a more resilient, secure, and simply better system at scale.

Message Headers

We’ve added the ability to optionally use headers, following the HTTP semantics familiar to developers. Headers naturally apply overhead, which was why we resisted adding them for so long. By creating new internal protocol messages transparent to developers, we maintain the extremely fast processing of simple NATS messages that we have always had while supporting headers for those who would like to leverage them. Adding headers to messages allows you to provide application-specific metadata, such as compression or encryption-related information, without touching the payload. We also provide some NATS specific headers for use in JetStream and other features.

Seamless Maintenance with Lame Duck Notifications

When taking down a server for maintenance, servers can be signaled to enter Lame Duck Mode where they do not accept new connections and evict existing connections over a period of time. Maintainer supported clients will notify applications that a server has entered this state and will be shutting down, allowing a client to smoothly transition to another server or cluster and better maintain business continuity during scheduled maintenance periods.

React Quicker with No-Responder Notifications

Why wait for timeouts when services aren’t available? When a request is made to a service (request-reply) and the NATS Server knows there are no services available the server will short circuit the request. A “no-responders” protocol message will be sent back to the requesting client which will break from blocking API calls. This allows applications to immediately react which further enables building a highly responsive system at scale, even in the face of application failures and network partitions.

Subject Mapping and Traffic Shaping

Reduce risk when onboarding new services. Canary deployments, A/B testing, and transparent teeing of data streams are now fully supported in NATS. The NATS Server allows accounts to form subject mappings from one subject to another for both client inbound and service import invocations and allows weighted sets for the destinations. Map any percentage - 1 to 100 percent of your traffic - to other subjects, and change this at runtime with a server configuration reload. You can even artificially drop a percentage of traffic to introduce chaos testing into your system. See Configuring Subject Mapping and Traffic Shaping in NATS Server configuration for more details.

Account Monitoring - More Meaningful Metrics

NATS now allows for fine-grained monitoring to identify usage metrics tied to a particular account. Inspect messages and bytes sent or received and various connection statistics for a particular account. Accounts can represent anything - a group of applications, a team or organization, a geographic location, or even roles. If NATS is enabling your SaaS solution you could use NATS account scoped metrics to bill users.

Streams with source and mirror configurations are best managed through a client API. If you intend to create such a configuration from command line with NATS CLI you should use a JSON configuration.

nats stream add --config stream_with_sources.json

Example stream configuration with two sources

Minimal example

{
  "name": "SOURCE_TARGET",
  "subjects": [
    "foo1.ext.*",
    "foo2.ext.*"
  ],
  "discard": "old",
  "duplicate_window": 120000000000,
  "sources": [
    {
      "name": "SOURCE1_ORIGIN",
    },
  ],
  "deny_delete": false,
  "sealed": false,
  "max_msg_size": -1,
  "allow_rollup_hdrs": false,
  "max_bytes": -1,
  "storage": "file",
  "allow_direct": false,
  "max_age": 0,
  "max_consumers": -1,
  "max_msgs_per_subject": -1,
  "num_replicas": 1,
  "name": "SOURCE_TARGET",
  "deny_purge": false,
  "compression": "none",
  "max_msgs": -1,
  "retention": "limits",
  "mirror_direct": false
}

With additional options

{
  "name": "SOURCE_TARGET",
  "subjects": [
    "foo1.ext.*",
    "foo2.ext.*"
  ],
  "discard": "old",
  "duplicate_window": 120000000000,
  "sources": [
    {
      "name": "SOURCE1_ORIGIN",
      "filter_subject": "foo1.bar",
      "opt_start_seq": 42,
      "external": {
        "deliver": "",
        "api": "$JS.domainA.API"
      }
    },
    {
      "name": "SOURCE2_ORIGIN",
      "filter_subject": "foo2.bar"
    }
  ],
  "consumer_limits": {
    
  },
  "deny_delete": false,
  "sealed": false,
  "max_msg_size": -1,
  "allow_rollup_hdrs": false,
  "max_bytes": -1,
  "storage": "file",
  "allow_direct": false,
  "max_age": 0,
  "max_consumers": -1,
  "max_msgs_per_subject": -1,
  "num_replicas": 1,
  "name": "SOURCE_TARGET",
  "deny_purge": false,
  "compression": "none",
  "max_msgs": -1,
  "retention": "limits",
  "mirror_direct": false
}

Example stream configuration with mirror

Minimal example

{
  "name": "MIRROR_TARGET"
  "discard": "old",
  "mirror": {
    "name": "MIRROR_ORIGIN"
  },
  "deny_delete": false,
  "sealed": false,
  "max_msg_size": -1,
  "allow_rollup_hdrs": false,
  "max_bytes": -1,
  "storage": "file",
  "allow_direct": false,
  "max_age": 0,
  "max_consumers": -1,
  "max_msgs_per_subject": -1,
  "num_replicas": 1,
  "name": "MIRROR_TARGET",
  "deny_purge": false,
  "compression": "none",
  "max_msgs": -1,
  "retention": "limits",
  "mirror_direct": false
}

With additional options

{
  "name": "MIRROR_TARGET"
  "discard": "old",
  "mirror": {
    "opt_start_time": "2024-07-11T08:57:20.4441646Z",
    "external": {
      "deliver": "",
      "api": "$JS.domainB.API"
    },
    "name": "MIRROR_ORIGIN"
  },
  "consumer_limits": {
    
  },
  "deny_delete": false,
  "sealed": false,
  "max_msg_size": -1,
  "allow_rollup_hdrs": false,
  "max_bytes": -1,
  "storage": "file",
  "allow_direct": false,
  "max_age": 0,
  "max_consumers": -1,
  "max_msgs_per_subject": -1,
  "num_replicas": 1,
  "name": "MIRROR_TARGET",
  "deny_purge": false,
  "compression": "none",
  "max_msgs": -1,
  "retention": "limits",
  "mirror_direct": false
}

When a stream is configured with a source or mirror, it will automatically and asynchronously replicate messages from the origin stream.

source or mirror are designed to be robust and will recover from a loss of connection. They are suitable for geographic distribution over high latency and unreliable connections. E.g. even a leaf node starting and connecting intermittently every few days will still receive or send messages over the source/mirror link. Another use case is when connecting streams cross-account.

There are several options available when declaring the configuration.

• Name - Name of the origin stream to source messages from.

• StartSeq - An optional start sequence of the origin stream to start mirroring from.

• StartTime - An optional message start time to start mirroring from. Any messages that are equal to or greater than the start time will be included.

• FilterSubject - An optional filter subject which will include only messages that match the subject, typically including a wildcard. Note, this cannot be used with SubjectTransforms.

• SubjectTransforms - An optional set of subject transforms to apply when sourcing messages from the origin stream. Note, in this context, the Source will act as a filter on the origin stream and the Destination can optionally be provided to apply a transform. Since multiple subject transforms can be used, disjoint subjects can be sourced from the origin stream while maintaining the order of the messages. Note, this cannot be used with FilterSubject.

• Domain - An optional JetStream domain of where the origin stream exists. This is commonly used in a hub cluster and leafnode topology.

The stream using a source or mirror configuration can have its own retention policy, replication, and storage type.

General behavior

• All configurations are made on the receiving side. The stream from which data is sourced and mirrored does not need to be configured. No cleanup is required on the origin side if the receiver disappears.

• A stream can be the origin (source) for multiple streams. This is useful for geographic distribution or for designing "fan out" topologies where data needs to be distributed reliable to a large number (up to millions) of client connections.

• Leaf nodes and leaf node domains are explicitly supported through the API prefix

Source specific

A stream defining Sources is a generalized replication mechanism and allows for sourcing data from one or more streams concurrently. A stream with sources can still act as a regular stream allowing direct write/publish by local clients to the stream. Essentially the source streams and local client writes are aggregated into a single interleaved stream. Combined with subject transformation and filtering sourcing allows to design sophisticated data distribution architectures.

Mirror specific

A mirror can source its messages from exactly one stream and a clients can not directly write to the mirror. Although messages cannot be published to a mirror directly by clients, messages can be deleted on-demand (beyond the retention policy), and consumers have all capabilities available on regular streams.

Expected behavior in edge conditions

• Source and mirror contracts are designed with one-way (geographic) data replication in mind. Neither configuration provides a full synchronization between streams, which would include deletes or replication of other stream attributes.

• The content of the stream from which a source or mirror is drawn needs to be reasonable stable. Quickly deleting messages after publishing them may result in inconsistent replication due to the asynchronous nature of the replication process.

• Sources and Mirror try to be be efficient in replicating messages and are lenient towards the source/mirror origin being unreachable (event for extended periods of time), e.g. when using leaf nodes, which are connected intermittently. For sake of efficiency the recovery interval in case of a disconnect is 10-20s.

• Mirror and source agreements do not create a visible consumer in the origin stream.

WorkQueue retention

Source and mirror works with origin stream with workqueue retention in a limited context. The source/mirror will act as a consumer removing messages from the origin stream.

The implementation is not resilient when connecting over intermittent leaf node connections though. Within a cluster where the target stream (with the source/mirror agreement) it will generally work well.

Interest base retention

The 2.0 version of NATS server introduced the idea of decentralized authentication based on JSON Web Tokens (JWT). Clients interact with this new scheme using a user JWT and corresponding NKey private key. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the nsc tool. The contents will look like the following and should be protected because it contains a private key. This credentials file is unused and only for example purposes.

-----BEGIN NATS USER JWT-----
eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiJUVlNNTEtTWkJBN01VWDNYQUxNUVQzTjRISUw1UkZGQU9YNUtaUFhEU0oyWlAzNkVMNVJBIiwiaWF0IjoxNTU4MDQ1NTYyLCJpc3MiOiJBQlZTQk0zVTQ1REdZRVVFQ0tYUVM3QkVOSFdHN0tGUVVEUlRFSEFKQVNPUlBWV0JaNEhPSUtDSCIsIm5hbWUiOiJvbWVnYSIsInN1YiI6IlVEWEIyVk1MWFBBU0FKN1pEVEtZTlE3UU9DRldTR0I0Rk9NWVFRMjVIUVdTQUY3WlFKRUJTUVNXIiwidHlwZSI6InVzZXIiLCJuYXRzIjp7InB1YiI6e30sInN1YiI6e319fQ.6TQ2ilCDb6m2ZDiJuj_D_OePGXFyN3Ap2DEm3ipcU5AhrWrNvneJryWrpgi_yuVWKo1UoD5s8bxlmwypWVGFAA
------END NATS USER JWT------

************************* IMPORTANT *************************
NKEY Seed printed below can be used to sign and prove identity.
NKEYs are sensitive and should be treated as secrets.

-----BEGIN USER NKEY SEED-----
SUAOY5JZ2WJKVR4UO2KJ2P3SW6FZFNWEOIMAXF4WZEUNVQXXUOKGM55CYE
------END USER NKEY SEED------

*************************************************************

Given a creds file, a client can authenticate as a specific user belonging to a specific account:

nc, err := nats.Connect("127.0.0.1", nats.UserCredentials("path_to_creds_file"))
if err != nil {
    log.Fatal(err)
}
defer nc.Close()

// Do something with the connection

Options options = new Options.Builder()
    .server("nats://localhost:4222")
    .authHandler(Nats.credentials("path_to_creds_file"))
    .build();
Connection nc = Nats.connect(options);

// Do something with the connection

nc.close();

// credentials file contains the JWT and the secret signing key
  const authenticator = credsAuthenticator(creds);
  const nc = await connect({
    port: ns.port,
    authenticator: authenticator,
});

nc = NATS()

async def error_cb(e):
    print("Error:", e)

await nc.connect("nats://localhost:4222",
                 user_credentials="path_to_creds_file",
                 error_cb=error_cb,
                 )

# Do something with the connection

await nc.close()

// dotnet add package NATS.Net
using NATS.Net;

await using var client = new NatsClient("127.0.0.1", credsFile: "/path/to/file.creds");

natsConnection      *conn      = NULL;
natsOptions         *opts      = NULL;
natsStatus          s          = NATS_OK;

s = natsOptions_Create(&opts);
if (s == NATS_OK)
    // Pass the credential file this way if the file contains both user JWT and seed.
    // Otherwise, if the content is split, the first file is the user JWT, the second
    // contains the seed.
    s = natsOptions_SetUserCredentialsFromFiles(opts, "path_to_creds_file", NULL);
if (s == NATS_OK)
    s = natsConnection_Connect(&conn, opts);

(...)

// Destroy objects that were created
natsConnection_Destroy(conn);
natsOptions_Destroy(opts);

All the client libraries maintained on the nats.io GitHub page will automatically attempt to re-connect if their current server connection gets disconnected for any reason. Upon re-connection the client library will automatically re-establish all the subscriptions, there is nothing for the application programmer to do.

Unless specifically disabled client will try to re-connect to one of the servers it knows about, either through the URLs provided in the connect call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self-heal and reconfigure itself with no additional configuration or intervention.

You can adjust the wait time between connections attempts, the maximum number of reconnection attempts, and adjust the size of the reconnection buffer.

Advisories

Your application can register callback to receive events to be notified about the following connection events:

• ClosedCB ConnHandler

The ClosedCB handler is called when a client will no longer be connected.

• DisconnectedCB ConnHandler

The DisconnectedCB handler is called whenever the connection is disconnected. It will not be called if DisconnectedErrCB is set DEPRECATED: Use DisconnectedErrCB instead which passes error that caused the disconnect event.

• DisconnectedErrCB ConnErrHandler

The DisconnectedErrCB handler is called whenever the connection is disconnected. Disconnected error could be nil, for instance when user explicitly closes the connection. NOTE: DisconnectedCB will not be called if DisconnectedErrCB is set

• ReconnectedCB ConnHandler

The ReconnectedCB handler is called whenever the connection is successfully reconnected.

• DiscoveredServersCB ConnHandler

The DiscoveredServersCB handler is called whenever a new server has joined the cluster.

• AsyncErrorCB ErrHandler

The AsyncErrorCB handler is called whenever asynchronous connection errors happen (e.g. slow consumer errors)

Connection timeout attributes

• Timeout time.Duration

Timeout sets the timeout for a Dial operation on a connection. Default is 2 * time.Second

• PingInterval time.Duration

PingInterval is the period at which the client will be sending ping commands to the server, disabled if 0 or negative. Default is 2 * time.Minute

• MaxPingsOut int

MaxPingsOut is the maximum number of pending ping commands that can be awaiting a response before raising an ErrStaleConnection error. Default is 2

Reconnection attributes

Besides the error and advisory callbacks mentioned above you can also set a few reconnection attributes in the connection options:

• AllowReconnect bool

AllowReconnect enables reconnection logic to be used when we encounter a disconnect from the current server. Default is true

• MaxReconnect int

MaxReconnect sets the number of reconnect attempts that will be tried before giving up. If negative, then it will never give up trying to reconnect. Default is 60

• ReconnectWait time.Duration

ReconnectWait sets the time to backoff after attempting to (and failing to) reconnect. Default is 2 * time.Second

• CustomReconnectDelayCB ReconnectDelayHandler

CustomReconnectDelayCB is invoked after the library tried every URL in the server list and failed to reconnect. It passes to the user the current number of attempts. This function returns the amount of time the library will sleep before attempting to reconnect again. It is strongly recommended that this value contains some jitter to prevent all connections to attempt reconnecting at the same time.

• ReconnectJitter time.Duration

ReconnectJitter sets the upper bound for a random delay added to ReconnectWait during a reconnect when no TLS is used. Note that any jitter is capped with ReconnectJitterMax. Default is 100 * time.Millisecond

• ReconnectJitterTLS time.Duration

ReconnectJitterTLS sets the upper bound for a random delay added to ReconnectWait during a reconnect when TLS is used. Note that any jitter is capped with ReconnectJitterMax. Default is 1 * time.Second

• ReconnectBufSize int

ReconnectBufSize is the size of the backing bufio during reconnect. Once this has been exhausted publish operations will return an error. Default is 8 * 1024 * 1024

• RetryOnFailedConnect bool

RetryOnFailedConnect sets the connection in reconnecting state right away if it can't connect to a server in the initial set. The MaxReconnect and ReconnectWait options are used for this process, similarly to when an established connection is disconnected. If a ReconnectHandler is set, it will be invoked on the first successful reconnect attempt (if the initial connect fails), and if a ClosedHandler is set, it will be invoked if it fails to connect (after exhausting the MaxReconnect attempts). Default is false

nats-top is a top-like tool for monitoring nats-server servers.

The nats-top tool provides a dynamic real-time view of a NATS server. nats-top can display a variety of system summary information about the NATS server, such as subscription, pending bytes, number of messages, and more, in real time. For example:

nats-top

nats-server version 0.6.4 (uptime: 31m42s)
Server:
  Load: CPU: 0.8%   Memory: 5.9M  Slow Consumers: 0
  In:   Msgs: 34.2K  Bytes: 3.0M  Msgs/Sec: 37.9  Bytes/Sec: 3389.7
  Out:  Msgs: 68.3K  Bytes: 6.0M  Msgs/Sec: 75.8  Bytes/Sec: 6779.4

Connections: 4
  HOST                 CID      SUBS    PENDING     MSGS_TO     MSGS_FROM   BYTES_TO    BYTES_FROM  LANG     VERSION SUBSCRIPTIONS
  127.0.0.1:56134      2        5       0           11.6K       11.6K       1.1M        905.1K      go       1.1.0   foo, hello
  127.0.1.1:56138      3        1       0           34.2K       0           3.0M        0           go       1.1.0    _INBOX.a96f3f6853616154d23d1b5072
  127.0.0.1:56144      4        5       0           11.2K       11.1K       873.5K      1.1M        go       1.1.0   foo, hello
  127.0.0.1:56151      5        8       0           11.4K       11.5K       1014.6K     1.0M        go       1.1.0   foo, hello

Installation

nats-top can be installed using go install. For example:

go install github.com/nats-io/nats-top

With newer versions of Go, you will be required to use go install github.com/nats-io/nats-top@latest.

NOTE: You may have to run the above command as user sudo depending on your setup. If you receive an error that you cannot install nats-top because your $GOPATH is not set, when in fact it is set, use command sudo -E go get github.com/nats-io/nats-top to install nats-top. The -E flag tells sudo to preserve the current user's environment.

Usage

Once installed, nats-top can be run with the command nats-top and optional arguments.

nats-top [-s server] [-m monitor] [-n num_connections] [-d delay_in_secs] [-sort by]

Options

Optional arguments inclde the following:

Commands

While in nats-top view, you can use the following commands.

option

Use the o<option> command to set the primary sort key to the <option> value. The option value can be one of the following: cid, subs, pending, msgs_to, msgs_from, bytes_to, bytes_from, lang, version.

You can also set the sort option on the command line using the -sort flag. For example: nats-top -sort bytes_to.

limit

Use the n<limit> command to set the sample size of connections to request from the server.

You can also set this on the command line using the -n num_connections flag. For example: nats-top -n 1.

Note that if n<limit> is used in conjunction with -sort, the server will respect both options allowing queries such as the following: Query for the connection with largest number of subscriptions: nats-top -n 1 -sort subs.

s, ? and q Commands

Use the s command to toggle displaying connection subscriptions.

Use the ? command to show help message with options.

Use the q command to quit nats-top.

Tutorial

For a walkthrough with nats-top check out the tutorial.

You can disable automatic reconnect with connection options:

// Disable reconnect attempts
nc, err := nats.Connect("demo.nats.io", nats.NoReconnect())
if err != nil {
    log.Fatal(err)
}
defer nc.Close()

// Do something with the connection

Options options = new Options.Builder()
    .server("nats://demo.nats.io:4222")
    .noReconnect() // Disable reconnect attempts
    .build();
Connection nc = Nats.connect(options);

// Do something with the connection

nc.close();

 const nc = await connect({
    reconnect: false,
    servers: ["demo.nats.io"],
});

nc = NATS()
await nc.connect(
   servers=[
      "nats://demo.nats.io:1222",
      "nats://demo.nats.io:1223",
      "nats://demo.nats.io:1224"
      ],
   allow_reconnect=False,
   )

# Do something with the connection

await nc.close()

// dotnet add package NATS.Net
using NATS.Net;
using NATS.Client.Core;

await using var client = new NatsClient(new NatsOpts
{
    Url = "nats://demo.nats.io:4222",
    
    // .NET client does not support disabling reconnects,
    // but you can set the maximum number of reconnect attempts
    MaxReconnectRetry = 1,
});

require 'nats/client'

NATS.start(servers: ["nats://127.0.0.1:1222", "nats://127.0.0.1:1223", "nats://127.0.0.1:1224"], reconnect: false) do |nc|
   # Do something with the connection

   # Close the connection
   nc.close
end

natsConnection      *conn    = NULL;
natsOptions         *opts    = NULL;
natsStatus          s        = NATS_OK;

s = natsOptions_Create(&opts);
if (s == NATS_OK)
    s = natsOptions_SetAllowReconnect(opts, false);
if (s == NATS_OK)
    s = natsConnection_Connect(&conn, opts);

(...)

// Destroy objects that were created
natsConnection_Destroy(conn);
natsOptions_Destroy(opts);

The NATS client libraries can take a full URL, nats://demo.nats.io:4222, to specify a specific server host and port to connect to.

Libraries are removing the requirement for an explicit protocol and may allow demo.nats.io:4222 or just demo.nats.io. In the later example the default port 4222 will be used. Check with your specific client library's documentation to see what URL formats are supported.

For example, to connect to the demo server with a URL you can use:

// If connecting to the default port, the URL can be simplified
// to just the hostname/IP.
// That is, the connect below is equivalent to:
// nats.Connect("nats://demo.nats.io:4222")
nc, err := nats.Connect("demo.nats.io")
if err != nil {
    log.Fatal(err)
}
defer nc.Close()

// Do something with the connection nc = Nats.connect("nats://demo.nats.io:4222");

// Connection is AutoCloseable
try (Connection nc = Nats.connect("nats://demo.nats.io:4222")) {
    // Do something with the connection
}

const nc = await connect({ servers: "demo.nats.io" });
// Do something with the connection
doSomething();
await nc.close();

nc = NATS()
await nc.connect(servers=["nats://demo.nats.io:4222"])

# Do something with the connection

await nc.close()

// dotnet add package NATS.Net
using NATS.Net;

await using var client = new NatsClient("nats://demo.nats.io:4222");

// It's optional to call ConnectAsync()
// as it will be called when needed automatically
await client.ConnectAsync();

require 'nats/client'

NATS.start(servers: ["nats://demo.nats.io:4222"]) do |nc|
   # Do something with the connection

   # Close the connection
   nc.close
end

natsConnection      *conn = NULL;
natsStatus          s;

// If connecting to the default port, the URL can be simplified
// to just the hostname/IP.
// That is, the connect below is equivalent to:
// natsConnection_ConnectTo(&conn, "nats://demo.nats.io:4222");
s = natsConnection_ConnectTo(&conn, "demo.nats.io");
if (s != NATS_OK)
  // handle error

// Destroy connection, no-op if conn is NULL.
natsConnection_Destroy(conn);

NATS account configurations are built using the nsc tool. The NSC tool allows you to:

• Create and edit Operators, Accounts, Users

• Manage publish and subscribe permissions for Users

• Define Service and Stream exports from an account

• Reference Service and Streams from another account

• Generate Activation tokens that grants access to a private service or stream

• Generate User credential files

• Describe Operators, Accounts, Users, and Activations

• Push and pull account JWTs to an account JWTs server

Installation

Installing nsc is easy:

curl -L https://raw.githubusercontent.com/nats-io/nsc/master/install.py | python

Additional ways of installing nsc are described at nsc's github repository

The script will download the latest version of nsc and install it into your system.

In case NSC is not initialized already do nsc init

Output of tree -L 2 nsc/

nsc/
├── accounts
│   ├── nats
│   └── nsc.json
└── nkeys
    ├── creds
    └── keys
5 directories, 1 file

IMPORTANT: nsc version 2.2.0 has been released. This version of nsc only supports nats-server v2.2.0 and nats-account-server v1.0.0. For more information please refer to the nsc 2.2.0 release notes.

Tutorials

You can find various task-oriented tutorials to working with the tool here:

• Basic Usage

• Configuring Account Streams Import/Export

• Configuring Account Services Import/Export

• Signing Keys

• Revoking Users or Activations

• Working with Managed Operators

Tool Documentation

For more specific browsing of the tool syntax, check out the nsc tool documentation. It can be found within the tool itself:

nsc help

Or an online version here.

NATS client applications use a PING/PONG protocol to check that there is a working connection to the NATS service. Periodically the client will send PING messages to the server, which responds with a PONG. This period is configured by specifying a ping interval on the client connection settings.

The connection will be closed as stale when the client reaches a number of pings which recieved no pong in response, which is configured by specifying the maximum pings outstanding on the client connection settings.

The ping interval and the maximum pings outstanding work together to specify how quickly the client connection will be notified of a problem. This will also help when there is a remote network partition where the operating system does not detect a socket error. Upon connection close, the client will attempt to reconnect. When it knows about other servers, these will be tried next.

In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction.

On connections with significant traffic, the client will often figure out there is a problem between PINGS, and as a result the default ping interval is typically on the order of minutes. To close an unresponsive connection after 100s, set the ping interval to 20s and the maximum pings outstanding to 5:

// Set Ping Interval to 20 seconds and Max Pings Outstanding to 5
nc, err := nats.Connect("demo.nats.io", nats.Name("API Ping Example"), nats.PingInterval(20*time.Second), nats.MaxPingsOutstanding(5))
if err != nil {
    log.Fatal(err)
}
defer nc.Close()

// Do something with the connection

Options options = new Options.Builder()
    .server("nats://demo.nats.io")
    .pingInterval(Duration.ofSeconds(20)) // Set Ping Interval
    .maxPingsOut(5) // Set max pings in flight
    .build();

// Connection is AutoCloseable
try (Connection nc = Nats.connect(options)) {
    // Do something with the connection
}

// Set Ping Interval to 20 seconds and Max Pings Outstanding to 5
const nc = await connect({
    pingInterval: 20 * 1000,
    maxPingOut: 5,
    servers: ["demo.nats.io:4222"],
});

nc = NATS()

await nc.connect(
   servers=["nats://demo.nats.io:4222"],
   # Set Ping Interval to 20 seconds and Max Pings Outstanding to 5
   ping_interval=20,
   max_outstanding_pings=5,
   )

# Do something with the connection.

// dotnet add package NATS.Net
using NATS.Net;
using NATS.Client.Core;

await using var client = new NatsClient(new NatsOpts
{
    Url = "nats://demo.nats.io:4222",
    
    // Set Ping Interval to 20 seconds and Max Pings Outstanding to 5
    PingInterval = TimeSpan.FromSeconds(20),
    MaxPingOut = 5,
});

require 'nats/client'
# Set Ping Interval to 20 seconds and Max Pings Outstanding to 5
NATS.start(ping_interval: 20, max_outstanding_pings: 5) do |nc|
   nc.on_reconnect do
    puts "Got reconnected to #{nc.connected_server}"
  end

  nc.on_disconnect do |reason|
    puts "Got disconnected! #{reason}"
  end

  # Do something with the connection
end

natsConnection      *conn    = NULL;
natsOptions         *opts    = NULL;
natsStatus          s        = NATS_OK;

s = natsOptions_Create(&opts);
if (s == NATS_OK)
    // Set Ping interval to 20 seconds (20,000 milliseconds)
    s = natsOptions_SetPingInterval(opts, 20000);
if (s == NATS_OK)
    // Set the limit to 5
    s = natsOptions_SetMaxPingsOut(opts, 5);
if (s == NATS_OK)
    s = natsConnection_Connect(&conn, opts);

(...)

// Destroy objects that were created
natsConnection_Destroy(conn);
natsOptions_Destroy(opts);

By default a NATS connection will echo messages if the connection also has interest in the published subject. This means that if a publisher on a connection sends a message to a subject any subscribers on that same connection will receive the message. Clients can opt to turn off this behavior, such that regardless of interest, the message will not be delivered to subscribers on the same connection.

The NoEcho option can be useful in BUS patterns where all applications subscribe and publish to the same subject. Usually a publish represents a state change that the application already knows about, so in the case where the application publishes an update it does not need to process the update itself.

Keep in mind that each connection will have to turn off echo, and that it is per connection, not per application. Also, turning echo on and off can result in a major change to your applications communications protocol since messages will flow or stop flowing based on this setting and the subscribing code won't have any indication as to why.

// Turn off echo
nc, err := nats.Connect("demo.nats.io", nats.Name("API NoEcho Example"), nats.NoEcho())
if err != nil {
    log.Fatal(err)
}
defer nc.Close()

// Do something with the connection

Options options = new Options.Builder()
    .server("nats://demo.nats.io:4222")
    .noEcho() // Turn off echo
    .build();
Connection nc = Nats.connect(options);

// Do something with the connection

nc.close();

const nc = await connect({
    servers: ["demo.nats.io"],
    noEcho: true,
});

const sub = nc.subscribe(subj, { callback: (_err, _msg) => {} });
nc.publish(subj);
await sub.drain();
// we won't get our own messages
t.is(sub.getProcessed(), 0);

ncA = NATS()
ncB = NATS()

await ncA.connect(no_echo=True)
await ncB.connect()

async def handler(msg):
   # Messages sent by `ncA' will not be received.
   print("[Received] ", msg)

await ncA.subscribe("greetings", cb=handler)
await ncA.flush()
await ncA.publish("greetings", b'Hello World!')
await ncB.publish("greetings", b'Hello World!')

# Do something with the connection

await asyncio.sleep(1)
await ncA.drain()
await ncB.drain()

// dotnet add package NATS.Net
using NATS.Net;
using NATS.Client.Core;

await using var client = new NatsClient(new NatsOpts
{
    Url = "nats://demo.nats.io:4222",
    
    // Turn off echo
    Echo = false
});

NATS.start("nats://demo.nats.io:4222", no_echo: true) do |nc|
  # ...
end

natsConnection      *conn    = NULL;
natsOptions         *opts    = NULL;
natsStatus          s        = NATS_OK;

s = natsOptions_Create(&opts);
if (s == NATS_OK)
    s = natsOptions_SetNoEcho(opts, true);
if (s == NATS_OK)
    s = natsConnection_Connect(&conn, opts);

(...)

// Destroy objects that were created
natsConnection_Destroy(conn);
natsOptions_Destroy(opts);

Developing with NATS involves a blend of distributed application techniques, common NATS features, and library-specific syntax. Besides this guide, most libraries provide auto-generated API documentation, along with language and platform-specific examples, guides, and other resources.

Not all libraries have their own documentation, depending on the language community, but be sure to check out the client libraries' README for more information.

There are many other NATS client libraries and examples contributed and maintained by the community and available on GitHub, such as:

• , and

• and

• and many ...

Managing the interaction with the server is primarily the job of the client library but most of the libraries also provide some insight into what is happening under the covers.

For example, the client library may provide a mechanism to get the connection's current status:

Some client libraries provide helpers to send structured data while others depend on the application to perform any encoding and decoding and just take byte arrays for sending. The following example shows how to send JSON but this could easily be altered to send a protocol buffer, YAML or some other format. JSON is a text format so we also have to encode the string in most languages to bytes. We are using UTF-8, the JSON standard encoding.

Take a simple stock ticker that sends the symbol and price of each stock:

This guide is tailored for existing NATS users upgrading from NATS version v2.10.x. This will read as a summary with links to specific documentation pages to learn more about the feature or improvement.

Features

Observability

• Distributed message tracing: Users can now trace messages as they move through the system by setting a Nats-Trace-Dest header to an inbox subject. Servers on the message path will return events to the provided subject that report each time a message enters or leaves a server, by which connection type, when subject mappings occur, or when messages traverse an account import/export boundary. Additionally, the Nats-Trace-Only header (if set to true) will allow tracing events to propagate on a specific subject without delivering them to subscribers of that subject.

Streams

• JetStream per-message TTLs: It is now possible to age out individual messages using a per-message TTL. The Nats-TTL header, in either string or integer format (in seconds) allows for individual message expiration independent of stream limits. This can be combined with other limits in place on the stream. More information is available in .

• Subject delete markers on MaxAge: The SubjectDeleteMarkerTTL stream configuration option now allows for the placement of delete marker messages in the stream when the configured MaxAge limit causes the last message for a given subject to be deleted. The delete markers include a Nats-Marker-Reason header explaining which limit was responsible for the deletion.

• Stream ingest rate limiting: New options max_buffered_size and max_buffered_msgs in the jetstream configuration block enable rate limiting on Core NATS publishing into JetStream streams, protecting the system from overload.

Consumers

• Pull consumer priority groups: Pull consumers now support priority groups with pinning and overflow, enabling flexible failover and priority management when multiple clients are pulling from the same consumer. Configurable policies based on the number of pending messages on the consumer, or the number of pending acks, can control when messages overflow from one client to another, enabling new design patterns or regional awareness.

• Consumer pausing: Message delivery to consumers can be temporarily suspended using the new pause API endpoint (or the PauseUntil configuration option when creating), ideal for maintenance or migrations. Message delivery automatically resumes once the configured deadline has passed. Consumer clients continue to receive heartbeat messages as usual to ensure that they do not surface errors during the pause.

Operations

• Replication traffic in asset accounts: Raft replication traffic can optionally be moved into the same account in which replicated assets live on a per-account basis, rather than being sent and received in the system account. When combined with multiple route connections, this can help to reduce latencies and avoid head-of-line blocking issues that may occur in heavily-loaded multi-tenant or multi-account deployments.

• TLS first on leafnode connections: A new handshake_first in the leafnode tls block allows setting up leafnode connections that perform TLS negotiation first, before any other protocol handshakes take place.

• Configuration state digest: A new -t command line flag on the server binary can generate a hash of the configuration file. The config_digest item in varz displays the hash of the currently running configuration file, making it possible to check whether a configuration file has changed on disk compared to the currently running configuration.

• TPM encryption on Windows: When running on Windows, the filestore can now store encryption keys in the TPM, useful in environments where physical access may be a concern.

MQTT

• SparkplugB: The built-in MQTT support is now compliant with SparkplugB Aware, with support for NBIRTH and NDEATH messages.

Improvements

• Replicated delete proposals: Message removals in clustered interest-based or workqueue streams are now propagated via Raft to guarantee consistent removal order across replicas, reducing a number of possible ways that a cluster failure can result in de-synced streams.

• Metalayer, stream and consumer consistency: A new leader now only responds to read/write requests after synchronizing with its Raft log, preventing desynchronization between KV key updates and the stream during leader changes.

• Replicated consumer reliability: Replicated consumers now consistently redeliver unacknowledged messages after a leader change.

• Consumer starting sequence: The consumer starting sequence is now always respected, except for internal hidden consumers for sources/mirrors.

Upgrade Considerations

Stream ingest rate limiting

The NATS Server can now return a 429 error with type JSStreamTooManyRequests when too many messages have been queued up for a stream. It should not generally be possible to hit this limit while using JetStream publishes and waiting for PubAcks, but may trigger if trying to publish into JetStream using Core NATS publishes without waiting for PubAcks, which is not advised.

The new max_buffered_size and max_buffered_msgs options control how many messages can be queued for each stream before the rate limit is hit, therefore if needed, you can increase these limits on your deployments. The default values for max_buffered_size and max_buffered_msgs are 128MB and 10,000 respectively, whereas in v2.10 these were unlimited.

You can detect in the server logs whether running into a queue limit with the following warning:

If your application starts to log the above warnings then you can first try to increase the limits to higher values while investigating the fast publishers, for example:

Replicated delete proposals

Since stream deletes are now replicated through group proposals in a replicated stream, there may be a slight increase in replication traffic on this version.

JetStream healthcheck

The js-server-only healthcheck no longer checks for the health of the metaleader on v2.11.0. Since this healthcheck was designed to detect the server readiness (or in k8s for the readiness probe) checking the metaleader would sometimes cause a NATS server to be considered unhealthy when restarting the servers. In v2.11, this should no longer be an issue. If the previous behavior from v2.10 is preferred, there is a new healthcheck option js-meta-only which can be used to check whether the meta group is healthy.

Exit code

Earlier versions of the NATS Server would return an exit code 1 when gracefully shut down, i.e. after SIGTERM. From v2.11, an exit code of 0 (zero) will now be returned instead.

Server, cluster and gateway names

Configurations that have server, cluster, and gateway names with spaces are now considered invalid, as this can cause problems at the protocol level. A server running NATS v2.11 will fail to start with spaces configured in these names. Please ensure that spaces are not used in server, cluster or gateway names.

Downgrade Considerations

Stream state

When downgrading from v2.11 to v2.10, the stream state files on disk will be rebuilt due to a change in the format of these files in v2.11. This requires re-scanning all stream message blocks, which may use higher CPU than usual and will likely take longer for the restarted node to report healthy. This will only happen on the first restart after downgrading and will not result in data loss.

The Key/Value Store is a JetStream feature, so we need to verify it is enabled by

which may return

In this case, you should enable JetStream.

Prerequisite: enabling JetStream

If you are running a local nats-server stop it and restart it with JetStream enabled using nats-server -js (if that's not already done)

You can then check that JetStream is enabled by using

Creating a KV bucket

A 'KV bucket' is like a stream; you need to create it before using it, as in nats kv add <KV Bucket Name>:

Storing a value

Now that we have a bucket, we can assign, or 'put', a value to a specific key:

which should return the key's value Value1

Getting a value

We can fetch, or 'get', the value for a key "Key1":

Deleting a value

You can always delete a key and its value by using

It is harmless to delete a non-existent key (check this!!).

Atomic operations

K/V Stores can also be used in concurrent design patterns, such as semaphores, by using atomic 'create' and 'update' operations.

E.g. a client wanting exclusive use of a file can lock it by creating a key, whose value is the file name, with create and deleting this key after completing use of that file. A client can increase the reslience against failure by using a timeout for the bucket containing this key. The client can use update with a revision number to keep the bucket alive.

Updates can also be used for more fine-grained concurrency control, sometimes known as optimistic locking, where multiple clients can try a task, but only one can successfully complete it.

Create (aka exclusive locking)

Create a lock/semaphore with the create operation.

Only one create can succeed. First come, first serve. All concurrent attempts will result in an error until the key is deleted

Update with CAS (aka optimistic locking)

We can also atomically update, sometimes known as a CAS (compare and swap) operation, a key with an additional parameter revision

A second attempt with the same revision 13, will fail

Watching a K/V Store

An unusual functionality of a K/V Store is being able to 'watch' a bucket, or a specific key in that bucket, and receive real-time updates to changes in the store.

For the example above, run nats kv watch my-kv. This will start a watcher on the bucket we have just created earlier. By default, the KV bucket has a history size of one, and so it only remembers the last change. In our case, the watcher should see a delete of the value associated with the key "Key1":

If we now concurrently change the value of 'my-kv' by

The watcher will see that change:

Cleaning up

When you are finished using a bucket, you can delete the bucket, and its resources, by using the rm operator:

You can use to monitor in realtime NATS server connections and message statistics.

Prerequisites

1. Install nats-top

You may need to run the following instead:

2. Start the NATS server with monitoring enabled

3. Start nats-top

Result:

4. Run NATS client programs

Run some NATS client programs and exchange messages.

For the best experience, you will want to run multiple subscribers, at least 2 or 3. Refer to the .

5. Check nats-top for statistics

6. Sort nats-top statistics

In nats-top, enter the command o followed by the option, such as bytes_to. You see that nats-top sorts the BYTES_TO column in ascending order.

7. Use different sort options

Use some different sort options to explore nats-top, such as:

cid, subs, pending, msgs_to, msgs_from, bytes_to, bytes_from, lang, version

You can also set the sort option on the command line using the -sort flag. For example: nats-top -sort bytes_to.

8. Display the registered subscriptions.

In nats-top, enter the command s to toggle displaying connection subscriptions. When enabled, you see the subscription subject in nats-top table:

9. Quit nats-top

Use the q command to quit nats-top.

10. Restart nats-top with a specified query

For example, to query for the connection with largest number of subscriptions:

Result: nats-top displays only the client connection with the largest number of subscriptions:

Synchronous subscriptions require the application to wait for messages. This type of subscription is easy to set-up and use, but requires the application to deal with looping if multiple messages are expected. For situations where a single message is expected, synchronous subscriptions are sometimes easier to manage, depending on the language.

For example, to subscribe to the subject updates and receive a single message you could do:

When connecting to a cluster, there are a few things to think about.

• Passing a URL for each cluster member (semi-optional)

• The connection algorithm

• The reconnect algorithm (discussed later)

• Server provided URLs

When a client library first tries to connect it will use the list of URLs provided to the connection options or function. These URLs are usually checked in random order as to not have every client connect to the same server. The first successful connection is used. Randomization can be .

After a client connects to the server, the server may provide a list of URLs for additional known servers. This allows a client to connect to one server and still have other servers available during reconnect.

To ensure the initial connection, your code should include a list of reasonable front line or seed servers. Those servers may know about other members of the cluster, and may tell the client about those members. But you don't have to configure the client to pass every valid member of the cluster in the connect method.

By providing the ability to pass multiple connect options, NATS can handle the possibility of a machine going down or being unavailable to a client. By adding the ability of the server to feed clients a list of known servers as part of the client-server protocol the mesh created by a cluster can grow and change organically while the clients are running.

Note, failure behavior is library dependent, please check the documentation for your client library on information about what happens if the connect fails.

Applications can set the maximum reconnect attempts per server. This includes the server provided to the client's connect call, as well as the server the client discovered through another server. Once reconnect to a server fails the specified amount of times in a row, it will be removed from the connect list. After a successful reconnect to a server, the client will reset that server's failed reconnect attempt count. If a server was removed from the connect list, it can be rediscovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to reconnect, it will close the connection and .

It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, especially when using TLS, libraries provide a wait setting. Generally clients make sure that between two reconnect attempts to the same server at least a certain amount of time has passed. The concrete implementation depends on the library used.

This setting not only prevents wasting client resources, it also alleviates a situation when additional servers are not available.