The official NATS documentation

NATS is a simple, secure and high performance open source data layer for cloud native applications, IoT messaging, and microservices architectures.

We feel that it should be the backbone of your communication between services. It doesn't matter what language, protocol, or platform you are using; NATS is the best way to connect your services.

10,000 foot view

• Publish and subscribe to messages at millions of messages per second. At most once delivery.

• Supports fan-in/out delivery patterns

• Request/reply

All of this in a single binary that is easy to deploy and manage. No external dependencies, just drop it in and add a configuration file to point to other NATS servers and you are ready to go. In fact, you can even embed NATS in your application (for Go users)!

Guided tour

1. In general we recommend trying to solve your problems first using .

2. If you need to share state between services, take a look at the or in JetStream.

3. When you need lower level access to persistence streams, move on to using directly for more advanced messaging patterns.

Contribute

NATS is Open Source as is this documentation. Please if you have updates and/or suggestions for these docs. You can also create a Pull Request using the Edit on GitHub link on each page.

Additional questions?

Feel free to chat with us on Slack .

Thank you from the entire NATS Team of Maintainers for your interest in NATS!

Core NATS is the foundational functionality in a NATS system. It operates on a publish-subscribe model using subject/topic-based addressing. This model offers two significant advantages: location independence and a default many-to-many (M:N) communication pattern. These fundamental concepts enable powerful and innovative solutions for common development patterns, such as microservices, without requiring additional technologies like load balancers, API gateways, or DNS configuration.

NATS systems can be enhanced with , which adds persistence capabilities. While Core NATS provides best-effort, at-most-once message delivery, JetStream introduces at-least-once and exactly-once semantics.

What is NATS? NATS is a connective technology that powers modern distributed systems. A connective technology is responsible for addressing, discovery and exchanging of messages that drive the common patterns in distributed systems; asking and answering questions, aka services/microservices, and making and processing statements, or stream processing.

Challenges faced by modern distributed systems Modern distributed systems are defined by an ever increasing number of hyper-connected moving parts and the additional data they generate. They employ both services and streams to drive business value. They are also being defined by location independence and mobility, and not just for things we would typically recognize as front end technologies. Today’s systems and the backend processes, microservices and stream processing are being asked to be location independent and mobile as well, all while being secure.

These modern systems present challenges to technologies that have been used to connect mobile front ends to fairly static backends. These incumbent technologies typically manage addressing and discovery via hostname (DNS) or IP and port, utilize a 1:1 communication pattern, and have multiple different security patterns for authentication and authorization. Although not perfect, incumbent technologies have been good enough in many situations, but times are changing quickly. As microservices, functions, and stream processing are being asked to move to the edge, these technologies and the assumptions they make are being challenged.

What makes the NATS connective technology unique for these modern systems?

Effortless M:N connectivity: NATS manages addressing and discovery based on subjects and not hostname and ports. Defaulting to M:N communications, which is a superset of 1:1, meaning it can do 1:1 but can also do so much more. If you have a 1:1 system that is successful in development, ask how many other moving parts are required for production to work around the assumption of 1:1? Things like load balancers, log systems, and network security models, as well as proxies and sidecars. If your production system requires all of these things just to get around the fact that the connective technology being used, e.g. HTTP or gRPC, is 1:1, it’s time to give NATS.io a look.

Deploy anywhere: NATS can be deployed nearly anywhere; on bare metal, in a VM, as a container, inside K8S, on a device, or whichever environment you choose. NATS runs well within deployment frameworks or without.

Secure: Similarly, NATS is secure by default and makes no requirements on network perimeter security models. When you start considering mobilizing your backend microservices and stream processors, many times the biggest roadblock is security.

Scalable, Future-Proof Deployments

NATS infrastructure and clients communicate all topology changes in real-time. This means that NATS clients do not need to change when NATS deployments change. Having to change clients with deployments would be like having to reboot your phone every time your cell provider added or changed a cell tower. This sounds ridiculous of course, but think about how many systems today have their front ends tied so closely to the backend, that any change requires a complete front end reboot or at least a reconfiguration. NATS clients and applications need no such change when backend servers are added and removed and changed. Even DNS is only used to bootstrap first contact, after that, NATS handles endpoint locations transparently.

Hybrid Deployments

Another advantage to utilizing a NATS is that it allows a hybrid mix of SaaS/Utility computing with separately owned and operated systems. Meaning you can have a shared NATS service with core microservices, streams and stream processing be extended by groups or individuals who have a need to run their own NATS infrastructure. You are not forced to choose one or the other.

Adaptability

Today’s systems will fall short with new demands. As modern systems continue to evolve and utilize more components and process more data, supporting patterns beyond 1:1 communications, with addressing and discovery tied to DNS is critical. Foundational technologies like NATS promise the most return on investment. Incumbent technologies will not work as modern systems unify cloud, Edge, IoT and beyond. NATS does.

Use Cases

NATS can run anywhere, from large servers and cloud instances, through edge gateways and even IoT devices. Use cases for NATS include:

• Cloud Messaging

  • Services (microservices, service mesh)

  • Event/Data Streaming (observability, analytics, ML/AI)

We have provided Walkthroughs for you to try NATS (and JetStream) on your own. In order to follow along with the walkthroughs, you could choose one of these options:

• The nats CLI tool must be installed, and a local NATS server must be installed (or you can use a remote server you have access to).

• You can use Synadia's NGS.

• You could even use the demo server from where you installed NATS. This is accessible via nats://demo.nats.io (this is a NATS connection URL; not a browser URL. You pass it to a NATS client application).

Installing the CLI Tool

Please refer to the .

Installing the NATS server locally (if needed)

If you are going to run a server locally you need to first install it and start it. Please refer to the

Alternatively if you already know how to use NATS on a remote server, you only need to pass the server URL to nats using the -s option or preferably create a context using nats context add, to specify the server URL(s) and credentials file containing your user JWT.

Start the NATS server (if needed)

To start a simple demonstration server locally, simply run:

(or nats-server -m 8222 if you want to enable the HTTP monitoring functionality)

When the server starts successfully, you will see the following messages:

The NATS server listens for client connections on TCP Port 4222.

NATS supports a form of load balancing using queue groups. Subscribers register a queue group name. A single subscriber in the group is randomly selected to receive the message.

Walkthrough prerequisites

If you have not already done so, you need to install the nats CLI Tool and optionally the nats-server on your machine.

1. Start the first member of the queue group

The nats reply instances don't just subscribe to the subject but also automatically join a queue group ("NATS-RPLY-22" by default)

2. Start a second member of the queue-group

In a new window

3. Start a third member of the queue-group

In a new window

4. Publish a NATS message

5. Verify message publication and receipt

You should see that only one of the my-queue group subscribers receives the message and replies it, and you can also see which one of the available queue-group subscribers processed the request from the reply message received (i.e. service instance A, B or C)

6. Publish another message

You should see that a different queue group subscriber receives the message this time, chosen at random among the 3 queue group members.

You can also send any number of requests back-to-back. From the received messages, you'll see the distribution of those requests amongst the members of the queue-group. For example: nats request foo --count 10 "Request {{Count}}"

7. Stop/start queue-group members

You can at any time start yet another service instance, or kill one and see how the queue-group automatically takes care of adding/removing those instances from the group.

See Also

Queue groups using the NATS CLI

Consider this architecture

While it is an incomplete architecture it does show a number of key points:

• Many related subjects are stored in a Stream

• Consumers can have different modes of operation and receive just subsets of the messages

• Multiple Acknowledgement modes are supported

A new order arrives on ORDERS.received, gets sent to the NEW Consumer who, on success, will create a new message on ORDERS.processed. The ORDERS.processed message again enters the Stream where a DISPATCH Consumer receives it and once processed it will create an ORDERS.completed message which will again enter the Stream. These operations are all pull based meaning they are work queues and can scale horizontally. All require acknowledged delivery ensuring no order is missed.

All messages are delivered to a MONITOR Consumer without any acknowledgement and using Pub/Sub semantics - they are pushed to the monitor.

As messages are acknowledged to the NEW and DISPATCH Consumers, a percentage of them are Sampled and messages indicating redelivery counts, ack delays and more, are delivered to the monitoring system.

Example Configuration

introduces the nats utility and how you can use it to create, monitor, and manage streams and consumers, but for completeness and reference this is how you'd create the ORDERS scenario. We'll configure a 1 year retention for order related messages:

NATS has a lot of security features:

• Connections can be encrypted with TLS

• Client connections can be authenticated in many ways:

  • You can also integrate NATS with your existing authentication/authorization system or create your own custom authentication using the

• Authenticated clients are identified as users and have a set of

You can use for multi-tenancy: each account has its own independent 'subject namespace' and you control the import/export of both streams of messages and services between accounts, and any number of users that client applications can be authenticated as. The subjects or subject wildcards that a user is allowed to publish and/or subscribe to can be controlled either through server configuration or as part of signed JWTs.

JWT authentication/authorization administration is decentralized because each account private key holder can manage their users and their authorizations on their own, without the need for any configuration change on the NATS servers by minting their own JWTs and distributing them to the users. There is no need for the NATS server to ever store any user private keys as they only need to validate the signature chain of trust contained in the user JWT presented by the client application to validate that they have the proper public key for that user.

The JetStream persistence layer of NATS also provides .

Publish-Subscribe

NATS implements a publish-subscribe message distribution model for one-to-many communication. A publisher sends a message on a subject and any active subscriber listening on that subject receives the message. Subscribers can also register interest in wildcard subjects that work a bit like a regular expression (but only a bit). This one-to-many pattern is sometimes called a fan-out.

Messages

Messages are composed of:

1. A subject.

2. A payload in the form of a byte array.

3. Any number of header fields.

Messages have a maximum size (which is set in the server configuration with max_payload). The size is set to 1 MB by default, but can be increased up to 64 MB if needed (though we recommend keeping the max message size to something more reasonable like 8 MB).

NATS supports several kinds of connectivity directly to the NATS servers.

• Plain NATS connections

• TLS encrypted NATS connections

• NATS connections

• client connections

There is also a number of adapters available to bridge traffic to and from other messaging systems

• which can also be used to bridge MQ and RabbitMQ, since they both offer a JMS interface

Software applications and services need to exchange data. NATS is an infrastructure that allows such data exchange, segmented in the form of messages. We call this a "message oriented middleware".

With NATS, application developers can:

• Effortlessly build distributed and scalable client-server applications.

• Store and distribute data in realtime in a general manner. This can flexibly be achieved across various environments, languages, cloud providers and on-premises systems.

NATS supports messaging. In this tutorial you explore how to exchange point-to-point messages using NATS.

Prerequisites

If you have not already done so, you need to the nats CLI Tool and optionally, the nats-server on your machine.

NATS is a client/server system in the fact that you have 'NATS client applications' (applications using one of the NATS client libraries) that connect to 'NATS servers' that provide the NATS service. The NATS servers work together to provide a NATS service infrastructure to their client applications.

NATS is extremely flexible and scalable and allows the service infrastructure to be as small as a single process running locally on your local machine and as large as an 'Internet of NATS' of Leaf Nodes, and Leaf Node clusters all interconnected in a secure way over a global shared NATS super-cluster.

Regardless of the size and complexity of the NATS service infrastructure being used, the only configuration needed by the client applications being the location (NATS URLs) of one or more NATS servers and depending on the required security, their credentials.

Note that if your application is written in Golang then you even have the option of embedding the NATS server functionality into the application itself (however you need to then configure your application instances with nats-server configuration information).

When subscribers register themselves to receive messages from a publisher, the 1:N fan-out pattern of messaging ensures that any message sent by a publisher, reaches all subscribers that have registered. NATS provides an additional feature named "queue", which allows subscribers to register themselves as part of a queue. Subscribers that are part of a queue, form the "queue group".

How queue groups function

As an example, consider message delivery occurring in the 1:N pattern to all subscribers based on the subject name (delivery happens even to subscribers that are not part of a queue group). If a subscriber is registered based on a queue name, it will always receive messages it is subscribed to, based on the subject name. However, if more subscribers are added to the same queue name, they become a queue group, and only one randomly chosen subscriber of the queue group will consume a message each time a message is received by the queue group. Such distributed queues are a built-in load balancing feature that NATS provides.

Using NATS from client application

The most common form of connecting to the NATS messaging system will be through an application built with any of the available for NATS.

The client application will connect to an instance of the NATS server, be it a single server, a cluster of servers or even a global super-cluster such as , sending and receiving messages via a range of subscribers contracts. If the application is written in GoLang the NATS server can even be application.

Client APIs will also allow access to almost all server configuration tasks when using an account with sufficient permissions.

JetStream, the persistence layer of NATS, not only allows for the higher qualities of service and features associated with 'streaming', but it also enables some functionalities not found in messaging systems.

One such feature is the Key/Value store functionality, which allows client applications to create buckets and use them as immediately (as opposed to eventually) consistent, persistent (or maps). Note that this is an abstraction on top of the Stream functionality. Buckets are materialized as Streams (with a name starting with KV_), everything you can do with a bucket you can do with a Stream, but you ultimately have more functionality and flexibility and control when using the Stream functionality directly.

Do note, while we do guarantee immediate consistency when it comes to and . We don't guarantee at this time, as reads through direct get requests may be served by followers or mirrors. More consistent results can be achieved by sending get requests to the underlying stream leader of the Key/Value store.

nk is a command line tool that generates nkeys. NKeys are a highly secure public-key signature system based on .

With NKeys the server can verify identity without ever storing secrets on the server. The authentication system works by requiring a connecting client to provide its public key and digitally sign a challenge with its private key. The server generates a random challenge with every connection request, making it immune to playback attacks. The generated signature is validated a public key, thus proving the identity of the client. If the public key validation succeeds, authentication succeeds.

NKey is an awesome replacement for token authentication, because a connecting client will have to prove it controls the private key for the authorized public key.

JetStream, the persistence layer of NATS, not only allows for the higher qualities of service and features associated with 'streaming', but it also enables some functionalities not found in messaging systems.

One such feature is the Object store functionality, which allows client applications to create buckets (corresponding to streams) that can store a set of files. Files are stored and transmitted in chunks, allowing files of arbitrary size to be transferred safely over the NATS infrastructure.

Note: Object store is not a distributed storage system. All files in a bucket will need to fit on the target file system.

Some libraries also provide a special way to connect to a default url, which is generally nats://localhost:4222:

In general, applications can receive messages asynchronously or synchronously. Receiving messages with NATS can be library dependent.

Some languages, like Go or Java, provide synchronous and asynchronous APIs, while others may only support one type of subscription.

In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. When an application is done with a subscription it unsubscribes telling the server to stop sending messages.

A client will receive a message for each matching subscription, so if a connection has multiple subscriptions using identical or overlapping subjects (say foo and >) the same message will be sent to the client multiple times.

NATS sends and receives messages using a protocol that includes a target subject, an optional reply subject and an array of bytes. Some libraries may provide helpers to convert other data formats to and from bytes, but the NATS server will treat all messages as opaque byte arrays.

All of the NATS clients are designed to make sending a message simple. For example, to send the string “All is Well” to the “updates” subject as a UTF-8 string of bytes you would do:

You can use nsc to administer multiple operators. Operators can be thought of as the owners of nats-servers, and fall into two categories: local and managed. The key difference is that managed operators are ones which you don't have the nkey for. An example of a managed operator is Synadia's NGS.

Accounts, as represented by their JWTs, are signed by the operator. Some operators may use local copies of JWTs (i.e. using the memory resolver), but most should use the NATS account resolver built-in to 'nats-server' to manage their JWTs. Synadia uses a custom server for their JWTs that works similarly to the open-sourced account server.

There are a few special commands when dealing with server based operators:

• Account JWTs can be pushed to the server using nsc push

• Account JWTs can be pulled from a server using nsc pull

For managed operators this push/pull behavior is built into nsc. Each time you edit your account JWT nsc will push the change to a managed operator's server and pull the signed response. If this fails the JWT on disk may not match the value on the server. You can always push or pull the account again without editing it. Note - push only works if the operator JWT was configured with an account server URL.

The managed operator will not only sign your account JWT with its key, but may also edit the JWT to include limits to constrain your access to their NATS servers. Some operators may also add demonstration or standard imports. Generally you can remove these, although the operator gets the final call on all Account edits. As with any deployment, the managed operator doesn't track user JWTs.

To start using a managed operator you need to tell nsc about it. There are a couple ways to do this. First you can manually tell nsc to download the operator JWT using the add operator command:

The operator JWT (or details) should be provided to you by the operator. The second way to add a managed operator is with the init command:

You can use the name of an existing operator, or a well known one (currently only "synadia").

Once you add a managed operator you can add accounts to it normally, with the caveat that new accounts are pushed and pulled as described above.

Defining "Well Known Operators"

To define a well known operator, you would tell nsc about an operator that you want people in your environment to use by name with a simple environment variable of the form nsc_<operator name>_operator the value of this environment variable should be the URL for getting the operator JWT. For example:

will tell nsc that there is a well known operator named zoom with its JWT at https://account-server-host/jwt/v1/operator. With this definition you can now use the -u flag with the name "zoom" to add the operator to an nsc store directory.

The operator JWT should have its account JWT server property set to point to the appropriate URL. For our example this would be:

You can also set one or more service urls. These allow the nsc tool actions like pub and sub to work. For example:

In order for a NATS client application to connect to the NATS service, and then subscribe or publish messages to subjects, it needs to be able to be configured with the details of how to connect to the NATS service infrastructure and of how to authenticate with it.

NATS URL

1. A 'NATS URL' is a string (in a URL format) that specifies the IP address and port where the NATS server(s) can be reached, and what kind of connection to establish:

   • TLS encrypted only TCP connection (i.e. NATS URLs starting with tls://...)

   • TLS encrypted if the server is configured for it or plain un-encrypted TCP connection otherwise (i.e. NATS URLs starting with nats://...)

Connecting to clusters

Note that when connecting to a NATS service infrastructure with clusters there is more than one URL and the application should allow for more than one URL to be specified in its NATS connect call (typically you pass a comma separated list of URLs as the URL, e.g. "nats://server1:port1,nats://server2:port2").

When connecting to a cluster it is best to provide the complete set of 'seed' URLs for the cluster.

Authentication details

1. If required: authentication details for the application to identify itself with the NATS server(s). NATS supports multiple authentication schemes:

   • (which can be passed as part of the NATS URL)

   • (where the application is configured with the location of 'credentials file' containing the JWT and private Nkey)

Runtime configuration

Your application should expose a way to be configured at run time with the NATS URL(s) to use. If you want to use a secure infrastructure, the application must provide for the definition of either the credentials file (.creds) to use, or the means to encode the token, or Nkey, in the URL(s).

Connection Options

Besides the connectivity and security details, there are numerous options for a NATS connection ranging from to to setting in your application.

See Also

WebSocket and NATS

NATS WebSockets and React

Request-Reply is a common pattern in modern distributed systems. A request is sent, and the application either waits on the response with a certain timeout, or receives a response asynchronously.

The increased complexity of modern systems necessitates features like location transparency, scale-up and scale-down, observability (measuring a system's state based on the data it generates) and more. In order to implement this feature-set, various other technologies needed to incorporate additional components, sidecars (processes or services that support the primary application) and proxies. NATS on the other hand, implemented Request-Reply much more easily.

NATS makes Request-Reply simple and powerful

• NATS supports the Request-Reply pattern using its core communication mechanism — publish and subscribe. A request is published on a given subject using a reply subject. Responders listen on that subject and send responses to the reply subject. Reply subjects are called "inbox". These are unique subjects that are dynamically directed back to the requester, regardless of the location of either party.

• Multiple NATS responders can form dynamic queue groups. Therefore, it's not necessary to manually add or remove subscribers from the group for them to start or stop being distributed messages. It’s done automatically. This allows responders to scale up or down as per demand.

• NATS applications "drain before exiting" (processing buffered messages before closing the connection). This allows the applications to scale down without dropping requests.

The pattern

Try NATS request-reply on your own, using a live server by walking through the

No responders

When a request is sent to a subject that has no subscribers, it can be convenient to know about it right away. For this use-case, a NATS client can . This requires a server and client that support headers. When enabled, a request sent to a subject with no subscribers will immediately receive a reply that has no body, and a 503 status.

Most clients will represent this case by raising or returning an error. For example:

NATS supports two types of revocations. Both of these are stored in the Account JWT, so that the nats-server can see the revocations and apply them.

Users are revoked by public key and time. Access to an export, called an activation, can be revoked for a specific account at a specific time. The use of time here can be confusing, but is designed to support the primary uses of revocation.

When a user or activation is revoked at time T, it means that any user JWT or activation token created before that time is invalid. If a new user JWT or new activation token is created after T it can be used. This allows an account owner to revoke a user and renew their access at the same time.

Let's look at an example. Suppose you created a user JWT with access to the subject "billing". Later you decide you don't want that user to have access to "billing". Revoke the user, say at noon on May 1st 2019, and create a new user JWT without access to "billing". The user can no longer log in with the old JWT because it is revoked, but they can log in with the new JWT because it was created after noon May 1st 2019.

nsc provides a number of commands to create, remove or list revocations:

Both add commands take the flag --at which defaults to 0, for now, which can be used to set the unix timestamp as described above. By default revocations are at the current time, but you can set them in the past for situations where you know when a problem occurred and was fixed.

Deleting a revocation is permanent and can allow an old activation or user JWT to be valid again. Therefore delete should only be used if you are sure the tokens in question have expired.

Pushing the changes to the nats servers

If your nats servers are configured to use the built-in NATS resolver, remember that you need to 'push' any account changes you may have done (locally) using nsc revocations to the servers for those changes to take effect.

i.e. nsc push -i or nsc push -a B -u nats://localhost

If there are any clients currently connected with as a user that gets added to the revocations, their connections will be immediately terminated as soon as you 'push' your revocations to a nats server.

The Core NATS client libraries try as much as possible to be fire and forget, and you should use JetStream functionalities to get higher qualities of service that can deal with Core NATS messages being dropped due to the server connection being interrupted. That said, one of the features that may be included in the library you are using is the ability to buffer outgoing messages when the connection is down.

During a short reconnect, the client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages once reconnected. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client and an error will be returned.

Be aware, while the message appears to be sent to the application it is possible that it is never sent because the connection is never remade. Your applications should use patterns like acknowledgements or use the JetStream publish call to ensure delivery.

For clients that support this feature, you are able to configure the size of this buffer with bytes, messages or both.

// Set reconnect buffer size in bytes (5 MB)
nc, err := nats.

As mentioned throughout this document, each client library may behave slightly differently. Please check the documentation for the library you are using.

Recently we have agreed upon an initial specification for a services protocol so that we can add first-class services support to NATS clients and support this in our tooling. This services protocol is an agreement between clients and tooling and doesn't require any special functionality from the NATS server or JetStream.

To check if the NATS client in your favorite language supports the new services API, make sure you check the docs and GitHub repository for that client. The services API is relatively new and not all clients may support it yet.

To see the services API in action in different languages, take a look at the NATS By Example samples.

Concepts

There are a few high level concepts in the services API worth understanding before you start developing your own services.

Service

The service is the highest level abstraction and refers to a group of logically related functionality. Services are required to have names and versions that conform to the rules. Services are discoverable within a NATS system.

Endpoint

A service endpoint is the entity with which clients interact. You can think of an endpoint as a single operation within a service. All services must have at least 1 endpoint.

Group

A group is a collection of endpoints. These are optional and can provide a logical association between endpoints as well as an optional common subject prefix for all endpoints.

Service Operations

The services API supports 3 operations for discoverability and observability. While the NATS client will take care of responding on these subjects, it is still the developer's responsibility to respond to requests made the service's actual endpoints.

• PING - Requests made on the $SRV.PING.> subject gather replies from running services. This facilitates service listing by tooling.

• STATS - Requests made on the $SRV.STATS.> subject query statistics from services. Available stats include total requests, total errors, and total processing time.

NATS is a messaging system . Subscribers listening on a subject receive messages published on that subject. If the subscriber is not actively listening on the subject, the message is not received. Subscribers can use the wildcard tokens such as * and > to match a single token or to match the tail of a subject.

NATS Pub/Sub Walkthrough

This simple walkthrough demonstrates some ways in which subscribers listen on subjects, and publishers send messages on specific subjects.

When a stream is configured with a source or mirror, it will automatically and asynchronously replicate messages from the origin stream.

source or mirror are designed to be robust and will recover from a loss of connection. They are suitable for geographic distribution over high latency and unreliable connections. E.g. even a leaf node starting and connecting intermittently every few days will still receive or send messages over the source/mirror link. Another use case is when .

There are several options available when declaring the configuration.

From a single process to a global super-cluster with leaf node servers, you can always adapt your NATS service deployment to your needs. From servers and VPCs in many clouds, to partially connected small edge devices and everything in between, you can always easily extend and scale your NATS service as your needs grow.

A single server

The simplest version of a NATS service infrastructure is a single nats-server process. The nats-server binary is highly optimized, very lightweight and extremely efficient in its resources' usage.

NATS account configurations are built using the nsc tool. The NSC tool allows you to:

• Create and edit Operators, Accounts, Users

• Manage publish and subscribe permissions for Users

Connections can be assigned a name which will appear in some of the server monitoring data. This name is not required, but is highly recommended as a friendly connection name will help in monitoring, error reporting, debugging, and testing.

You can disable automatic reconnect with connection options:

By default a NATS connection will echo messages if the connection also has interest in the published subject. This means that if a publisher on a connection sends a message to a subject any subscribers on that same connection will receive the message. Clients can opt to turn off this behavior, such that regardless of interest, the message will not be delivered to subscribers on the same connection.

The NoEcho option can be useful in BUS patterns where all applications subscribe and publish to the same subject. Usually a publish represents a state change that the application already knows about, so in the case where the application publishes an update it does not need to process the update itself.

Keep in mind that each connection will have to turn off echo, and that it is per connection, not per application. Also, turning echo on and off can result in a major change to your applications communications protocol since messages will flow or stop flowing based on this setting and the subscribing code won't have any indication as to why.

This guide is tailored for existing NATS users upgrading from NATS version v2.11.x. This will read as a summary with links to specific documentation pages to learn more about the feature or improvement.

Features

Streams

• Atomic batch publish: The AllowAtomicPublish stream configuration option allows to atomically publish N messages into a stream. This includes support for replicated and non-replicated streams, as well as doing per-message consistency checks prior to committing the batch. More information is available in .

• Distributed Counter CRDT: The AllowMsgCounter stream configuration option allows increment/decrement counter semantics on a stream. These counter streams can also be mirrored or aggregated through stream mirroring and sourcing. More information is available in

Consumers

• Prioritized pull consumer policy: In addition to the consumer policies like overflow or client pinning, a new prioritized policy has been added. In contrast with the overflow policy, this allows a consumer to receive messages sooner instead of delaying failover, but at the cost of potentially flip-flopping work between clients. More information is available in

Operations

• Server metadata: Similar to server_tags which contains a set of tags describing the server, server_metadata is a map containing string keys and values describing metadata of the server.

• Promoting mirrors: A stream that’s mirroring can now be promoted to be the primary, enabling new disaster recovery methodology. The current primary stream should be deleted or have its configured subjects removed prior to promoting mirrors, before configuring the promoted mirrors to start listening on those subjects.

Improvements

• Async stream flushing: Replicated streams will now asynchronously flush data to the underlying store on disk, resulting in a significant improvement in performance. Writes to a replicated stream are still persisted synchronously in the Raft log prior to committing them, so the improved performance has no downsides with respect to consistency.

• Elastic pointers in the filestore: File-based streams now use elastic pointers for its write-through caches. This allows the server to better respond during garbage collection, these caches can be evicted early to avoid out-of-memory conditions (see upgrade considerations below).

• Use cipher suites from

Upgrade Considerations

Memory usage

With the new elastic pointers in the filestore, it is expected that a NATS Server running 2.12 may show a different memory usage pattern to before. In some systems this may result in lower resident set size (RSS) reported, in others it may result in higher, depending on the number of assets and publish/access patterns.

For the first time, the server will be able to respond to memory pressure by freeing filestore caches on demand and returning the memory to the operating system. This reduces the chance that sudden spikes in utilisation will result in an out-of-memory (OOM) kill. However, this means that the server can more optimistically retain caches in memory when available resources allow in order to facilitate improved read access times.

This behaviour is largely controlled by the GC thresholds as set by the GOMEMLIMIT . You may wish to tune this value in your environment based on available system memory, or in the case of Kubernetes environments, memory reservations.

Strict JetStream API

Starting from version v2.11, the server would start logging the following statement if an invalid JetStream request was received:

Starting from version v2.12, the server will not only log, but also return an error to the client as “strict mode” is now enabled by default. This means invalid JetStream requests will be rejected by default.

If the above log message is observed, please make sure that the application or client is sending correct requests to the server and that NATS client libraries are up-to-date. Strict mode can be temporarily disabled in the server configuration, allowing you more time to fix the issue:

Downgrade Considerations

Stream state

When downgrading from v2.12 to v2.11, the stream state files on disk will be rebuilt due to a change in the format of these files in v2.12. This requires re-scanning all stream message blocks, which may use higher CPU than usual and will likely take longer for the restarted node to report healthy. This will only happen on the first restart after downgrading and will not result in data loss.

When downgrading, only downgrade to v2.11.9 or higher. Starting from this version, the server will recognize the use of new v2.12 features and will safely put the stream and/or consumer that uses these new features into an unsupported/offline mode. Importantly, this will both protect the data as well as the server itself from accessing unsupported features or data.

nats-top is a top-like tool for monitoring nats-server servers.

The nats-top tool provides a dynamic real-time view of a NATS server. nats-top can display a variety of system summary information about the NATS server, such as subscription, pending bytes, number of messages, and more, in real time. For example:

Installation

nats-top can be installed using go install. For example:

With newer versions of Go, you will be required to use go install github.com/nats-io/nats-top@latest.

NOTE: You may have to run the above command as user sudo depending on your setup. If you receive an error that you cannot install nats-top because your $GOPATH is not set, when in fact it is set, use command sudo -E go get github.com/nats-io/nats-top to install nats-top. The -E flag tells sudo to preserve the current user's environment.

Usage

Once installed, nats-top can be run with the command nats-top and optional arguments.

Options

Optional arguments inclde the following:

Commands

While in nats-top view, you can use the following commands.

option

Use the o<option> command to set the primary sort key to the <option> value. The option value can be one of the following: cid, subs, pending, msgs_to, msgs_from, bytes_to, bytes_from, lang, version.

You can also set the sort option on the command line using the -sort flag. For example: nats-top -sort bytes_to.

limit

Use the n<limit> command to set the sample size of connections to request from the server.

You can also set this on the command line using the -n num_connections flag. For example: nats-top -n 1.

Note that if n<limit> is used in conjunction with -sort, the server will respect both options allowing queries such as the following: Query for the connection with largest number of subscriptions: nats-top -n 1 -sort subs.

s, ? and q Commands

Use the s command to toggle displaying connection subscriptions.

Use the ? command to show help message with options.

Use the q command to quit nats-top.

Tutorial

For a walkthrough with nats-top check out the .

The 2.0 version of NATS server introduced the idea of decentralized authentication based on JSON Web Tokens (JWT). Clients interact with this new scheme using a user JWT and corresponding NKey private key. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the nsc tool. The contents will look like the following and should be protected because it contains a private key. This credentials file is unused and only for example purposes.

Given a creds file, a client can authenticate as a specific user belonging to a specific account:

nc, err := nats.Connect("127.0.0.1

Options options = new Options.Builder()
    

// credentials file contains the JWT and the secret signing key
  const authenticator = credsAuthenticator(

The Key/Value Store is a JetStream feature, so we need to verify it is enabled by

which may return

In this case, you should enable JetStream.

Prerequisite: enabling JetStream

If you are running a local nats-server stop it and restart it with JetStream enabled using nats-server -js (if that's not already done)

You can then check that JetStream is enabled by using

Creating a KV bucket

A 'KV bucket' is like a stream; you need to create it before using it, as in nats kv add <KV Bucket Name>:

Storing a value

Now that we have a bucket, we can assign, or 'put', a value to a specific key:

which should return the key's value Value1

Getting a value

We can fetch, or 'get', the value for a key "Key1":

Deleting a value

You can always delete a key and its value by using

It is harmless to delete a non-existent key (check this!!).

Atomic operations

K/V Stores can also be used in concurrent design patterns, such as semaphores, by using atomic 'create' and 'update' operations.

E.g. a client wanting exclusive use of a file can lock it by creating a key, whose value is the file name, with create and deleting this key after completing use of that file. A client can increase the reslience against failure by using a timeout for the bucket containing this key. The client can use update with a revision number to keep the bucket alive.

Updates can also be used for more fine-grained concurrency control, sometimes known as optimistic locking, where multiple clients can try a task, but only one can successfully complete it.

Create (aka exclusive locking)

Create a lock/semaphore with the create operation.

Only one create can succeed. First come, first serve. All concurrent attempts will result in an error until the key is deleted

Update with CAS (aka optimistic locking)

We can also atomically update, sometimes known as a CAS (compare and swap) operation, a key with an additional parameter revision

A second attempt with the same revision 13, will fail

Watching a K/V Store

An unusual functionality of a K/V Store is being able to 'watch' a bucket, or a specific key in that bucket, and receive real-time updates to changes in the store.

For the example above, run nats kv watch my-kv. This will start a watcher on the bucket we have just created earlier. By default, the KV bucket has a history size of one, and so it only remembers the last change. In our case, the watcher should see a delete of the value associated with the key "Key1":

If we now concurrently change the value of 'my-kv' by

The watcher will see that change:

Cleaning up

When you are finished using a bucket, you can delete the bucket, and its resources, by using the rm operator:

Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to establish a connection to a server. Should multiple URLs be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds:

nc, err := nats.Connect("demo.nats.io

Options options = new Options.Builder()
    

const nc = await connect({
    

nc = NATS()
await nc.connect(

NATS client applications use a PING/PONG protocol to check that there is a working connection to the NATS service. Periodically the client will send PING messages to the server, which responds with a PONG. This period is configured by specifying a ping interval on the client connection settings.

The connection will be closed as stale when the client reaches a number of pings which recieved no pong in response, which is configured by specifying the maximum pings outstanding on the client connection settings.

The ping interval and the maximum pings outstanding work together to specify how quickly the client connection will be notified of a problem. This will also help when there is a remote network partition where the operating system does not detect a socket error. Upon connection close, the client will attempt to reconnect. When it knows about other servers, these will be tried next.

In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction.

On connections with significant traffic, the client will often figure out there is a problem between PINGS, and as a result the default ping interval is typically on the order of minutes. To close an unresponsive connection after 100s, set the ping interval to 20s and the maximum pings outstanding to 5:

Synchronous subscriptions require the application to wait for messages. This type of subscription is easy to set-up and use, but requires the application to deal with looping if multiple messages are expected. For situations where a single message is expected, synchronous subscriptions are sometimes easier to manage, depending on the language.

For example, to subscribe to the subject updates and receive a single message you could do:

nc, err := nats.Connect("demo.nats.io

Connection nc = Nats.connect("nats://demo.nats.io:4222"

Managing the interaction with the server is primarily the job of the client library but most of the libraries also provide some insight into what is happening under the covers.

For example, the client library may provide a mechanism to get the connection's current status:

nc, err := nats.Connect("demo.nats.io

Applications can set the maximum reconnect attempts per server. This includes the server provided to the client's connect call, as well as the server the client discovered through another server. Once reconnect to a server fails the specified amount of times in a row, it will be removed from the connect list. After a successful reconnect to a server, the client will reset that server's failed reconnect attempt count. If a server was removed from the connect list, it can be rediscovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to reconnect, it will close the connection and raise an error.

// Set max reconnects attempts
nc, err := nats.

Options options = new Options.Builder()
    

const nc = await connect({
    

When a server goes down, there is a possible anti-pattern called the Thundering Herd where all of the clients try to reconnect immediately, thus creating a denial of service attack. In order to prevent this, most NATS client libraries randomize the servers they attempt to connect to. This setting has no effect if only a single server is used, but in the case of a cluster, randomization, or shuffling, will ensure that no one server bears the brunt of the client reconnect attempts.

However, if you want to disable the randomization process for connect and reconnect, so that servers are always checked in the same order, you can do that in most libraries with a connection option:

servers := []string{"nats://127.0.0.1:1222",

Options options = new Options.Builder()
    

It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, especially when using TLS, libraries provide a wait setting. Generally clients make sure that between two reconnect attempts to the same server at least a certain amount of time has passed. The concrete implementation depends on the library used.

This setting not only prevents wasting client resources, it also alleviates a thundering herd situation when additional servers are not available.

// Set reconnect interval to 10 seconds
nc, err := nats.

Options options = new Options.Builder()
    

const nc = await connect({
    

NATS 2.2 is the largest feature release since version 2.0. The 2.2 release provides highly scalable, highly performant, secure and easy-to-use next generation streaming in the form of JetStream, allows remote access via websockets, has simplified NATS account management, native MQTT support, and further enables NATS toward our goal of securely democratizing streams and services for the hyperconnected world we live in.

Next Generation Streaming

JetStream is the next generation streaming platform for NATS, highly resilient, highly available, and easy to use. We’ve spent a long time listening to our community, learning from our experiences, looking at the needs of today, and thinking deeply about the needs of tomorrow. We built JetStream to address these needs.

Streams with source and mirror configurations are best managed through a client API. If you intend to create such a configuration from command line with NATS CLI you should use a JSON configuration.

Example stream configuration with two sources

Minimal example

With additional options

You can use to monitor in realtime NATS server connections and message statistics.

Prerequisites

Developing with NATS involves a blend of distributed application techniques, common NATS features, and library-specific syntax. Besides this guide, most libraries provide auto-generated API documentation, along with language and platform-specific examples, guides, and other resources.

The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call . The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. (A printable private NKey is referred to as seed). This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server, which never has access to it!

Handling challenge response may require more than just a setting in the connection options, depending on the client library.

When connecting to a cluster, there are a few things to think about.

• Passing a URL for each cluster member (semi-optional)

• The connection algorithm

For performance reasons, most if not all, of the client libraries will buffer outgoing data so that bigger chunks can be written to the network at one time. This may be as simple as a byte buffer that stores a few messages before being pushed to the network.

These buffers do not hold messages forever, generally they are designed to hold messages in high throughput scenarios, while still providing good latency in low throughput situations.

It is the libraries job to make sure messages flow in a high performance manner. But there may be times when an application needs to know that a message has "hit the wire." In this case, applications can use a flush call to tell the library to move data through the system.

The NATS client libraries can take a full URL, nats://demo.nats.io:4222, to specify a specific server host and port to connect to.

Libraries are removing the requirement for an explicit protocol and may allow demo.nats.io:4222 or just demo.nats.io. In the later example the default port 4222 will be used. Check with your specific client library's documentation to see what URL formats are supported.

For example, to connect to the demo server with a URL you can use:

All the client libraries maintained on the will automatically attempt to re-connect if their current server connection gets disconnected for any reason. Upon re-connection the client library will automatically re-establish all the subscriptions, there is nothing for the application programmer to do.

Unless specifically client will try to re-connect to one of the servers it knows about, either through the URLs provided in the connect call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self-heal and reconfigure itself with no additional configuration or intervention.

You can adjust the between connections attempts, the number of reconnection attempts, and adjust the size of the reconnection .

This guide is tailored for existing NATS users upgrading from NATS version v2.10.x. This will read as a summary with links to specific documentation pages to learn more about the feature or improvement.

Features