Enabling System Events with Decentralized Authentication/Authorization

To enable and access system events, you'll have to:

• Create an Operator, Account and User

• Run a NATS Account Server (or Memory Resolver)

Create an Operator, Account, User

Let's create an operator, system account and system account user:

Add the system account

Add a system account user

By default, the operator JWT can be found in ~/.nsc/nats/<operator_name>/<operator.name>.jwt.

NATS-Account-Server

To vend the credentials to the nats-server, we'll use a . Let's start a nats-account-server to serve the JWT credentials:

The server will by default vend JWT configurations on the an endpoint at: http(s)://<server_url>/jwt/v1/accounts/.

NATS Server Configuration

The server configuration will need:

• The operator JWT - (~/.nsc/nats/<operator_name>/<operator.name>.jwt)

• The URL where the server can resolve accounts (http://localhost:9090/jwt/v1/accounts/)

• The public key of the system_account

The only thing we don't have handy is the public key for the system account. We can get it easy enough:

Because the server has additional resolver implementations, you need to enclose the server url like: URL(<url>).

Let's create server config with the following contents and save it to server.conf:

Let's start the nats-server:

Inspecting Server Events

Let's add a subscriber for all the events published by the system account:

Very quickly we'll start seeing messages from the server as they are published by the NATS server. As should be expected, the messages are just JSON, so they can easily be inspected even if just using a simple nats sub to read them.

To see an account update:

The subscriber will print the connect and disconnect:

User Services

$SYS.REQ.USER.INFO - Request Connected User Information

For the active connection, get basic user information including the account name, permissions, and expiry, if applicable. Note, this works with any connected user, not just a system account user.

System Services

$SYS.REQ.SERVER.PING.IDZ - Discovering Servers

To discover servers in the cluster to get their ID and name, publish a request to $SYS.REQ.SERVER.PING.IDZ.

$SYS.REQ.SERVER.PING - Discovering Servers + Stats

To discover servers in the cluster, and get a small health summary, publish a request to $SYS.REQ.SERVER.PING. Note that while the example below uses nats-req, only the first answer for the request will be printed. You can easily modify the example to wait until no additional responses are received for a specific amount of time, thus allowing for all responses to be collected.

$SYS.REQ.SERVER.<id>.STATSZ - Requesting Server Stats Summary

If you know the server id for a particular server (such as from a response to $SYS.REQ.SERVER.PING), you can query the specific server for its health information:

$SYS.REQ.SERVER.<id>.PROFILEZ - Request Profiling Information

If profiling is enabled for a server, this service enables requesting it from the server. The request payload must specify the name of the profile being requested with an optional debug level, including:

• allocs - 0, 1

• block - 0

• goroutine - 0, 1, 2

$SYS.REQ.SERVER.<id>.RELOAD - Hot Reload Configuration

Sending a request to this service will attempt to hot reload the server configuration, akin to nats-server --signal reload. If there are errors with the new configuration, they will be returned in an error field in the response.

NATS servers leverage Accounts support and generate events such as:

• account connect/disconnect

• authentication errors

• server shutdown

• server stat summary

In addition the server supports a limited number of requests that can be used to query for account connections, server stat summaries, and pinging servers in the cluster.

These events are enabled by configuring system_account and using a system account user.

are used so that subscriptions from your applications, say >, do not receive system events and vice versa. Using accounts requires either:

• and listing one of the accounts in system_account

• Or by using decentralized authentication and authorization via as shown in this . In this case system_account contains the account public key.

N.B. The default global account $G does not publish advisories.

Available Events and Services

System Account

The system account publishes messages under well known subject patterns.

Server initiated events:

• $SYS.ACCOUNT.<id>.CONNECT (client connects)

• $SYS.ACCOUNT.<id>.DISCONNECT (client disconnects)

• $SYS.ACCOUNT.<id>.SERVER.CONNS (connections for an account changed)

In addition other tools with system account privileges, can initiate requests (Examples can be found ):

• $SYS.REQ.SERVER.<id>.STATSZ (request server stat summary)

• $SYS.REQ.SERVER.PING (discover servers - will return multiple messages)

as listed in the table below are accessible as system services using the following subject pattern:

• $SYS.REQ.SERVER.<id>.<endpoint-name> (request server monitoring endpoint corresponding to endpoint name.)

• $SYS.REQ.SERVER.PING.<endpoint-name> (from all server, request server monitoring endpoint corresponding to endpoint name - will return multiple messages)

• "$SYS.REQ.ACCOUNT.<account-id>.<endpoint-name>(from all server, request account specific monitoring endpoint corresponding to account id and endpoint name - will return multiple messages)

Servers like nats-account-server publish system account messages when a claim is updated, the nats-server listens for them, and updates its account information accordingly:

• $SYS.ACCOUNT.<id>.CLAIMS.UPDATE

With these few messages you can build useful monitoring tools:

• health/load of your servers

• client connects/disconnects

• account connections

• authentication errors

Local Configuration

To make use of System events, just using accounts, your configuration can look like this:

Please note that applications now have to authenticate such that a connection can be associated with an account. In this example username and password were chosen for simplicity of the demonstration. Subscribe to all system events like this nats sub -s nats://admin:changeit@localhost:4222 ">" and observe what happens when you do something like nats pub -s "nats://a:a@localhost:4222" foo bar. Examples on how to use system services can be found .