Using a Load Balancer for External Access to NATS
In the example below, you can find how to use an AWS Network Load Balancer to connect externally to a cluster that has TLS setup.
One-line installer creates a secure cluster named 'nats'
curl -sSL https://nats-io.github.io/k8s/setup.sh | sh
Create AWS Network Load Balancer service
echo '
apiVersion: v1
kind: Service
metadata:
name: nats-nlb
namespace: default
labels:
app: nats
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- name: nats
port: 4222
protocol: TCP
targetPort: 4222
selector:
app.kubernetes.io/name: nats
' | kubectl apply -f -
Check that it worked
kubectl get svc nats-nlb -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nats-nlb LoadBalancer 10.100.67.123 a18b60a948fc611eaa7840286c60df32-9e96a2af4b5675ec.elb.us-east-2.amazonaws.com 4222:30297/TCP 151m app=nats
Publish a test message
nats pub -s nats://a18b60a948fc611eaa7840286c60df32-9e96a2af4b5675ec.elb.us-east-2.amazonaws.com:4222 -creds nsc/nkeys/creds/KO/A/test.creds test.foo bar
Also, it would be recommended to set no_advertise to
true in order to avoid gossiping internal addresses from pods in Kubernetes to NATS clients.With the following, you can create a 3-node NATS Server cluster:
kubectl apply -f https://raw.githubusercontent.com/nats-io/k8s/b55687a97a5fd55485e1af302fbdbe43d2d3b968/nats-server/leafnodes/nats-cluster.yaml
The configuration map from the NATS cluster that was created can be found below.
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nats-config
data:
nats.conf: |
pid_file: "/var/run/nats/nats.pid"
http: 8222
# debug: true
ping_interval: 30s
cluster {
port: 6222
no_advertise: true
routes: [
nats://nats-0.nats.default.svc:6222
nats://nats-1.nats.default.svc:6222
nats://nats-2.nats.default.svc:6222
]
}
leaf {
port: 7422
authorization {
timeout: 3s
users = [
{ user: "foo", pass: "bar" }
]
}
}
Now let's expose the NATS Server by creating an L4 load balancer on Azure:
kubectl apply -f https://raw.githubusercontent.com/nats-io/k8s/b55687a97a5fd55485e1af302fbdbe43d2d3b968/nats-server/leafnodes/lb.yaml
Confirm the public IP that was allocated to the
nats-lb service that was created, in this case it is 52.155.49.45:kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 81d <none>
nats ClusterIP None <none> 4222/TCP,6222/TCP,8222/TCP,7777/TCP,7422/TCP,7522/TCP 7h46m app=nats
nats-lb LoadBalancer 10.0.107.18 52.155.49.45 4222:31161/TCP,7422:30960/TCP 7h40m app=nats
Notice that the leafnode configuration requires authorization, so in order to connect to it we will need to configuration as follows:
You can also add a NATS Streaming cluster into the cluster connecting to the port 4222:
kubectl apply -f https://raw.githubusercontent.com/nats-io/k8s/b55687a97a5fd55485e1af302fbdbe43d2d3b968/nats-server/leafnodes/stan-server.yaml
Now if you create two NATS Servers that connect to the same leafnode port, they will be able to receive messages to each other:
nats-server -c leafnodes/leaf.conf -p 4222 &
nats-server -c leafnodes/leaf.conf -p 4223 &
Create a subscriber and publish a test message
nats sub -s localhost:4222 foo &
nats pub -s localhost:4223 foo hello
Listening on [foo]
[#1] Received on [foo] : 'hello'
You may need to set the
advertise value in the gateway scope of your NATS server to the hostname and port of the Load Balancer. See the gateway settings for more information.