High Availability


#1

How can I build a High Availability architecture?

Thanks


How LoRaServer ensure high performance, scalability and high availability
Clustering Network Server
How to run a cluster of multiple LoRa Server instances?
How to run a cluster of multiple LoRa Server instances?
#2

You can run a cluster of multiple LoRa Server and LoRa App Server instances. This should work without any issues. You only need to put a load-balancer in between the APIs.

Also you can look into setting up a cluster for:

  • PostgreSQL
  • Redis
  • MQTT broker

#3

Can you expand on this a bit? What would a clustered configuration look like? What is the “clustering” method, other than just separate instances behind a load balancer? I assume the instances would share SQL and Redis databases (themselves being clustered)? How should gateway connections be “allocated” across the LoRa Server instances? Does gateway management and deduplication just work “magically” in such a configuration, or are there other factors to consider?


#4

Gateway connection to LoRa Server (MQTT broker)

The easiest is to run the LoRa Gateway Bridge on each gateway. In that case the HA is handled by the MQTT broker (LoRa Gateway Bridge <> MQTT Broker <> LoRa Server). You have to refer to the MQTT broker documentation as each MQTT broker handles clustering in a different way. See for example: https://stackoverflow.com/questions/26280208/cluster-forming-with-mosquitto-broker

LoRa Server connection to LoRa App Server (and back)

gRPC is a HTTP based RPC framework, so to make both LoRa Server and LoRa App Server HA, you can run multiple instances of each service and put them behind a load balancer. Both LoRa Server and LoRa App Server are designed so that you can run multiple instances, so this will not break de-duplication etc.

PostgreSQL

Please see: https://www.postgresql.org/docs/9.6/static/high-availability.html

Redis

Please see: https://redis.io/topics/sentinel


#5

It appears LoraServer uses the ‘garyburd/redigo’ Redis client? I wasn’t able to find any information indicating that this client supports Sentinel. Has anyone actually setup high availability for Redis (with automatic failover). If so, how did you do it?

Thanks


#6

Haven’t tried it myself, but redigo mentions this repo, which states is a

Redis Sentinel support for redigo library.


#7

With the lora app server behind the load balancer, what would be “as-public-server” for each one of them. Do we use the server’s hostname or load-balancers?


#8

The hostname of the loadbalancer as that is how you want LoRa Server to connect to LoRa App Server.


#9

Lets assume we have 2 NS running and talking to same App Server.and some of the devices activated with NS1 and others with NS2:

  1. What happens to messages from devices which were activated against NS1, if NS1 goes down.
  2. Since communcation from Gateway to NS is through MQTT topics are both these servers capable of handling the messages?

#10

Please note that none of the NS instances have a local state. All states are stored in either Redis or PostgreSQL.


#11

So is it correct to say that all my network servers will behave as home network servers for all the devices? Is it that each NS in the cluster is exact replica of the other and all connected to same set of MQTT topics?


#12

Yes that is correct.


#13

Hi brocaar,

i’m trying to put HA on LoRa, the port that network server have in commons with application server are: 1883, 8001, 8003. Are these ports that mus be set with load balancer? Thanks


#14

Hi, brocaar
does this means these loraserver instances have the same DSN to share Devices and use the same band region?


#15

I want to make the lora-server highly available.

I have read this topic about High Availability:
high%20availability

I don’t know the detail steps to achieve it. Has anyone ever done this ?Could you share me some resources like a guide book ?

Thanks !


#16

This isn’t directly related to loraserver but more of a common problem. There’s plenty of information out there, and what’s useful will depend on your particular needs. But if it’s only a quick start to load balancing you need, you could check this out on how to achieve it easily using Nginx as reverse proxy and balancer.

Please use this as a way of getting started and do not follow it blindly. This is just a basic example on this topic, and you should do your research to implement this correctly. Consider also that this only deals with load balancing, and running clusters of Postgres and Redis is a whole other thing. Be sure to check the links Orne posted, and for Postgres give this a look too.


#17

Thank you very very much for your answer and your tips !