[release] LoRa Gateway Bridge 2.3.0 / LoRa Server 0.24.0 / LoRa App Server 0.18.0

I’ve just published a new release of all components! The two major changes:

  • The configuration of all components has been refactored
  • LoRa (App) Server now supports streaming frame-logging

LoRa Gateway Bridge

2.3.0

Features:

  • LoRa Gateway Bridge uses a new configuration file format.
    See configuration for more information.

  • Support MQTT client certificate authentication (#74).

Upgrade notes:

When upgrading using the .deb package / using apt or apt-get, your
configuration will be automatically migrated for you. In any other case,
please see configuration.

LoRa Server

0.24.0

Features:

  • LoRa Server uses a new configuration file format.
    See configuration for more information.
  • StreamFrameLogsForGateway API method has been added to stream frames for a given gateway MAC.
  • StreamFrameLogsForDevice API method has been added to stream frames for a given DevEUI.
  • Support MQTT client certificate authentication (#284).

Changes:

  • GetFrameLogsForDevEUI API method has been removed. The frame_log table
    will be removed from the database in the next release!

Upgrade notes:

When upgrading using the .deb package / using apt or apt-get, your
configuration will be automatically migrated for you. In any other case,
please see configuration.

LoRa App Server

0.18.0

Features:

  • LoRa App Server uses a new configuration file format.
    See configuration for more information.
  • Frame-logs for device are now streaming and can be downloaded as JSON file.
    • Note: the /api/devices/{devEUI}/frames (formerly Device.GetFrameLogs)
      endpoint has changed (and the gRPC method has been renamed to Device.StreamFrameLogs).
    • You need LoRa Server 0.24+ in order to use this feature.
  • Added streaming frame-logs for gateways (which also can be downloaded as JSON file).
    • You need LoRa Server 0.24+ in order to use this feature.
  • Support MQTT client certificate authentication (#201).

Upgrade notes:

When upgrading using the .deb package / using apt or apt-get, your
configuration will be automatically migrated for you. In any other case,
please see configuration.

3 Likes

Hello again!

I have generated a new image using loraserver-pi-gen, which caused the new versions of the loraserver applications to be installed.

Then, I updated the configuration files using

$ lora-app-server configfile > lora-app-server.toml
$ loraserver configfile > loraserver.toml
$ lora-gateway-bridge configfile > lora-gateway-bridge.toml

respectively while in the following directories

$HOME/.config/lora-app-server
$HOME/.config/loraserver
$HOME/.config/lora-gateway-bridge

I managed to point the packet_forwarder to localhost sucessfully, and I see PULL_ACKs, which I believe indicate that everything is running.

I can ping my Raspberry Pi, but, when I try to browse to https://192.168.0.174:8080, I get an ERR_CONNECTION_REFUSED.

Strangely, I do not have access (Permission denied) to the dicrectories

/etc/lora-app-server/
/etc/loraserver/
/etc/lora-gateway-bridge

so I haven’t tried to place the configuration files there.

What might be blocking my connection?

I tried to see if there were open connections at port 8080 using lsof -i and I checked that there are many (around 30) entries like the following

lora-app- 786 appserver 56u IPv6 22263 0t0 TCP localhost:45662->localhost:postgresql (CLOSE_WAIT)
lora-app- 786 appserver 57u IPv6 24009 0t0 TCP localhost:45666->localhost:postgresql (CLOSE_WAIT)
lora-app- 786 appserver 64u IPv6 24582 0t0 TCP localhost:45450->localhost:postgresql (CLOSE_WAIT)
loraserve 787 loraserver 3u IPv6 24585 0t0 TCP localhost:45452->localhost:postgresql (CLOSE_WAIT)
loraserve 787 loraserver 5u IPv6 22008 0t0 TCP localhost:45456->localhost:postgresql (CLOSE_WAIT)
loraserve 787 loraserver 6u IPv6 24592 0t0 TCP localhost:45460->localhost:postgresql (CLOSE_WAIT)

What do these mean?

Best regards.


Possibly unrelated, I checked that GW_SERVER_JWT_SECRET, which caused me trouble as seen in https://forum.loraserver.io/t/can-not-register-network-server/407/16, did not appear in loraserver.toml. However, loraserver seems to be running fine.

Another possibly unrelated point: I changed the frequency to US_902_928 at loraserver-pi-gen stage3/04-loraserver/files/loraserver file, but it didn’t carry to loraserver.toml, remaining EU_433.

Please put your config under /etc/lora.... The process is started as a different user and $HOME/.config/lora... is therefore more suitable for testing / development.

You should be able to access the /etc/lora-app-server /etc/loraserver and /etc/lora-gateway-bridge paths when using sudo. In fact, the .deb installer probably already created (dummy) configuration files for you at these locations :slight_smile:

Also, because of a typo, the $HOME/.config/loraserver/loraserver.toml is not loaded, see:

Hi!.
Just redone the config for my test server and I only miss one option. I don’t know if it is intendended or if I missed it.

LOG_NODE_FRAMES.

Any way, it is running perfectly in docker with mounted volumes

EDIT: Just saw it now. The server does not log anymore automatically? I have to open the device to actually log?

EDIT_2: Okay tested. Logs only persist while stream log is active. Is this final? We found permanent logs very useful we have around 500 devices to check daily and seems to be imposible now.

As was mentioned and warning in the UI, the previous implementation of the frame-logs were only for debugging. Please use the provided application integrations (MQTT and / or HTTP) for integrating the device-data with your application.

In case you want to log all frames, you could also subscribe to the LoRa Gateway Bridge topics.

Yes I know it is for debugging. That is how I use them, log during night, check inthe morning. Some of the devices enter in a join loop and it was useful to check DR configuration from ALL of them passing through the web. If now I have to subscribe to mqtt or something more I have to check with my Software Engineer colleages and they usually refuse to do this kind of thing.
I feel like there should be some kind of switch, like before, that i could simply stop all log or activate it.

One of the main problems with this devices is that they send every 6 hours (cannot be modified). That is why I cannot open a stream log to the 500 that we have and that is why we found permanent log usufull, as this is a test for a very large number of devices in the near future.

But It is your decission not mine.

Please note, you’re still able to monitor individual devices and gateways through the LoRa Server gRPC API. With a small modification I could also facilitate subscribing to a pattern if needed. E.g. prefix of a DevEUI or all DevEUIs / gateway MACs on the network.

That LoRa Server is not storing the logs anymore for you, does not mean it is not possible anymore :slight_smile: All it takes to start streaming the gateway-logs is:

https://github.com/brocaar/lora-app-server/blob/master/internal/api/gateway.go#L772

See also: https://github.com/brocaar/loraserver/blob/master/api/ns/ns.proto#L143.

Hi!

I have created a new image using loraserver-pi-gen, which I believe retrieved the new code with the typo fix.

Also, I managed to create the configuration files inside /etc/lora... according to your instructions.

However, I still get ERR_CONNECTION_REFUSED when trying to connect to the web interface.

Moreover, those CLOSE_WAIT messages are still there when I try a lsof -i.

Do you have any ideas on where I can find more information to solve this?

Thanks.

Edit 1:
I killed the loraserver process and tried sudo -u loraserver loraserver, but I have the following error:
ERRO[0000] ping database error, will retry in 2s: pq: password authentication failed for user "loraserver"

Shouldn’t the user loraserver have the correct credentials to run the process?

Actually, it depends on the authentication method and granted privileges for the user at postgres’ configuration, but I wouldn’t recommend having your loraserver user authenticate through the peer method. Instead, assign a password to it, verify (and change if necessary) that your postgres users authenticate via md5 method (check pg_hba.conf), and then rectify your postgres DSN to include correct username and password in order to connect to the DB.

Hi, @iegomez.

Thanks for your reply.

When I checked the pg_hba.conf file, I couldn’t find one!
Postgresql 9.4 had a pg_hba.conf.example, which I copied to pg_hba.conf with no modifications, but there wasn’t much there to make things work.

So, I restored an image file containing the previous version of LoRaServer, which I had managed to get up and running to check pg_hba.conf.
Its relevant parts are:

local   all             postgres                                peer
local   all             all                                     peer
host    all             all             127.0.0.1/32            md5
host    all             all             ::1/128                 md5

Strange thing is that this LoRaServer image was running before, but now, I can’t access its lora-app-server web interface either.

I managed to find the postgresql log file, and every 2 seconds, it receives a new entry like this:

2018-02-16 16:44:38.497 UTC [1755] loraserver_as@loraserver_as FATAL: could not find tuple for opclass 1981

What is opclass 1981?
Why could this work before, but it stopped without any modifications?

Thanks again for your support.


Possibly related, why is postgresql 9.4 being installed in the new images when there was postgresql 9.6 in the Raspbian image generated for the previous LoRaServer version?

Edit 1: I generated another image using loraserver-pi-gen and it came with postgresql 9.6. I wonder what’s going on.

Edit 2: There is a pg_hba.conf now with the same content I provided. However, I still get logs like these:
loraserver@loraserver_ns FATAL: password authentication failed for user "loraserver"
appserver@loraserver_as FATAL: password authentication failed for user "appserver"

I’ll try to change these passwords and will inform the result.

@brocaar, I managed to run lora-app-server and loraserver.

Here are the steps I needed to follow:

In /etc/lora-app-server/lora-app-server.toml

  1. In postgresql.dsn, replace the example from the Quick-Install Guide (https://docs.loraserver.io/install/quick-install/) by dsn="postgres://loraserver_as:dbpassword@localhost/loraserver_as?sslmode=disable". (Change from localhost/loraappserver to loraserver_as).
  2. Provide the path to application_server.external_api.tls_cert and application_server.external_api.tls_key since they are initially empty. My correct parameters are tls_cert="/etc/lora-app-server/certs/http.pem" and tls_key="/etc/lora-app-server/certs/http-key.pem".
  3. Provide a value for application_server.external_api.jwt_secret (notice that in the Quick-Install Guide, there is a typo: application_server.external_api.jwt_serect). E.g., jwt_secret="abc". A better option is to use the tool indicated in the comment just before this parameter in the configuration file.
  4. Then, restart lora-app-server and it should initiate correctly.

In /etc/loraserver/loraserver.toml

  1. In postgresql.dsn, replace the example from the Quick-Install Guide (https://docs.loraserver.io/install/quick-install/) by dsn="postgres://loraserver_ns:dbpassword@localhost/loraserver_ns?sslmode=disable". (Change from localhost/loraserver to loraserver_ns).
  2. As done for lora-app-server, provide a value for network_server.gateway.api.jwt_secret.
  3. Restart loraserver.

Also, to avoid the configuration indicated by @iegomez for now, in /etc/postgresql/9.6/main/pg_hba.conf, I’ve changed the line host all all ::1/128 md5 to host all all ::1/128 trust. I know this isn’t the right way to make it work, but I got the servers running this way for now. I will work on a way to correct it soon.

Thanks.

2 Likes

Thanks for the feedback! I haven’t looked into the Raspberry Pi image generator for a while and will also do some testing / integrate your feedback into it.

1 Like