LoRa App Server InfluxDB integration (with Grafana)


#1

A new feature that will be coming to LoRa App Server is InfluxDB integration :slight_smile: (https://www.influxdata.com). This makes it possible to let LoRa App Server write the sensor data directly into InfluxDB so that it can be graphed with Grafana (https://grafana.com).

Note: don’t look at the scale of this sensor data, it might be completely wrong :slight_smile:

When you have configured the Cayenne LPP codec or have implemented a custom JavaScript decoder, it will automatically write these fields as measurements into InfluxDB. Nested objects e.g.

{
    "object": {
        "foo": {
            "bar": 123.456
        }
    }
}

will be written as device_frmpayload_data_foo_bar with value 123.456. It will also store the application and device-name, devEUI and fPort as tags so that data can be aggregated / filtered per application, device and fPort.

Next to the sensor data from the frmPayload, it will also write some uplink data so that you can aggregate on the channel, data-rate etc… usage as you can see in the above screenshot.

Note that this still in development. However I’m looking forward to you feedback!


[release] LoRa App Server 0.21.0
#2

HI @brocaar!!!

These integrations is the because parts!

Yessss!! :rocket:


#3

GREAT! Very nice feature!
Is it possible to be a beta tester?


#4

That would be great. I’ll share a link in this topic with test builds when I have the configuration through the UI part ready.


#5

if you need someone to help testing - let me know.
:slight_smile:


#6

@RogerioCassares @Silvano_Pisoni @vincems / whoever is interested:

You can find a precompiled test release here: https://www.dropbox.com/sh/t9x6ehgpskfcmb5/AACKINP5WUAHjhwggBWGaD4fa?dl=0

(you will find the source here https://github.com/brocaar/lora-app-server/tree/influx_integration)

Setting up an InfluxDB integration works the same as setting up a HTTP integration. Only you have to select the “InfluxDB integration” :slight_smile: Looking forward to your feedback!

An additional feature that I have added is that when LoRa App Server finds two fields (within the same object) named latitude and longitude, it will combine these as a single InfluxDB measurement with values latitude=...,longitude=...,geohash="...". This should make it easier to use geo plugins for Grafana :slight_smile:


#7

Hi, Orne. This looks cool, but I have one question (consider I’ve never used Grafana): why is InfluxDB used instead of directly using the same Postgres instance from lora-app-server? Does it have better integration with Grafana?


#8

Good question :slight_smile: InfluxDB is a time series database which is great for sensor measurements + it really works well with Grafana (although lately Grafana also added PostgreSQL support). Some helpful features are setting retention policies in InfluxDB and continuous queries.

With the first you define how long the data should be persisted (e.g. one day, week, month…) and using a continuous query you can downsample the data. E.g. when you device sends data every 10 minutes, you could store this raw data for one day, and then downsample this to daily, weekly, monthly aggregates for example.

An other thing why I think InfluxDB is easier for these kind of things is that you don’t have to pre-define the database schema. Where you have to create the correct indices with PostgreSQL, is is all done automatically for you with InfluxDB. Each measurement consists of a measurement name, tags, values and a timestamp. InfluxDB will make sure that tags and timestamps are indexed for you.

Also note that the integration works over HTTP, meaning that when LoRa App Server is offered as a hosted solution, this means that each user can setup his / her own (self hosted) InfluxDB database with retention policies suitable for the linked application(s).


#9

Installed in 5 minutes and works perfectly.
Great Brocaar


#10

Hi! Do I understand correctly that this only works when there is a custom payload codec (or the built in codec) present? And it is limited to payload data?
What about gateway data (like CRC error rates, etc.)?


#11

Hi! Do I understand correctly that this only works when there is a custom payload codec (or the built in codec) present? And it is limited to payload data?

Yes, you need to decode the payload as LoRa App Server needs to know what to store and under which measurement names.

What about gateway data (like CRC error rates, etc.)?

One step at a time :wink: But yes, that could definitely be something for the future.


#12

This has been released as LoRa App Server 0.21.0 :slight_smile:

If you have a PyCom device with PySense or PyTracker shield, you could use https://github.com/brocaar/pycom-examples as these examples work out of the box with CayenneLPP (and thus can be written directly into InfluxDB).


#13

When you upgrade to LoRa App Server 0.21.0, please note that there was a small change in the naming of measurements. E.g. _temperaturesensor_ (derived from the JSON temperatureSensor) is now named _temperature_sensor_). See also https://www.loraserver.io/lora-app-server/integrate/sending-receiving/influxdb/.


#14

Absolutely amazing feature to have. One middle app for saving to InfluxDB less for me =) Thank you!