Packet Forwarder not sending packets back to node in LoRa Gateway OS


#1

Hey folks,

I’m running the latest version of LoRa-Gateway-OS on a Pi 3 B+ with an IC880A radio board.

The data for a node join currently flows as expected node -> lora_pkt_fwd -> lora-gateway-bridge -> loraserver -> lora-app-server and I see the JoinRequest and JoinAccept in the control panel on the lora app server.

I then see the JoinAccept head back the other way lora-app-server -> loraserver -> lora-gateway-bridge but that’s where it stops. I don’t see any packets being sent from lora_pkt_fwd back to the gateway, the logs from the packet forwarder when I run it manually are as follows:

SON up: {"stat":{"time":"2019-03-26 06:50:45 GMT","rxnb":0,"rxok":0,"rxfw":0,"ackr":100.0,"dwnb":0,"txnb":0}}
INFO: [up] PUSH_ACK received in 1 ms
INFO: [down] PULL_ACK received in 1 ms
INFO: [down] PULL_ACK received in 0 ms
^C
##### 2019-03-26 06:51:03 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 0
# CRC_OK: 0.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
# RF packets forwarded: 0 (0 bytes)
# PUSH_DATA datagrams sent: 1 (113 bytes)
# PUSH_DATA acknowledged: 100.00%
### [DOWNSTREAM] ###
# PULL_DATA sent: 2 (100.00% acknowledged)
# PULL_RESP(onse) datagrams received: 0 (0 bytes)
# RF packets sent to concentrator: 0 (0 bytes)
# TX errors: 0
# BEACON queued: 0
# BEACON sent so far: 0
# BEACON rejected: 0
### [JIT] ###
# SX1301 time (PPS): 542402159
src/jitqueue.c:448:jit_print_queue(): INFO: [jit] queue is empty
### [GPS] ###
# GPS sync is disabled
##### END #####

With this being LoRa Gateway OS, I’m unable to use TCPDump without building my own image, so the steps recorded at https://www.loraserver.io/guides/troubleshooting/gateway/ aren’t an option for me.

The gateway-bridge logs for the same time as the above packet forwarder logs show the following:

INFO[0395] gateway: rxpk packet received                 addr="127.0.0.1:54426" data="AAAAAAAAAAAA6zwiwpGkCwByPK2yoHs=" mac=b827ebfffeedc724
INFO[0395] backend: publishing packet                    qos=0 topic=gateway/b827ebfffeedc724/rx
DEBU[0395] gateway: sending udp packet to gateway        addr="127.0.0.1:54426" protocol_version=2 type=PushACK
DEBU[0400] gateway: received udp packet from gateway     addr="127.0.0.1:54426" protocol_version=2 type=PushData
INFO[0400] gateway: stat packet received                 addr="127.0.0.1:54426" mac=b827ebfffeedc724
DEBU[0400] gateway: sending udp packet to gateway        addr="127.0.0.1:54426" protocol_version=2 type=PushACK
INFO[0400] backend: publishing packet                    qos=0 topic=gateway/b827ebfffeedc724/stats
DEBU[0404] gateway: received udp packet from gateway     addr="127.0.0.1:47169" protocol_version=2 type=PullData
DEBU[0404] gateway: sending udp packet to gateway        addr="127.0.0.1:47169" protocol_version=2 type=PullACK
DEBU[0414] gateway: received udp packet from gateway     addr="127.0.0.1:47169" protocol_version=2 type=PullData
DEBU[0414] gateway: sending udp packet to gateway        addr="127.0.0.1:47169" protocol_version=2 type=PullACK
DEBU[0425] gateway: received udp packet from gateway     addr="127.0.0.1:47169" protocol_version=2 type=PullData
DEBU[0425] gateway: sending udp packet to gateway        addr="127.0.0.1:47169" protocol_version=2 type=PullACK
DEBU[0430] gateway: received udp packet from gateway     addr="127.0.0.1:54426" protocol_version=2 type=PushData
INFO[0430] gateway: stat packet received                 addr="127.0.0.1:54426" mac=b827ebfffeedc724
DEBU[0430] gateway: sending udp packet to gateway        addr="127.0.0.1:54426" protocol_version=2 type=PushACK
INFO[0430] backend: publishing packet                    qos=0 topic=gateway/b827ebfffeedc724/stats
DEBU[0435] gateway: received udp packet from gateway     addr="127.0.0.1:47169" protocol_version=2 type=PullData
DEBU[0435] gateway: sending udp packet to gateway        addr="127.0.0.1:47169" protocol_version=2 type=PullACK
DEBU[0445] gateway: received udp packet from gateway     addr="127.0.0.1:47169" protocol_version=2 type=PullData
DEBU[0445] gateway: sending udp packet to gateway        addr="127.0.0.1:47169" protocol_version=2 type=PullACK

I’m reasonably sure this is a configuration issue somewhere, so here’s the relevant config files:

[general]
# debug=5, info=4, warning=3, error=2, fatal=1, panic=0
log_level = 5


# Configuration which relates to the packet-forwarder.
[packet_forwarder]
# ip:port to bind the UDP listener to
#
# Example: 0.0.0.0:1700 to listen on port 1700 for all network interfaces.
# This is the listeren to which the packet-forwarder forwards its data
# so make sure the 'serv_port_up' and 'serv_port_down' from your
# packet-forwarder matches this port.
udp_bind = "0.0.0.0:1700"

# Skip the CRC status-check of received packets
#
# This is only has effect when the packet-forwarder is configured to forward
# LoRa frames with CRC errors.
skip_crc_check = false


  # # Managed packet-forwarder configuration.
  # #
  # # By configuring one or multiple managed packet-forwarder sections, the
  # # LoRa Gateway Bridge updates the configuration when the backend receives
  # # a configuration change, after which it will restart the packet-forwarder.
  # [[packet_forwarder.configuration]]
  # # Gateway MAC.
  # #
  # # The LoRa Gateway Bridge will only apply the configuration updates for this
  # # gateway MAC.
  # mac="0102030405060708"

  # # Base configuration file.
  # #
  # # This file will be used as base-configuration and will not be overwritten on
  # # a configuration update. This file needs to exist and contains the base
  # # configuration and vendor specific
  # base_file="/etc/lora-packet-forwarder/global_conf.json"

  # # Output configuration file.
  # #
  # # This will be the final configuration for the packet-forwarder, containing
  # # a merged version of the base configuration + the requested configuration
  # # update.
  # # Warning: this file will be overwritten on a configuration update!
  # output_file="/etc/lora-packet-forwarder/local_conf.json"

  # # Restart command.
  # #
  # # This command is issued by the LoRa Gateway Bridge on a configuration
  # # change. Make sure the LoRa Gateway Bridge process has sufficient
  # # permissions to execute this command.
  # restart_command="/etc/init.d/lora-packet-forwarder restart"


# Configuration for the MQTT backend.
[backend.mqtt]
# MQTT topic templates for the different MQTT topics.
#
# The meaning of these topics are documented at:
# https://docs.loraserver.io/lora-gateway-bridge/use/data/
#
# The default values match the default expected configuration of the
# LoRa Server MQTT backend. Therefore only change these values when
# absolutely needed.
# Use "{{ .MAC }}" as an substitution for the LoRa gateway MAC.
uplink_topic_template="gateway/{{ .MAC }}/rx"
downlink_topic_template="gateway/{{ .MAC }}/tx"
stats_topic_template="gateway/{{ .MAC }}/stats"
ack_topic_template="gateway/{{ .MAC }}/ack"
config_topic_template="gateway/{{ .MAC }}/config"

# Payload marshaler.
#
# This defines how the MQTT payloads are encoded. Valid options are:
# * v2_json:   The default LoRa Gateway Bridge v2 encoding (will be deprecated and removed in LoRa Gateway Bridge v3)
# * protobuf:  Protobuf encoding (this will become the LoRa Gateway Bridge v3 default)
# * json:      JSON encoding (easier for debugging, but less compact than 'protobuf')
marshaler="v2_json"

  # MQTT authentication.
  [backend.mqtt.auth]
  # Type defines the MQTT authentication type to use.
  #
  # Set this to the name of one of the sections below.
  # Note: when the 'v2_json marhaler' is configured, the generic backend will
  # always be used.
  type="generic"

    # Generic MQTT authentication.
    [backend.mqtt.auth.generic]
    # MQTT server (e.g. scheme://host:port where scheme is tcp, ssl or ws)
    server="tcp://<MY MQTT SERVER>:1883"

    # Connect with the given username (optional)
    username="<MY USERNAME>"

    # Connect with the given password (optional)
    password="<MY PASSWORD>"

    # Quality of service level
    #
    # 0: at most once
    # 1: at least once
    # 2: exactly once
    #
    # Note: an increase of this value will decrease the performance.
    # For more information: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels
    qos=0

    # Clean session
    #
    # Set the "clean session" flag in the connect message when this client
    # connects to an MQTT broker. By setting this flag you are indicating
    # that no messages saved by the broker for this client should be delivered.
    clean_session=true

    # Client ID
    #
    # Set the client id to be used by this client when connecting to the MQTT
    # broker. A client id must be no longer than 23 characters. When left blank,
    # a random id will be generated. This requires clean_session=true.
    client_id="ilsgwos1"

    # CA certificate file (optional)
    #
    # Use this when setting up a secure connection (when server uses ssl://...)
    # but the certificate used by the server is not trusted by any CA certificate
    # on the server (e.g. when self generated).
    ca_cert=""

    # mqtt TLS certificate file (optional)
    tls_cert=""

    # mqtt TLS key file (optional)
    tls_key=""

    # Maximum interval that will be waited between reconnection attempts when connection is lost.
    # Valid units are 'ms', 's', 'm', 'h'. Note that these values can be combined, e.g. '24h30m15s'.
    max_reconnect_interval="10m0s"


# Metrics configuration.
[metrics]

  # Metrics stored in Prometheus.
  #
  # These metrics expose information about the state of the LoRa Gateway Bridge
  # instance like number of messages processed, number of function calls, etc.
  [metrics.prometheus]
  # Expose Prometheus metrics endpoint.
  endpoint_enabled=true

  # The ip:port to bind the Prometheus metrics server to for serving the
  # metrics endpoint.
  bind=""
{
    "SX1301_conf": {
        "lorawan_public": true,
        "clksrc": 1,
        "antenna_gain": 0,
        "radio_0": {
            "enable": true,
            "type": "SX1257",
            "freq": 867500000,
            "rssi_offset": -166.0,
            "tx_enable": true,
            "tx_freq_min": 863000000,
            "tx_freq_max": 870000000
        },
        "radio_1": {
            "enable": true,
            "type": "SX1257",
            "freq": 868500000,
            "rssi_offset": -166.0,
            "tx_enable": false
        },
        "chan_multiSF_0": {
            "enable": true,
            "radio": 1,
            "if": -400000
        },
        "chan_multiSF_1": {
            "enable": true,
            "radio": 1,
            "if": -200000
        },
        "chan_multiSF_2": {
            "enable": true,
            "radio": 1,
            "if": 0
        },
        "chan_multiSF_3": {
            "enable": true,
            "radio": 0,
            "if": -400000
        },
        "chan_multiSF_4": {
            "enable": true,
            "radio": 0,
            "if": -200000
        },
        "chan_multiSF_5": {
            "enable": true,
            "radio": 0,
            "if": 0
        },
        "chan_multiSF_6": {
            "enable": true,
            "radio": 0,
            "if": 200000
        },
        "chan_multiSF_7": {
            "enable": true,
            "radio": 0,
            "if": 400000
        },
        "chan_Lora_std": {
            "enable": true,
            "radio": 1,
            "if": -200000,
            "bandwidth": 250000,
            "spread_factor": 7
        },
        "chan_FSK": {
            "enable": true,
            "radio": 1,
            "if": 300000,
            "bandwidth": 125000,
            "datarate": 50000
        },
        "tx_lut_0": {
            "pa_gain": 0,
            "mix_gain": 8,
            "rf_power": -6,
            "dig_gain": 0
        },
        "tx_lut_1": {
            "pa_gain": 0,
            "mix_gain": 10,
            "rf_power": -3,
            "dig_gain": 0
        },
        "tx_lut_2": {
            "pa_gain": 0,
            "mix_gain": 12,
            "rf_power": 0,
            "dig_gain": 0
        },
        "tx_lut_3": {
            "pa_gain": 1,
            "mix_gain": 8,
            "rf_power": 3,
            "dig_gain": 0
        },
        "tx_lut_4": {
            "pa_gain": 1,
            "mix_gain": 10,
            "rf_power": 6,
            "dig_gain": 0
        },
        "tx_lut_5": {
            "pa_gain": 1,
            "mix_gain": 12,
            "rf_power": 10,
            "dig_gain": 0
        },
        "tx_lut_6": {
            "pa_gain": 1,
            "mix_gain": 13,
            "rf_power": 11,
            "dig_gain": 0
        },
        "tx_lut_7": {
            "pa_gain": 2,
            "mix_gain": 9,
            "rf_power": 12,
            "dig_gain": 0
        },
        "tx_lut_8": {
            "pa_gain": 1,
            "mix_gain": 15,
            "rf_power": 13,
            "dig_gain": 0
        },
        "tx_lut_9": {
            "pa_gain": 2,
            "mix_gain": 10,
            "rf_power": 14,
            "dig_gain": 0
        },
        "tx_lut_10": {
            "pa_gain": 2,
            "mix_gain": 11,
            "rf_power": 16,
            "dig_gain": 0
        },
        "tx_lut_11": {
            "pa_gain": 3,
            "mix_gain": 9,
            "rf_power": 20,
            "dig_gain": 0
        }
    },

    "gateway_conf": {
        "gateway_ID": "b827ebFFFEedc724",
        "server_address": "localhost",
        "serv_port_up": 1700,
        "serv_port_down": 1700,
        "keepalive_interval": 10,
        "stat_interval": 30,
        "push_timeout_ms": 100,
        "forward_crc_valid": true,
        "forward_crc_error": false,
        "forward_crc_disabled": false
    }
}

If anyone can shed some light on this (or even just tell me how to install tcpdump on the gateway!) that would be very much appreciated.


#2

This is now fixed.

The configuration on the application server had gateway/{{ .MAC }}tx and not gateway/{{ .MAC }}/tx (note the missing /)