Basic Station failing to stay conned to bridge

Hi,
I’ve been having a play to see if I can get basics station working with the lora gateway bridge, but have run into some issues.

I’m currently running the station and gateway bridge on a rpi as a test, the station examples/live-s2.sm.tc works and connects and using package forwarder the gateway bridge work with my lora server setup.

But when using station, it exits with a invalid pointer when started like station --home /opt/semtech/basicatation:

station --version
Station: 2.0.3(rpi/std) 2019-08-23 21:06:30
Package: (null)

station log:

2019-09-21 20:23:27.940 [SYS:INFO] Logging     : stderr (maxsize=10000000, rotate=3)
2019-09-21 20:23:27.940 [SYS:INFO] Station Ver : 2.0.3(rpi/std) 2019-08-23 21:06:30
2019-09-21 20:23:27.940 [SYS:INFO] Package Ver : (null)
2019-09-21 20:23:27.940 [SYS:INFO] proto EUI   : 0:b827:eb10:168a       (/sys/class/net/enxb827eb10168a/address)
2019-09-21 20:23:27.941 [SYS:INFO] prefix EUI  : ::1    (builtin)
2019-09-21 20:23:27.941 [SYS:INFO] Station EUI : b827:ebff:fe10:168a
2019-09-21 20:23:27.941 [SYS:INFO] Station home: /opt/semtech/basicatation/     (--home)
2019-09-21 20:23:27.941 [SYS:INFO] Station temp: /var/tmp/      (builtin)
2019-09-21 20:23:27.941 [SYS:WARN] Station in NO-CUPS mode
2019-09-21 20:23:28.142 [TCE:INFO] Starting TC engine
2019-09-21 20:23:28.144 [TCE:INFO] Connecting to INFOS: ws://127.0.0.1:3001
2019-09-21 20:23:28.145 [TCE:INFO] Infos: b827:ebff:fe10:168a b827:ebff:fe10:168a ws://127.0.0.1:3001/gateway/b827ebfffe10168a
2019-09-21 20:23:28.145 [AIO:DEBU] [3] ws_close reason=1000
2019-09-21 20:23:28.145 [AIO:DEBU] Echoing close - reason=1000
2019-09-21 20:23:28.146 [AIO:ERRO] Recv failed: NET - Connection was reset by peer
2019-09-21 20:23:28.146 [AIO:DEBU] [3] WS connection shutdown...
2019-09-21 20:23:28.147 [TCE:VERB] Connecting to MUXS...
2019-09-21 20:23:28.148 [TCE:VERB] Connected to MUXS.
2019-09-21 20:23:28.330 [RAL:INFO] Lora gateway library version: Version: 5.0.1;
2019-09-21 20:23:28.330 [RAL:DEBU] SX1301 txlut table (0 entries)
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 rxrfchain 0: enable=1 freq=915600000 rssi_offset=-166.000000 type=2 tx_enable=1 tx_notch_freq=0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 rxrfchain 1: enable=0 freq=0 rssi_offset=-166.000000 type=2 tx_enable=0 tx_notch_freq=0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  0: enable=1 rf_chain=0 freq=-400000 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  1: enable=1 rf_chain=0 freq=-200000 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  2: enable=1 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  3: enable=0 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  4: enable=0 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  5: enable=0 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  6: enable=0 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  7: enable=0 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  8: enable=0 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 ifchain  9: enable=0 rf_chain=0 freq=0 bandwidth=0 datarate=0 sync_word=0/0
2019-09-21 20:23:28.330 [RAL:VERB] SX1301 LBT not enabled
2019-09-21 20:23:28.330 [RAL:INFO] Station device: /dev/spidev0.0 (PPS capture disabled)
2019-09-21 20:23:31.245 [S2E:INFO] Configuring for region: AU915 -- 915.0MHz..928.0MHz
2019-09-21 20:23:31.245 [S2E:VERB]   DR0  SF12/BW125
2019-09-21 20:23:31.245 [S2E:VERB]   DR1  SF11/BW125
2019-09-21 20:23:31.245 [S2E:VERB]   DR2  SF10/BW125
2019-09-21 20:23:31.245 [S2E:VERB]   DR3  SF9/BW125
2019-09-21 20:23:31.245 [S2E:VERB]   DR4  SF8/BW125
2019-09-21 20:23:31.245 [S2E:VERB]   DR5  SF7/BW125
2019-09-21 20:23:31.245 [S2E:VERB]   DR6  SF8/BW500
2019-09-21 20:23:31.245 [S2E:VERB]   DR7  undefined
2019-09-21 20:23:31.245 [S2E:VERB]   DR8  SF12/BW500 (DN only)
2019-09-21 20:23:31.245 [S2E:VERB]   DR9  SF11/BW500 (DN only)
2019-09-21 20:23:31.245 [S2E:VERB]   DR10 SF10/BW500 (DN only)
2019-09-21 20:23:31.245 [S2E:VERB]   DR11 SF9/BW500 (DN only)
2019-09-21 20:23:31.245 [S2E:VERB]   DR12 SF8/BW500
2019-09-21 20:23:31.245 [S2E:VERB]   DR13 SF7/BW500 (DN only)
2019-09-21 20:23:31.245 [S2E:VERB]   DR14 undefined
2019-09-21 20:23:31.245 [S2E:VERB]   DR15 undefined
2019-09-21 20:23:31.245 [S2E:VERB]   TX power: 14.0 dBm EIRP
2019-09-21 20:23:31.245 [S2E:VERB]   JoinEui list: 10 entries
2019-09-21 20:23:31.245 [S2E:VERB]   NetID filter: 00000001-00000001-00000001-00000001
2019-09-21 20:23:31.245 [S2E:VERB]   Dev/test settings: nocca=0 nodc=0 nodwell=0
2019-09-21 20:24:13.249 [SYN:INFO] MCU/SX1301 drift stats: min: -3.8ppm  q50: -10.5ppm  q80: -12.4ppm  max: -17.1ppm - threshold q90: -13.3ppm
2019-09-21 20:24:13.249 [SYN:INFO] Avg MCU drift vs SX1301#0: 1.0ppm
2019-09-21 20:24:28.149 [AIO:WARN] [3] Ignoring WS ping/pong message
2019-09-21 20:24:32.151 [SYN:INFO] Time sync qualities: min=35 q90=68 max=90 (previous q90=2147483647)
2019-09-21 20:24:33.149 [AIO:DEBU] [3] Connection closed unexpectedly
2019-09-21 20:24:33.149 [AIO:DEBU] [3] WS connection shutdown...
2019-09-21 20:24:33.150 [TCE:VERB] Connection to MUXS closed in state 4
2019-09-21 20:24:33.150 [TCE:INFO] MUXS reconnect backoff 1s (retry 0)
free(): invalid pointer
Aborted

gateway log:

Sep 22 06:23:28 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:23:28+10:00" level=info msg="backend/basicstation: router-info request received" gateway_id=b827ebfffe10168a remote_addr="127.0.0.1:49398" router_uri="ws://127.0.0.1:3001/gateway/b827ebfffe10168a"
Sep 22 06:23:28 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:23:28+10:00" level=info msg="backend/basicstation: gateway connected" gateway_id=b827ebfffe10168a remote_addr="127.0.0.1:49400"
Sep 22 06:23:28 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:23:28+10:00" level=info msg="integration/mqtt: subscribing to topic" qos=0 topic="gateway/b827ebfffe10168a/command/#"
Sep 22 06:23:28 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:23:28+10:00" level=info msg="backend/basicstation: gateway version received" firmware= gateway_id=b827ebfffe10168a model=rpi package= protocol=2 station="2.0.3(rpi/std)"
Sep 22 06:23:28 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:23:28+10:00" level=info msg="integration/mqtt: publishing event" event=stats qos=0 topic=gateway/b827ebfffe10168a/event/stats
Sep 22 06:23:28 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:23:28+10:00" level=info msg="integration/mqtt: gateway configuration received" topic=gateway/b827ebfffe10168a/command/config
Sep 22 06:23:28 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:23:28+10:00" level=info msg="backend/basicstation: router-config message sent to gateway" gateway_id=b827ebfffe10168a
Sep 22 06:24:33 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:24:33+10:00" level=info msg="backend/basicstation: gateway disconnected" gateway_id=b827ebfffe10168a remote_addr="127.0.0.1:49400"
Sep 22 06:24:33 lora-gateway lora-gateway-bridge[623]: time="2019-09-22T06:24:33+10:00" level=info msg="integration/mqtt: unsubscribe topic" topic="gateway/b827ebfffe10168a/command/#"
ls -ahl /opt/semtech/basicatation/
total 20K
drwxr-xr-x 2 root root 4.0K Sep 22 05:37 .
drwxr-xr-x 5 root root 4.0K Sep 22 04:42 ..
-rw-r--r-- 1 root root  958 Sep 22 04:43 station.conf
-rw-r----- 1 root root    0 Sep 22 05:33 tc-bak.done
-rw-r----- 1 root root   20 Sep 22 05:33 tc-bak.uri
-rw-r----- 1 root root   20 Sep 22 05:32 tc.uri

/opt/semtech/basicatation/tc.uri

ws://127.0.0.1:3001

/opt/semtech/basicatation/station.conf

{
    /* If slave-X.conf present this acts as default settings */
    "SX1301_conf": {                    /* Actual channel plan is controlled by server */
        "lorawan_public": true,         /* is default */
        "clksrc": 1,                    /* radio_1 provides clock to concentrator */
        /* path to the SPI device, un-comment if not specified on the command line e.g., RADIODEV=/dev/spidev0.0 */
        "device": "/dev/spidev0.0",
        /* freq/enable provided by LNS - only HW specific settings listed here */
        "radio_0": {
                "type": "SX1257",
                "rssi_offset": -166.0,
                "tx_enable": true,
                "antenna_gain": 0
        },
        "radio_1": {
                "type": "SX1257",
                "rssi_offset": -166.0,
                "tx_enable": false
        }
        /* chan_multiSF_X, chan_Lora_std, chan_FSK provided by LNS */
    },
    "station_conf": {
        "log_file":  "stderr",
        "log_level": "DEBUG",           /* XDEBUG,DEBUG,VERBOSE,INFO,NOTICE,WARNING,ERROR,CRITICAL */
        "log_size":  10000000,
        "log_rotate":  3,
        "CUPS_RESYNC_INTV": "1s"
    }
}

/etc/lora-gateway-bridge/lora-gateway-bridge.toml

[general]
# debug=5, info=4, warning=3, error=2, fatal=1, panic=0
log_level = 4


# Filters.
#
# These can be used to filter LoRaWAN frames to reduce bandwith usage between
# the gateway and LoRa Gateway Bride. Depending the used backend, filtering
# will be performed by the Packet Forwarder or LoRa Gateway Bridge.
[filters]

# NetIDs filters.
#
# The configured NetIDs will be used to filter uplink data frames.
# When left blank, no filtering will be performed on NetIDs.
#
# Example:
# net_ids=[
#   "000000",
#   "000001",
# ]
net_ids=[
]

# JoinEUI filters.
#
# The configured JoinEUI ranges will be used to filter join-requests.
# When left blank, no filtering will be performed on JoinEUIs.
#
# Example:
# join_euis=[
#   ["0000000000000000", "00000000000000ff"],
#   ["000000000000ff00", "000000000000ffff"],
# ]
join_euis=[
]


# Gateway backend configuration.
[backend]

# Backend type.
#
# Valid options are:
#   * semtech_udp
#   * basic_station
type="basic_station"


  # Semtech UDP packet-forwarder backend.
  [backend.semtech_udp]

  # ip:port to bind the UDP listener to
  #
  # Example: 0.0.0.0:1700 to listen on port 1700 for all network interfaces.
  # This is the listeren to which the packet-forwarder forwards its data
  # so make sure the 'serv_port_up' and 'serv_port_down' from your
  # packet-forwarder matches this port.
  udp_bind = "0.0.0.0:1700"

  # Skip the CRC status-check of received packets
  #
  # This is only has effect when the packet-forwarder is configured to forward
  # LoRa frames with CRC errors.
  skip_crc_check = false

  # Fake RX timestamp.
  #
  # Fake the RX time when the gateway does not have GPS, in which case
  # the time would otherwise be unset.
  fake_rx_time=false


  # Basic Station backend.
  [backend.basic_station]

  # ip:port to bind the Websocket listener to.
  bind=":3001"

  # TLS certificate and key files.
  #
  # When set, the websocket listener will use TLS to secure the connections
  # between the gateways and LoRa Gateway Bridge (optional).
  tls_cert=""
  tls_key=""

  # TLS CA certificate.
  #
  # When configured, LoRa Gateway Bridge will validate that the client
  # certificate of the gateway has been signed by this CA certificate.
  ca_cert=""

  # Ping interval.
  ping_interval="1m0s"

  # Read timeout.
  #
  # This interval must be greater than the configured ping interval.
  read_timeout="1m5s"

  # Write timeout.
  write_timeout="1s"

  # Region.
  #
  # Please refer to the LoRaWAN Regional Parameters specification
  # for the complete list of common region names.
  region="AU915"

  # Minimal frequency (Hz).
  frequency_min=915000000

  # Maximum frequency (Hz).
  frequency_max=928000000


# Integration configuration.
[integration]
# Payload marshaler.
#
# This defines how the MQTT payloads are encoded. Valid options are:
# * protobuf:  Protobuf encoding (this will become the LoRa Gateway Bridge v3 default)
# * json:      JSON encoding (easier for debugging, but less compact than 'protobuf')
marshaler="protobuf"

  # MQTT integration configuration.
  [integration.mqtt]
  # Event topic template.
  event_topic_template="gateway/{{ .GatewayID }}/event/{{ .EventType }}"

  # Command topic template.
  command_topic_template="gateway/{{ .GatewayID }}/command/#"

  # Maximum interval that will be waited between reconnection attempts when connection is lost.
  # Valid units are 'ms', 's', 'm', 'h'. Note that these values can be combined, e.g. '24h30m15s'.
  max_reconnect_interval="10m0s"


  # MQTT authentication.
  [integration.mqtt.auth]
  # Type defines the MQTT authentication type to use.
  #
  # Set this to the name of one of the sections below.
  type="generic"

    # Generic MQTT authentication.
    [integration.mqtt.auth.generic]
    # MQTT server (e.g. scheme://host:port where scheme is tcp, ssl or ws)
    server="ssl://mqtt.example.com:8883"

    # Connect with the given username (optional)
    username="user"

    # Connect with the given password (optional)
    password="password"

    # Quality of service level
    #
    # 0: at most once
    # 1: at least once
    # 2: exactly once
    #
    # Note: an increase of this value will decrease the performance.
    # For more information: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels
    qos=0

    # Clean session
    #
    # Set the "clean session" flag in the connect message when this client
    # connects to an MQTT broker. By setting this flag you are indicating
    # that no messages saved by the broker for this client should be delivered.
    clean_session=true

    # Client ID
    #
    # Set the client id to be used by this client when connecting to the MQTT
    # broker. A client id must be no longer than 23 characters. When left blank,
    # a random id will be generated. This requires clean_session=true.
    client_id=""

    # CA certificate file (optional)
    #
    # Use this when setting up a secure connection (when server uses ssl://...)
    # but the certificate used by the server is not trusted by any CA certificate
    # on the server (e.g. when self generated).
    ca_cert=""

    # mqtt TLS certificate file (optional)
    tls_cert=""

    # mqtt TLS key file (optional)
    tls_key=""


    # Google Cloud Platform Cloud IoT Core authentication.
    #
    # Please note that when using this authentication type, the MQTT topics
    # will be automatically set to match the MQTT topics as expected by
    # Cloud IoT Core.
    [integration.mqtt.auth.gcp_cloud_iot_core]
    # MQTT server.
    server="ssl://mqtt.googleapis.com:8883"

    # Google Cloud IoT Core Device id.
    device_id=""

    # Google Cloud project id.
    project_id=""

    # Google Cloud region.
    cloud_region=""

    # Google Cloud IoT registry id.
    registry_id=""

    # JWT token expiration time.
    jwt_expiration="24h0m0s"

    # JWT token key-file.
    #
    # Example command to generate a key-pair:
    #  $ ssh-keygen -t rsa -b 4096 -f private-key.pem
    #  $ openssl rsa -in private-key.pem -pubout -outform PEM -out public-key.pem
    #
    # Then point the setting below to the private-key.pem and associate the
    # public-key.pem with this device / gateway in Google Cloud IoT Core.
    jwt_key_file=""


    # Azure IoT Hub
    #
    # This setting will preset uplink and downlink topics that will only
    # work with Azure IoT Hub service.
    [integration.mqtt.auth.azure_iot_hub]

    # Device connection string (symmetric key authentication).
    #
    # This connection string can be retrieved from the Azure IoT Hub device
    # details when using the symmetric key authentication type.
    device_connection_string=""

    # Token expiration (symmetric key authentication).
    #
    # LoRa Gateway Bridge will generate a SAS token with the given expiration.
    # After the token has expired, it will generate a new one and trigger a
    # re-connect (only for symmetric key authentication).
    sas_token_expiration="24h0m0s"

    # Device ID (X.509 authentication).
    #
    # This will be automatically set when a device connection string is given.
    # It must be set for X.509 authentication.
    device_id=""

    # IoT Hub hostname (X.509 authentication).
    #
    # This will be automatically set when a device connection string is given.
    # It must be set for X.509 authentication.
    # Example: iot-hub-name.azure-devices.net
    hostname=""

    # Client certificates (X.509 authentication).
    #
    # Configure the tls_cert (certificate file) and tls_key (private-key file)
    # when the device is configured with X.509 authentication.
    tls_cert=""
    tls_key=""


# Metrics configuration.
[metrics]

  # Metrics stored in Prometheus.
  #
  # These metrics expose information about the state of the LoRa Gateway Bridge
  # instance like number of messages processed, number of function calls, etc.
  [metrics.prometheus]
  # Expose Prometheus metrics endpoint.
  endpoint_enabled=false

  # The ip:port to bind the Prometheus metrics server to for serving the
  # metrics endpoint.
  bind=""


# Gateway meta-data.
#
# The meta-data will be added to every stats message sent by the LoRa Gateway
# Bridge.
[meta_data]

  # Static.
  #
  # Static key (string) / value (string) meta-data.
  [meta_data.static]
  # Example:
  # serial_number="A1B21234"

  # Dynamic meta-data.
  #
  # Dynamic meta-data is retrieved by executing external commands.
  # This makes it possible to for example execute an external command to
  # read the gateway temperature.
  [meta_data.dynamic]

  # Execution interval of the commands.
  execution_interval="1m0s"

  # Max. execution duration.
  max_execution_duration="1s"

  # Commands to execute.
  #
  # The value of the stdout will be used as the key value (string).
  # In case the command failed, it is ignored. In case the same key is defined
  # both as static and dynamic, the dynamic value has priority (as long as the)
  # command does not fail.
  [meta_data.dynamic.commands]
  # Example:
  # temperature="/opt/gateway-temperature/gateway-temperature.sh"


# Executable commands.
#
# The configured commands can be triggered by sending a message to the
# LoRa Gateway Bridge.
[commands]
  # Example:
  # [commands.commands.reboot]
  # max_execution_duration="1s"
  # command="/usr/bin/reboot"

any help would be appreciated.

I sounds like there are two different issues here:

  • the “Connection closed unexpectedly” which is what you actually want to understand and fix.

  • “free(): invalid pointer” appears to be a bug in Semtech’s code, likely they haven’t well exercised their reconnect loop and have some sort of mistake like trying to free something that was never actually allocated. Of course things like this should be fixed, but it is probably not your immediate problem.

Thanks, @cstratton.
Doning some more internet digging and found https://github.com/lorabasics/basicstation/issues/31 , this seems to be my issue.

Paraphrasing for the lined issue:

The problem is rooted in a bug which surfaces when the JoinEui filter list is parsed. The recommended way is to leave out the JoinEui field in the router_config message. This will disable JoinEui filtering and Station will route all join requests.

Now I just have to work out where to disable JoinEui filtering, so I can get no with testing till they fix the issue.

1 Like

Did you ever figure this out? I am having the same issue too. I can make the JoinEUI list empty, which I think disables the JoinEUI filtering, and removes the invalid pointer message, but still I get the Connection to MUXS closed in state 4 message before basic station closes.

Sorry never found out how to disable JoinEui filtering, so I’m not using basic station for now.

1 Like

That was my resolution too. Thanks @Yendor.