Help testing the next LoRa (App) Server release with LoRaWAN 1.1 support

I’ve also got the same problem here

Upon making a POST call to /api/devices/{devEUI}/queue with pretty much anything such as

{
"Anything":"BLABLABLA"
}

the result is the same output as stated by Alexandre.

{
      "devEUI": "600a8c29b2ce8277",
      "reference": "",
      "confirmed": false,
      "fPort": 0,
      "data": null,
      "jsonObject": ""
} 
3 Likes

Thanks @Alexandre_Pelagaggi and @Felipe this is really helpful feedback :slight_smile: I’ll make sure this gets fixed (probably today) and will end up in the next testing release (probably next week). If you would need a new .deb between the fix and the next testing release, please let me know and I can create a snapshot for you.

Update: I have checked. The Skip frame-counter field was removed from the activation API as this was moved to the device object. In 1.x this was still at two places to be backwards compatible. In v2 this will be available only on configuration of the device.

With regards to your enqueue issue, please see the /api interface for examples (and maybe make sure your cache is cleared?). Note that between the latest testing version I have made some additional changes: https://github.com/brocaar/lora-app-server/commit/effa41b016169cd89f5a1fcddefb99dc52d412ca. This means that you’ll now get the fCnt back (which maps to the fCnt in case of an acknowledgement) and the reference has been removed.

This works as expected:

(I do see that this doesn’t expose the fCnt, I’ll add this).

2 Likes

A preview of what will probably be in the next testing release (this is still wip):

This doesn’t add new features to the web-interface, but it should make the whole UI easier to use as the main components are now located in the sidebar :slight_smile:

(@iegomez this is what I was referring to in my DM to you)

2 Likes

That looks pretty nice!

On a side note, yesterday we found a bug in the UI where when entering as an admin you’d see the gateways link, but after pressing some other tab it would disappear and you’d had to refresh to get the gateways link back. I didn’t have time to fix it and knew you were preparing a new UI, but thought I’d mention it just in case it remained.

Yes, I noticed the same (I think the state update isn’t handled correctly when the organization is loaded). However I’m planning to merge the new UI into master hopefully somewhere next week :slight_smile:

I’ve just released a new testing release of LoRa App Server (v2.0.0-alpha.2). This includes the new UI. I hope you all like it :slight_smile: Happy testing and please share your feedback!

Please see the above Debian / Ubuntu and Docker (Compose) instructions.

1 Like

Hi, I’ve merged the changes and am testing everything right now. I still need to reimplement our modifications, but so far I’ve encountered a couple of little bugs:

Gateway details fails on getting stats because of nil timestamps. Just changing the timestamps fields names from camel to snake case fixed it:

start_timestamp: start,
end_timestamp: end,

Surely it’s something related to how they are defined at the proto, but i’m just getting familiar with new protobuf notations, new ways of calling the swagger client instead of fetching from the uris, etc.

After deleting an organization, it is fetched when returning to the organizations list, resulting in an “object does not exist” warning. I see that getPage is passed as a prop to DataTable, but again, being new to Material UI, I prefer just to let you know.

One suggestion: maybe service profile list and edit views could mention the organization id (which is already fetched), or better, the organization name (though it involves adding the message field and having to get the organization when getting the service profile). I noticed it because after you explained what’s the service profile “role” I created default ones programatically when users register, all named the same. Of course one could be careful with the name selection, but it wouldn’t hurt to be able to link the SP with the organziation visually.

I’ll keep digging during the day as I add our implementations and let you know of any other detail that I may find, but it’s looking really promising and we like the feeling of the new UI very much. Great work!

1 Like

Thanks @iegomez

Gateway details fails on getting stats because of nil timestamps. Just changing the timestamps fields names from camel to snake case fixed it:

That is odd, when testing using the v2.0.0-alpha.2 tag (git checkout v2.0.0-alpha.2, which maps to commit b127a33ccd92fc44364cc92083de127d34ec636e) I don’t have any issues with getting the gateway stats. You mentioned you have merged your changes. Could it be that your version is not exactly the same as the v2.0.0-alpha.2 tag?

After deleting an organization, it is fetched when returning to the organizations list, resulting in an “object does not exist” warning. I see that getPage is passed as a prop to DataTable, but again, being new to Material UI, I prefer just to let you know.

Will look into this. Thanks!

One suggestion: maybe service profile list and edit views could mention the organization id (which is already fetched), or better, the organization name

Note that this is already visible. The selected organization (to which the service-provide, device-profile, …) is shown in the sidebar:

Or do I misunderstand you?

Thanks for all the feedback :+1:

Update: I already know what you mean with the start_ and stop_timestamp. I made a local change to the Swagger API generator which is called when calling make api. See also:

diff --git a/protoc-gen-swagger/genswagger/template.go b/protoc-gen-swagger/genswagger/template.go
index c4f2b8d..0d206b7 100644
--- a/protoc-gen-swagger/genswagger/template.go
+++ b/protoc-gen-swagger/genswagger/template.go
@@ -107,7 +107,7 @@ func queryParams(message *descriptor.Message, field *descriptor.Field, prefix st
                        desc = strings.TrimSpace(schema.Title + ". " + schema.Description)
                }
                param := swaggerParameterObject{
-                       Name:        prefix + field.GetName(),
+                       Name:        prefix + field.GetJsonName(),
                        Description: desc,
                        In:          "query",
                        Type:        schema.Type,
@@ -143,7 +143,7 @@ func queryParams(message *descriptor.Message, field *descriptor.Field, prefix st
                return nil, fmt.Errorf("unknown message type %s", fieldType)
        }
        for _, nestedField := range msg.Fields {
-               p, err := queryParams(msg, nestedField, prefix+field.GetName()+".", reg, pathParams)
+               p, err := queryParams(msg, nestedField, prefix+field.GetJsonName()+".", reg, pathParams)
                if err != nil {
                        return nil, err
                }
@@ -251,7 +251,7 @@ func renderMessagesAsDefinition(messages messageMap, d swaggerDefinitionsObject,
                                panic(err)
                        }

-                       schema.Properties = append(schema.Properties, keyVal{f.GetName(), fieldValue})
+                       schema.Properties = append(schema.Properties, keyVal{f.GetJsonName(), fieldValue})
                }
                d[fullyQualifiedNameToSwaggerName(msg.FQMN(), reg)] = schema
        }

I assume you re-generated the API code which was causing a mismatch of names because of the above change I made. What I did was kind of a local quickfix to work around this issue, then I forgot I made it as the swagger code is comitted in the repo, so the build server doesn’t complain :wink: I will make a fork of the code generator and will add my fork as a requirement so when you run make requirements and then make api you will end up with the same result.

Yeah, I regenerated the API with make api after merging, so that was it.

As for the service profiles, I mean when I enter as an admin, I may fully list all service profiles:

There are 2 profiles belonging to 2 different organizations with id 1 (for organization"loraserver") and 23 (for organization “org-1”), but with no way to know about that other than checking with the developer tools. Moreover, if I hit the “loraserver service profile” one, which belongs to the “loraserver” organization, “org-1” remains selected at the side bar:

I think this is related to the same issue :slight_smile: Service-profile listing should be per-organization, not per system. E.g. in my case the request looks like:

https://test.loraserver.io/api/service-profiles?limit=10&organizationID=1

As with your re-generated API, organization_id=1 instead of organizationID=1 is expected, it is considered as 0 thus the filter is not applied. Therefore you see the full list. You will probably see many more issues related to this.

(the background behind this issue is that https://developers.google.com/protocol-buffers/docs/proto3#json defines that fields are lowerCamelCase and can be overridden using the json_name field option in .proto file, which was incorrectly implemented by the gRPC gateway as it was using .getName() instead of .getJsonName()).

Ah, great, I’ll wait for you to fork and include the grpc-gateway then and omit any more of these for now.
Thanks!

If you push the latest master, run make requirements then make api should generate the correct code :slight_smile:

Thanks! Though I’m having a hard time getting it to work.

If I just run make requirements, dep tells me that the fork is unused and make api renders the same old results.

If i change "github.com/grpc-ecosystem/grpc-gateway/runtime" import at root_run.go to point to "github.com/brocaar/grpc-gateway/runtime", then make complains with:

vendor/github.com/brocaar/grpc-gateway/runtime/handler.go:12:2: use of internal package not allowed

Maybe I’m missing something, possibly just a coffee to see my mistake, but I thought I’d ask anyway. :sweat_smile:

Not sure what is going wrong. No need to change the imports. What you need to run is:

go get -u github.com/brocaar/grpc-gateway/protoc-gen-grpc-gateway
go get -u github.com/brocaar/grpc-gateway/protoc-gen-swagger

This is only for generating code so you need to make sure that the protoc-gen-grpc-gateway and protoc-gen-swagger binaries are updated.

Btw, I fixed the error on organization delete!

It’s really odd, I had tried just go get -u ... but it wasn’t working either. I also tried cleaning things and getting them again:

go clean -i github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway...
go clean -i github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger...
go clean -i github.com/brocaar/grpc-gateway/protoc-gen-swagger...
go clean -i github.com/brocaar/grpc-gateway/protoc-gen-swagger...
go get -u github.com/brocaar/grpc-gateway/protoc-gen-grpc-gateway
go get -u github.com/brocaar/grpc-gateway/protoc-gen-swagger

Everything remained the same. I then changed gen.sh with GOSUBPATHS="/src:/src/github.com/brocaar/grpc-gateway/third_party/googleapis", but again, no luck.

A couple of times I wiped out every *.pb.go, *.pb.gw.go and *.swagger.json file under api/ before generating them without luck.

You could try which protoc-gen-swagger and which protoc-gen-grpc-gateway to find out where the protoc-gen-... binaries are located and remove these. Or try again using the provided development Docker container?

They where installed at my gopath/bin, I removed both, got your forked versions which installed to the same path but still the swagger.json files are being generated with snake case fields. I’m starting to think there’s some packages cache going on or something like that (though I reran the clean commands with the -cache flag, but still no luck).

I’ll let you know if I fix it.

:man_facepalming: my mistake, I did push my own fork, but as I made the change under $GOPATH/src/github.com/grpc-ecosystem/... it works for me. When you do a go get, it still sees the original code as a requirement as go get installs it under github.com/brocaar/grpc-gateway/..., which then relies on github.com/grpc-ecosystem/grpc-gateway/...

I’m not sure what the best way is, I could update all the import paths in my fork, but that makes it really hard to stay up-to-date with the upstream. Best would be if the GetJsonName change is merged asap into the original repo.

1 Like

Ah, that was it. Thanks for catching that.

I agree it’d be messy and it’s probably better that those of us developing on top of the source do our own forks of grpc-gateway for the meantime, and then just go back to the original repo when it gets merged. We are all introducing custom stuff and have to keep up with your source anyway.

Edit: Worked like a charm!

1 Like

In a somewhat related note and to continue with the feedback, when navigating as an admin, changing the selected organization will always navigate to the applications list of the organization instead of the section in which one was on the previous organization. In other words, if I wanted to check all service profiles for some reason, it’s not easy as I need to select the first organization, then select service profiles; select the second organization, then select service profiles; and so on. On the other hand, always going to the applications list is consistent, makes the code simpler and is only an “issue” when entering as an admin, so no burden on the users.

Thought I’d comment on it to see what others think.