Having httplib callback option

[arnoutdekimo]

Hi,

For my application, I'd require a firmware update of HTTPS. I am considering using the HTTP functionality from the ublox modem for this, and using the APIs in ubxlib.

However:

• For POSTS, apparently there is a module AT-command limit of 128B to post. (Not an issue for firmware update, but for status update for instance this is)
• For GET, the modem presumably does support a larger buffer, but the library expects a single pointer to a data structure that will be filled in (during cellCallback)

For this last part, in the microcontroller world, it is not possible to supply a RAM pointer for huge transfers. Instead, when a piece comes in, some callback function should be called to then write this to flash. Currently, the HTTP client library API does not support this, although the module does.

(A workaround could possibly be to split the HTTP transfer up into mulitple pieces, supplying an offset in the haeder, OR, just reverting to socket mode)

Given the POST HTTP HW limitation, I'll probably revert to sockets anyway, but posting here anyway for reference.

Kind regards,
Arnout

[arnoutdekimo]

(As an update, apparently the library -does- offer the callback (pResponseCallback), but I don't think it supports not supplying a pointer to a buffer that is not the total size of the transfer)

[RobMeades]

The HTTP POST request is only size-limited at the AT interface for the "POST data command", AT+UHTTPC=<profile_id>,5..., there is no such limitation with the "POST file command", AT+UHTTPC=<profile_id>,4.... The cell API exposes uCellHttp and uCellFile and then the common uHttpClient() mashes the two together to make a non-length-limited HTTP API that supports blocking or asynchronous operation.

That said, using the HTTP Client in the module is always a bit of a compromise: in the cellular case, as indicated, it does its "non-length-limitedness" by using the module file-system as an intermediate, which means it is not as fast as it could be and for very large things (100's of kbytes) you might hit a module file-system limit. If you have lots of space in your MCU file system, so there is no particular advantage in using the module's file system, then it may be better to download directly, as you say.

AlexMaron had a similar problem and I wrote him some sample code that did the HTTP file download over a secure socket to his MCU where he could store it in his own file system, which he managed to use successfully: you might find that a useful reference.

EDIT:

but I don't think it supports not supplying a pointer to a buffer that is not the total size of the transfer

Yes, that's true, I had thought about having some sort of "chunked" callback at this level, since the data coming from the module is in any case "chunked". Will put on the TODO list, a bit lower down :-).

[arnoutdekimo]

Thanks for that clarification, and for the sample code, very useful!

I think I can then use the http library for everything aside from really big files, and use direct socket calls for the big ones.
I'll probably also look into increasing the baudrate, as for +- 240kB worst case updates, even theoretically at 115200 baud, this is much slower than the cellular link, which would be a shame.
I am slightly concerned with your comments regarding

setting the module to run the UART at 230,400 or 460,800, and hoping that doesn't have any undesirable side-effects

Have you experienced issues at higher baudrates for the sara-R4 before?

Anyway thanks already for the feedback

[RobMeades]

Have you experienced issues at higher baudrates for the sara-R4 before?

No, but unfortunately ubxlib does not test that. There is a TODO even lower down in the list "test high cellular UART baud rate, logging off"; the issue with some of our ubxlib test rigs, and unfortunately all of the SARA-R422 ones fall into this category, is that they are formed by one of our standard cellular module EVKs connected to a separate MCU board via a 3V3 <-> 1V8 bidirectional level shifter on the UART lines, and those bi-directional level-shifters don't work for flow control lines, hence UART flow control is not connected in those cases. I would guess that higher baud rates likely would require UART flow control.

I don't anticipate that SARA-R422, of itself, would present a problem with higher baud rates, it is more that the dynamics of the system as a whole become more important, are more stressed: things like call-back reaction time, potentially RTOS tick rate, task occupancy, etc. likely all matter rather more. Another "suck it and see" I'm afraid.

[RobMeades]

Hi Arnout. I am now implementing the "chunked" HTTP Client API that you asked for here. I have posted what I think the API might be:

master...preview_http_chunked_api_rmea

If you have time to take a look I would be interested in your feedback. Not an issue if you don't have the time.

[RobMeades]

The chunked API is now available on master here in commit 61dc45a. I will now delete the preview branch and close this issue.