Next steps

[ivmarkov]

Status Quo

With the latest release, embedded-svc lost quite some weight. Following traits (did anybody know about them??) no longer there:

• Ping, EventBus+Postbox, SysTime, Timer
• Also 99% of the "utilities" removed, including the asyncify adaptors which are now merged into esp-idf-svc directly

What is still in here?

• ??? Some IPv4 structures which are not in core::net
• Wifi trait + structures
• ??? Eth trait (do we need this? it is basically just start/stop/is_up)
• OTA trait + structures
• HTTP client & server traits and structures - many of these common between the two, hence the common shared super module
• MQTT client traits and structures
• WS send/receive traits and structures
• ??? "Storage" trait - this is really just key-value in disguise, perhaps we don't need this???

What the future looks like?

• Option A: Retire completely. It was not a smashing success anyway!
• Option B: Continue it somehow, but then:
  • Split into micro-crates yes/no? What would that give us? I.e. embedded-hal is monolitic modulo the blocking-async split
  • What traits are useful and should remain there?
  • Remove heapless from the APIs? Might result in less ergonomic traits

[ivmarkov]

@MabezDev @jessebraham @bjoernQ @JurajSadel @bugadani @Vollbrecht - Please forward to others who might be interested

@lulf @Dirbaio - Topic might be interesting to you, given the existence of embedded-nal-async etc.

[bjoernQ]

Currently the wifi trait is the primary API for esp-wifi ... we want to change that. If we do that probably very few of our users will need the wifi trait anymore.

In bare-metal we don't implement any of the other traits currently.

I'm not sure about the OTA trait - currently our users need to do things manually since there is no ready-made implementation. We have an (outdated) example somewhere how to do that. Once we implement such a thing, we might implement the OTA trait (or maybe not)

Given embedded-svc shouldn't be the primary API for other crates it would only be useful when someone wants to write portable code but I'm not sure how realistic that is in reality. Maybe the traits like http,ws and mqtt might be useful for libraries but probably not wifi, eth and ota? I could imagine storage would also be something that could be useful in libraries but as you said maybe not the way it is currently.

I have no idea how much effort it will be to maintain this in future. If we expect a lot of effort which is better spent on other things, I guess everyone could live with option A.

Otherwise splitting it up into micro-crates sounds good since it's really a collection of very unrelated traits - but it will most likely increase the maintenance effort and give us nothing but making things look a bit more clean - so not sure if it's worth it.

I agree that an API which relies on another crate (heapless) is probably not ideal but I also see how not using it will make the API less ergonomic

TL;DR I don't have a strong opinion here. But given "It was not a smashing success" and regardless how much effort it is - there will be some effort - it might not be worth it long term.

[MabezDev]

I think so far, the WiFi trait is the most useful, and also the most likely to be used across different vendors. I think it could quite easily be implemented for the rp2040 with WiFi for example. I guess the question is why it hasn't already. Are the traits too esp specific that implementing other targets is difficult (IMO no)? Are we not doing enough to promote the crate and no one other than esp-rs users know about it? Maybe there isn't a need for this abstraction layer? It would be very interesting to hear the opinions of someone from outside of the esp-rs ecosystem. Regarding the other traits, maybe we could take a similar approach to getting ehal to 1.0, strip it back completely to the "known" valuable traits, and incrementally add stuff when there is demand for it. Heapless probably needs to be removed from the API, unless we see a heapless 1.0 on the horizon, it's probably worth exploring what some of the traits look like without heapless. It might be possible to implement scanning with an iterator approach instead of the alloc/heapless way, for example.

IMO embedded-svc is still useful, and in the same way, you can use ehal to switch from rp to esp or vice versa with relative ease you could conceivably do the same with embedded-svc for the networking part of an application. It could also provide a platform to create a generic captive portal crate that could work on anything that implemented at least the WiFi trait (there would also be some HTTP things involved here, but you get my drift, without this layer there will be X number of captive portal implementations).

[ivmarkov]

I think so far, the WiFi trait is the most useful, and also the most likely to be used across different vendors. I think it could quite easily be implemented for the rp2040 with WiFi for example. I guess the question is why it hasn't already. Are the traits too esp specific that implementing other targets is difficult (IMO no)?

I don't think so. Some might be, but I've put so much effort to make - say - the HTTP threads as generic as possible - yet - as efficient as possible for embedded scenarios that I kind of doubt this.

For Websockets, not sure it can get any simpler than what's in there already.

For the MQTT client... I know the traits are implementable for the ESP IDF MQTT client (obviously) and for rumqttc as well.

Are we not doing enough to promote the crate and no one other than esp-rs users know about it?

I think so. It does not have enough visibility for sure.

Maybe there isn't a need for this abstraction layer?

That's my biggest concern indeed. Perhaps network layer 5 ("app layer") is simply not the place where we should put abstractions for the rust embedded world. Sure, it had been successful in other domains, but these are as far from Rust embedded as it gets.

Here's my recent thinking: Say, the likes of edge-net, pico-serve and rust-mqtt become popular. (To use those on anything else than a hard-coded embassy-net dependency, they might need a rich, functioning embedded-nal-async but more on that later.)

So again, assuming the above become successful, why would I consider any alternatives when targeting Embedded? And how often would I switch between the alternatives? And are there any meaningful alternatives in the first place or are we solving a non-existent problem?? I mean:

• All of these crates from above would work with baremetal / embassy
• Assuming a decent embedded-nal-async impl for STD, they would work for embedded Linux too (if you are really into bean counting on this "big iron")
• They would also work for ESP IDF and whatever other embedded "OS"-es that care to provide a Posix/STD layer. Even for WASM, for testing purposes - provided an embedded-nal-async-to-websockets gateway sees the day of light. And - I dare say - these crates might be much better than their native "OS" equivalents. At least for async Rust, that is.

Where I'm going is that at least for the Networking traits, perhaps we should standardize on layer 3/4 instead of the application layer? (It is another topic that this ain't going easy, either, and I've heard concerns that even on layer 3/4 we are trying to standardize the non-standardizable, which is another way to say it is too much work, I guess.)

It would be very interesting to hear the opinions of someone from outside of the esp-rs ecosystem.

Tell me about it. :) But... crickets.

Regarding the other traits, maybe we could take a similar approach to getting ehal to 1.0, strip it back completely to the "known" valuable traits, and incrementally add stuff when there is demand for it. Heapless probably needs to be removed from the API, unless we see a heapless 1.0 on the horizon, it's probably worth exploring what some of the traits look like without heapless. It might be possible to implement scanning with an iterator approach instead of the alloc/heapless way, for example.

Yes. Or taking FnOnce callbacks.

IMO embedded-svc is still useful, and in the same way, you can use ehal to switch from rp to esp or vice versa with relative ease you could conceivably do the same with embedded-svc for the networking part of an application. It could also provide a platform to create a generic captive portal crate that could work on anything that implemented at least the WiFi trait (there would also be some HTTP things involved here, but you get my drift, without this layer there will be X number of captive portal implementations).

As per above, for the networking traits specifically not sure the app layer is the right place anymore. For example, you don't need app layer abstractions to write "a" captive portal plus some minimal HTTP server. You need network/transport layer. Abstracting the captive portal itself is app layer, but then again and as per above, do we really need this? (Also, not sure an already-useful captive portal implementation needs to interface with layer 2, unless you also want to automate this aspect of operating a captive portal in the captive portal crate itself of course.)

To summarize - my feedback is probably not getting us anywhere, but wanted to share a bigger context as to why I'm raising the topic of what to do with the embedded-svc traits in the first place.

EDIT: rephrased and fixed the network layers numbering, as I managed to make a mess with the numbers. :)

[ivmarkov]

The above speech does not cover the Wifi trait of course, which is not app layer. But then how much application code would you have around bringing up the wifi and keeping it up all the time?... So that you can mitigate a potentially big porting effort of that code? Not sure...

[Vollbrecht]

I added this issue as an topic on today's rust-embedded meeting on Matrix, to get more feedback from outside.

[ivmarkov]

Can you share a link to the mtg here?

[Vollbrecht]

matrix link to the discussion: https://matrix.to/#/!BHcierreUuwCMxVqOf:matrix.org/$5RsNCpr-nWBcIMK1TfUUruuFFBmhrl4szVcI_q3XX3E?via=matrix.org&via=catircservices.org&via=tchncs.de
hackmd meeting link : https://hackmd.io/CfejK9IDTIaLGzULJZTuug?both

[lulf]

Sorry it took a while to respond, but I've had to think about this for a while. I really appreciate the effort that has gone into embedded-svc because it raises a lot of interesting questions. I think the value of having a trait increases the more implementations you have, and also ties into what types of applications people write for embedded.

• Things like embedded-io Read + Write makes a ton of sense to me because it allows upper layers to not care where that TCP connection came from. It also allows transparently making 'layers' of protocol handling on top (TCP + TLS).
• Things like embedded-nal TcpClient/TcpServer makes sense to me because there are multiple network stacks and it allows upper layers to own the setup and teardown of a TCP connection. Challenge here is that different stacks may support different options, so traits end up being lowest common denominator.
• Things like embedded-svc HTTP and MQTT traits make less sense to me because they are often used 'directly' in the application logic, and there are few abstractions on top. Yes you have things like servlets/javabeans, but they actually provide a different higher level interface and tend to hardcode their http implementation.
• Things like embedded-svc OTA traits. OTA feels highly dependent on the firmware distribution mechanism which is 'company specific', a specific implementation of a protocol like Eclipse hawkBit makes more sense to me.

Now consider other programming language ecosystems, I've never seen a HTTP or MQTT 'interface type' in Java or Go, even though there are multiple clients in the wild.

That being said, I think perhaps embedded-svc might be a bit early to the game, in the sense that there not that many implementations of said traits because there is not a need for it yet. But the need for such traits rise naturally over time.

[ivmarkov]

Appreciate the feedback! Comments inline.

• Things like embedded-io Read + Write makes a ton of sense to me because it allows upper layers to not care where that TCP connection came from. It also allows transparently making 'layers' of protocol handling on top (TCP + TLS).

100%

• Things like embedded-nal TcpClient/TcpServer makes sense to me because there are multiple network stacks and it allows upper layers to own the setup and teardown of a TCP connection. Challenge here is that different stacks may support different options, so traits end up being lowest common denominator.

Indeed. e-nal-async ending up as a lowest common denominator by necessity is not concerning me, as we are in the embedded space, so we probably don't need all the bells and whistles of the transport protocols exposed via these traits.

What is concerning for me is that currently e-nal-async might not be passing (my) threshold for a lowest common denominator. It is missing essential (even for embedded) functionality. Namely:

• Splittable connections and more importantly - factories (UdpStack and TcpConnect) - that offer splittable connections.
• A TcpServer factory that has some form of the accept metaphor
• Multicast trait for the sockets returned by the UDP factory

I think folks are underestimating the roles of the factories here, where IMO these are kind of the core value of e-nal-async. I.e. if e.g. the UdpStack / TcpConnect factories are not rich enough in expressiveness to offer me - say - splittable connections, I won't use these factories, which undermines the very purpose of e-nal-async in the first place - as just using sockets without the factories means I can stick to embedded-io-async's Read + Write and my own couple of simplistic send/receive traits for UDP send/receive. Wy would I need e-nal-async then? Hence rust-embedded-community/embedded-nal#106 and hopefully more to come.

• Things like embedded-svc HTTP and MQTT traits make less sense to me because they are often used 'directly' in the application logic, and there are few abstractions on top.

Indeed. The higher you climb in the network stack, the less likely you'll see useful / used abstractions.

Yes you have things like servlets/javabeans, but they actually provide a different higher level interface and tend to hardcode their http implementation.

A nit: I tend to disagree w.r.t. the Servlet API. It is created precisely to allow one to create cross-platform and HTTP-server- agnostic HTTP handling code. Which is what embedded_svc::http::server does as well.

• Things like embedded-svc OTA traits. OTA feels highly dependent on the firmware distribution mechanism which is 'company specific', a specific implementation of a protocol like Eclipse hawkBit makes more sense to me.

Let me check this.

Now consider other programming language ecosystems, I've never seen a HTTP or MQTT 'interface type' in Java or Go, even though there are multiple clients in the wild.

A nit: the HttpUrlConnection abstract class in Java serves the purpose of an HTTP client, and Apache's HTTP client used to have an impl that implemented it. If memory serves right, we had a production deployment that could switch between the JRE and the Apache one. Anyway - past life. :)

But yeah, the higher in the network stack, the less reusability, I guess.

That being said, I think perhaps embedded-svc might be a bit early to the game, in the sense that there not that many implementations of said traits because there is not a need for it yet. But the need for such traits rise naturally over time.

So either we'll retire them, or perhaps better yet - let them wane for a few months (years?) and either the need arises, or we retire them by then for real!