Use of cloud functions ... who are you using and what do you need?

[kolban]

Howdy folks,
As we know, cloud is an important force in our ESP32 world. I view this as the ability for our ESP32s to generate data and make that data available over the Internet. To that end, I envisage a cloud service provider. That provider makes available the capability for the ESP32 to transmit data and have it received by the provider. What the provider then does with it is a matter of question. Typical answers would include:

* Warehousing the data
* Relaying the data to some other service/application/device
* Processing the data with server side functions

I am also assuming that a cloud service provider would provide a mechanism where data can be pushed down to our ESP32s.

Other qualities of service would include areas such as device management and security.

Now the question ... for cloud service providers ... who are we using?

I'm imagining the usual suspects:

* Amazon AWS
* Google Cloud Platform
* Microsoft Azure
* IBM Cloud

but are there others? What qualities of these providers are you enjoying or disliking?

And from an ESP32 specific perspective, are you finding the support for the cloud provider on ESP32 satisfactory? Is it easy to hook into them? Are their APIs sufficiently easy to use?

[Deouss]

Microsoft Azure is the best

https://azure.microsoft.com/en-us/pricing/

[kolban]

Howdy Deouss and many thanks for taking the time to respond. If I may ask, what are the qualities you are liking of Azure relative to the other players? Is it just price? Are you connecting your ESP32 to an Azure cloud? What protocol does Azure use (MQTT?)? What are you doing with the data when it hits Azure? If you are using an ESP32, was it easy to connect and publish/write data?

[fly135]

My project is 100% HTTP REST client for communicating to cloud. I also use Pubnub as a REST client. Previously used an MQTT library, but had problems with waiting for data and thought it might be that Pubnub didn't support MQTT correctly. So I switched to a REST client approach using Pubnub's REST API. Still have issues with data being delivered with it. Might be a Pubnub thing.

On the cloud end there are programmers who handle that and define the REST API for me to use. So I use both cloud and mqtt type messaging.

John A

[Deouss]

Howdy Deouss and many thanks for taking the time to respond. If I may ask, what are the qualities you are liking of Azure relative to the other players? Is it just price? Are you connecting your ESP32 to an Azure cloud? What protocol does Azure use (MQTT?)? What are you doing with the data when it hits Azure? If you are using an ESP32, was it easy to connect and publish/write data?

With Azure you have full Windows Server access with choice of your services. Complete freedom of operations.
You can use Mosquitto, EMQ or MQQTBox - https://www.microsoft.com/en-us/p/mqttbox/9nblggh55jzg
Or just write your own service as you wish

[PatrikB]

Hi,

Also using Azure, have a solution based on the Azure IoT Hub, using their C SDK.
https://github.com/Azure/azure-iot-sdk-c
and the PAL libraries from
https://github.com/Azure/azure-iot-pal-esp32

Using these message types Device Twin, D2C and Direct Methods together with their Serializer lib.
https://github.com/Azure/azure-iot-sdk- ... serializer

The rest of the solution in Azure uses PaaS components like,
SQL Server
App Service with a web app
Azure AD
Functions for event driven processing

This simplifies things a lot to not have to setup and manage VMs.

[p-rimes]

Would you know, I just went through a process of trying several different clouds and architectures on a recent project! So I can report some brief findings you might find useful.

My goal in this project was lowest latency between an ESP32 submitting an event (E.g. some sensor readings), and a website updating to display the details of the event. It wasn't a high priority to get a timely response back, but one was expected for acknowledgement and to display health status on an OLED display. Events cannot be dropped, downsampled or batched, each is considered significant and should be sent ASAP, and held in a queue until the response is acknowledged.

Now let me tell you that I am a big Google Cloud fan, and I've used nearly their whole stack professionally as well as quite a bit of AWS.

Google Cloud (and Firebase or App Engine even more so) is by far the easiest to use. The things you need for an app are all right there, usually named something sensible, and there are things that are very convenient that would be tricky to set up in AWS (and probably not free!). There is no AWS substitute for things like Cloud Dataflow, or Spanner, or Firebase RTDB. And DynamoDB from AWS is not in the same league as Google's BigQuery in my opinion. So for your serverless functions, you are probably going to be using some cloud database(s), and the scalability / flexibility of that database seems like it would make more of an impact than the particularities of the serverless implementation.

Be sure you are using a fast runtime (such as NodeJS, or Go). Something like Python comes at a cost for your users, but with serverless it comes with a literal cost for you too! So you might as well be using something performant. And afterwards you can reduce the RAM size requested to save a bit more.

Also, I use the serverless.com framework to deploy my cloud functions. So it is possible to use the same function handler on multiple clouds with only a few tweaks to the serverless.yml config file. Very handy for experimentation, and on AWS it sets up a lot of the connecting infrastructure when you run it (those steps aren't necessary on Google Cloud, but they would be automated here too).

Here's what I tried for my low-latency use-case:
- Google Cloud Functions, HTTPS
- Google Cloud Functions, MQTT
- AWS Lambda, HTTPS
- AWS Lambda, MQTT
(all in the US-West regions -- closest to me, except Google Cloud which is in US-central)

I found that Google Cloud Functions had a P99 response time of ~4s and worst case of around 9s(!). Approx 1.5s is the HTTPS connection setup time from the ESP32, reusing the TLS state if possible.
When using the Google IoT Core MQTT bridge, the ESP32 could certainly upload the data faster than with HTTPS (no need to re-establish a connection), but there was a frustrating variable latency of the Google PubSub layer delivering the message to subscribers. So the response times were all over the place, sometimes 0.5s, but other times 6s.

With AWS Lambda triggered by MQTT, there was still a variable delay (not sure what happens in AWS-land between MQTT and Lambda. They don't mention a "PubSub"-like layer, but there may be one. So it took a while for the lambda to be invoked). About 2s to 5s, evenly distributed.

Finally, AWS Lambda triggered by HTTPS won easily. ~2.5s for the first few requests after a deployment, and then a rock-solid 0.8s for all requests after that (they rotate the server every once in a while and it goes back to 2.5as for a bit). As well, it seems the TCP/TLS connection state is nearly always re-used, contributing to a big savings on the ESP32. This might be different with a service under full scale, but in my case there are only a few devices.

[Deouss]

Taken from this forum Board index from top )

Azure IoT Workbench now supports ESP32 devices

[Pcborges]

Hi, I am trying ESP32 with PlatformIO to send temperature and humidity data to Azure.
I have got the ESP32 script to send the data as per terminal information:

Code: Select all

{"deviceId":"ESP32-DevKitC", "messageId":237, "temperature":17.000000, "humidity":72.800003}
Info: >>>IoTHubClient_LL_SendEventAsync accepted message for transmission to IoT Hub.
Info: >>>Confirmation[236] received for message tracking id = 236 with result = IOTHUB_CLIENT_CONFIRMATION_OK
Send Confirmation Callback finished.

I also can get information from inside Azure that there is activity, data seems to be arriving at the defined IOTHub.

I tried to "consume" this data and chanel it to SQL, BLOBs and other destinations to no avail.
I am particularly puzled with this Stream Analytics job Query, I just seem not to be getting it right.

SQL table: sensorData. Columns: myTemperature, myHumidity

Stream Analytics job Query:

Code: Select all

SELECT
    humidity as myHumidity,
    temperature as myTemperature
INTO
    myTemperature,
    myHumidity
FROM
    temperature,
    humidity

Assistance welcome.
Regards
Paulo