AWS IoT SDK integration

[mjmottram]

Hi,

I am trying to expand on some of the demos in the AWS IoT SDK / ESP-IDF in order to interact with a shadow. Right now I'm unable to compile my code though: I have an app that works without the IoT functions, but cannot work out how to compile against the libraries and headers from https://github.com/aws/aws-iot-device-sdk-embedded-C (in fact, I can't event compile the libraries from the SDK).

Right now I have the following structure:

- app_dir/:
- Makefile
- main/:
- component.mk
- Kconfig.projbuild
- app.c
- certs/ (certificates)
- include/
- aws_iot_config.h
- aws_dir/ (link to the aws-iot-device-sdk-embedded-C directory)

I have the headers necessary included in the component.mk (COMPONENT_ADD_INLUDEDIRS := include aws_dir/include aws_dir/platform/linux/common aws_dir/platform/linux/mbedtls); but compilation fails because no libraries for the AWS code exist. I can't find concrete examples on how to compile this; the AWS example shadow READMEs all use custom makefiles, the same examples ported into the esp-idf repo use the component.mk format but do not appear to work.

Thanks in advance!
Matt

[mjmottram]

OK, so I followed the simple method used by https://github.com/carsonmcdonald/esp32-aws-iot to create a component.mk in an aws-iot sdk folder and then build the app using that sdk within a components folder of my app. This wasn't very clear to me!

I'm now having trouble as, in the aws sdk examples, the path to the aws certificates are used. In the thing_shadow example in esp-idf they are embedded in the binaries, but the code seems to be expecting paths for the certs whereas in the embedded examples it sets the values of certificates directly:

Code: Select all

#if defined(CONFIG_EXAMPLE_EMBEDDED_CERTS)                                                                                    
      sp.pClientCRT = (const char *)certificate_pem_crt_start;                                                        
      sp.pClientKey = (const char *)private_pem_key_start;                                                            
      sp.pRootCA = (const char *)aws_root_ca_pem_start;                                                                                
#elif defined(CONFIG_EXAMPLE_FILESYSTEM_CERTS)                                                                          
      sp.pClientCRT = DEVICE_CERTIFICATE_PATH;                                                                        
      sp.pClientKey = DEVICE_PRIVATE_KEY_PATH;                                                                        
      sp.pRootCA = ROOT_CA_PATH;                                                                                      
#endif   

This embedded version doesn't seem to work (and I cannot get the version that uses the SD card to transfer the certs to compile).

[ESP_Angus]

Hi Matt,

ESP-IDF bundles the AWS IoT C SDK, so it shouldn't be necessary to include the AWS IoT SDK code directly in your project.

However, to save compile time the AWS IoT component is disabled by default. You can enable it under "Component Config" when running "make menuconfig" for your project:
https://docs.espressif.com/projects/esp ... ws-iot-sdk

I strongly encourage you to start with one of the two AWS IoT examples in ESP-IDF, first:
https://github.com/espressif/esp-idf/tr ... ls/aws_iot

These examples have full instructions in the README at the link above, which should get you up and running. Once you have an example fully working, you can copy the example project into a new (external) directory to create your own project or you can adapt your existing project using the example as a reference.

carsonmcdonald's repo and project was a fantastic community resource back in 2016, before ESP-IDF had AWS IoT SDK bundled with it. However since ESP-IDF added native support I don't think it's being maintained.


Angus

[mjmottram]

Thanks for the help, I have a version that is working with the shadow correctly now (I guess I wasn't familiar with the sdkconfig parameters).

One question - if I change the while loop interval to reduce the rate of shadow updates past ~5 seconds then I get timeout errors from the shadow. https://forums.aws.amazon.com/thread.js ... dID=233180 suggests that there was a keepalive variable, but the only one I can see in the esp-idf/components/aws-iot code has been updated to 600 seconds. Is there any way to reduce the frequency of shadow updates without getting this error (I see that the updates are in fact getting through OK...)

[ESP_Angus]

One question - if I change the while loop interval to reduce the rate of shadow updates past ~5 seconds then I get timeout errors from the shadow. https://forums.aws.amazon.com/thread.js ... dID=233180 suggests that there was a keepalive variable, but the only one I can see in the esp-idf/components/aws-iot code has been updated to 600 seconds. Is there any way to reduce the frequency of shadow updates without getting this error (I see that the updates are in fact getting through OK...)

Making these changes to the example works for me, no timeouts seen:

Code: Select all

diff --git a/examples/protocols/aws_iot/thing_shadow/main/thing_shadow_sample.c b/examples/protocols/aws_iot/thing_shadow/main/thing_shadow_sample.c
index 0a382b9d6..ff6299a07 100644
--- a/examples/protocols/aws_iot/thing_shadow/main/thing_shadow_sample.c
+++ b/examples/protocols/aws_iot/thing_shadow/main/thing_shadow_sample.c
@@ -287,9 +287,9 @@ void aws_iot_task(void *param) {
 
     // loop and publish a change in temperature
     while(NETWORK_ATTEMPTING_RECONNECT == rc || NETWORK_RECONNECTED == rc || SUCCESS == rc) {
-        rc = aws_iot_shadow_yield(&mqttClient, 200);
+        rc = aws_iot_shadow_yield(&mqttClient, 10);
         if(NETWORK_ATTEMPTING_RECONNECT == rc || shadowUpdateInProgress) {
-            rc = aws_iot_shadow_yield(&mqttClient, 1000);
+            rc = aws_iot_shadow_yield(&mqttClient, 10);
             // If the client is attempting to reconnect, or already waiting on a shadow update,
             // we will skip the rest of the loop.
             continue;
@@ -314,8 +314,6 @@ void aws_iot_task(void *param) {
         }
         ESP_LOGI(TAG, "*****************************************************************************************");
         ESP_LOGI(TAG, "Stack remaining for task '%s' is %d bytes", pcTaskGetTaskName(NULL), uxTaskGetStackHighWaterMark(NULL));
-
-        vTaskDelay(1000 / portTICK_RATE_MS);
     }

The actual update rate is still only about one every 600-800ms though, I believe this is limited at the AWS end. (Note that because of the shadowUpdateInProgress variable in the loop, the client doesn't send a new shadow update until the previous one has completed.)

If you're still seeing problems, please post the exact code that you're running and the exact output that you're getting. Note that if you remove the check for shadowUpdateInProgress and pump in updates at a maximum rate then it's inevitable that a queue (either in the ESP32, or maybe even at the AWS end) will eventually overflow due to too many updates being queued up at once. The lost updates may show up as SHADOW_ACK_TIMEOUT.

[ESP_Angus]

(If you need high frequency updates, it may be better to use raw AWS MQTT messages rather than the shadow API. Probably cheaper, too!)