v4.0-beta1 breaks esp-azure due to #include search order differences

[jsam589]

I am trying to port a working application from v3.3 to v4.0-beta1.

The application uses Espressif's port of Azure IoT component from: https://github.com/espressif/esp-azure.git

I find that the application fails to build under v4.0-beta1.

To focus on the root problem, I found that it can be reproduced using just the IDF's example console application and esp-azure.

The problem I am about to describe occurs whether using CMake or legacy make build systems.
It also occurs on both Windows and Linux hosts.

It builds fine under v3.3.
It encounters fatal errors very quickly under v4.0-beta.

In both cases, I copy the console example and then add esp-azure under components. I end up with this simple structure:

console/
. . components/
. . . . esp-azure/
. . . . cmd_nvs/
. . . . cmd_system/
. . main/

Under v3.3 (crosstool-ng 5.2.0 toolchain) and make, it builds with no complaint.

Under v4.0-beta1 (esp32-2019r1 8.2.0 toolchain) and make, IT APPEARS TO INCLUDE THE WRONG sntp.h AND strings.h HEADER FILES.

The first casualty in the build sequence is asio.cpp, which craps out with the messages shown below (just after libapp_update.a is built).

Note that the locations of sntp.h and strings.h are in the azure component, which I think is wrong. I wonder about them using the same names for these files as C library, but it worked in v3.3...

Also, I can mess around with hiding the azure sntp.h file, but then strings.h is still wrong and different messages will appear. This will happen for most files, so it seem a fundamental break of the #include search order or something like that.

= = = = = = = = = = = = = = =

Jerry@JW10VMESP1 MINGW32 ~/esp40/projy/conso2
$ make
Python requirements from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/requirements.txt are satisfied.
GENCONFIG
Project is not inside a git repository, will not use 'git describe' to determine PROJECT_VER.
App "console" version: 1
CC build/bootloader/bootloader_support/src/bootloader_random.o
CC build/bootloader/bootloader_support/src/flash_encrypt.o
CC build/bootloader/bootloader_support/src/esp_image_format.o
CC build/bootloader/bootloader_support/src/flash_partitions.o
CC build/bootloader/bootloader_support/src/bootloader_flash_config.o
CC build/bootloader/bootloader_support/src/bootloader_clock.o
CC build/bootloader/bootloader_support/src/bootloader_common.o
CC build/bootloader/bootloader_support/src/bootloader_utility.o
CC build/bootloader/bootloader_support/src/bootloader_init.o
CC build/bootloader/bootloader_support/src/flash_qio_mode.o
CC build/bootloader/bootloader_support/src/bootloader_flash.o
CC build/bootloader/bootloader_support/src/esp32/flash_encrypt.o
CC build/bootloader/bootloader_support/src/esp32/bootloader_sha.o
CC build/bootloader/bootloader_support/src/esp32/secure_boot.o
CC build/bootloader/bootloader_support/src/esp32/secure_boot_signatures.o
AR build/bootloader/bootloader_support/libbootloader_support.a
CC build/bootloader/log/log.o
AR build/bootloader/log/liblog.a
CC build/bootloader/spi_flash/spi_flash_rom_patch.o
AR build/bootloader/spi_flash/libspi_flash.a
CC build/bootloader/micro-ecc/micro-ecc/uECC.o
AR build/bootloader/micro-ecc/libmicro-ecc.a
CC build/bootloader/soc/esp32/emac_hal.o
CC build/bootloader/soc/esp32/rtc_clk.o
CC build/bootloader/soc/esp32/rtc_time.o
CC build/bootloader/soc/esp32/rtc_sleep.o
CC build/bootloader/soc/esp32/gpio_periph.o
CC build/bootloader/soc/esp32/rtc_clk_init.o
CC build/bootloader/soc/esp32/spi_periph.o
CC build/bootloader/soc/esp32/rtc_init.o
CC build/bootloader/soc/esp32/sdio_slave_periph.o
CC build/bootloader/soc/esp32/rtc_wdt.o
CC build/bootloader/soc/esp32/sdmmc_periph.o
CC build/bootloader/soc/esp32/cpu_util.o
CC build/bootloader/soc/esp32/rtc_periph.o
CC build/bootloader/soc/esp32/rtc_pm.o
CC build/bootloader/soc/esp32/soc_memory_layout.o
CC build/bootloader/soc/src/soc_include_legacy_warn.o
CC build/bootloader/soc/src/memory_layout_utils.o
CC build/bootloader/soc/src/lldesc.o
CC build/bootloader/soc/src/hal/spi_hal.o
CC build/bootloader/soc/src/hal/spi_hal_iram.o
CC build/bootloader/soc/src/hal/spi_flash_hal.o
CC build/bootloader/soc/src/hal/spi_flash_hal_iram.o
CC build/bootloader/soc/src/hal/spi_slave_hal.o
CC build/bootloader/soc/src/hal/spi_slave_hal_iram.o
AR build/bootloader/soc/libsoc.a
CC build/bootloader/main/bootloader_start.o
AR build/bootloader/main/libmain.a
CC build/bootloader/efuse/esp32/esp_efuse_table.o
CC build/bootloader/efuse/src/esp_efuse_fields.o
CC build/bootloader/efuse/src/esp_efuse_utility.o
CC build/bootloader/efuse/src/esp_efuse_api.o
AR build/bootloader/efuse/libefuse.a
LD build/bootloader/bootloader.elf
esptool.py v2.8-dev
Python requirements from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/requirements.txt are satisfied.
Building partitions from /home/Jerry/esp40/projy/conso2/partitions_example.csv...

CC build/app_trace/heap_trace_tohost.o
CC build/app_trace/app_trace.o
CC build/app_trace/host_file_io.o
CC build/app_trace/app_trace_util.o
CC build/app_trace/gcov/gcov_rtio.o
AR build/app_trace/libapp_trace.a
Project is not inside a git repository, will not use 'git describe' to determine PROJECT_VER.
App "console" version: 1
CC build/app_update/esp_app_desc.o
CC build/app_update/esp_ota_ops.o
AR build/app_update/libapp_update.a
CXX build/asio/asio/asio/src/asio.o
In file included from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:7,
from c:\msys32_19\msys32\opt\xtensa-esp32-elf\xtensa-esp32-elf\sys-include\string.h:24,
from c:\msys32_19\msys32\opt\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\8.2.0\cstring:42,
from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/macro_utils.h:18,
from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/pal/inc/sntp.h:16,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/port/esp32/include/lwipopts.h:46,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/opt.h:51,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/sockets.h:42,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error_code.hpp:21,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error.hpp:19,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/error.ipp:20,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/src.hpp:22,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/src/asio.cpp:11:
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:18:1: error: 'COUNT_ARG' was not declared in this scope
MOCKABLE_FUNCTION(, STRING_HANDLE, STRING_new);
^~~~~~~~~~~~~~~~~
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:18:1: note: the macro 'COUNT_ARG' had not yet been defined
In file included from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/pal/inc/sntp.h:16,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/port/esp32/include/lwipopts.h:46,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/opt.h:51,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/sockets.h:42,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error_code.hpp:21,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error.hpp:19,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/error.ipp:20,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/src.hpp:22,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/src/asio.cpp:11:
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/macro_utils.h:4351: note: it was later defined here
#define COUNT_ARG(...) C2(COUNT_ARG_, ISEMPTY(__VA_ARGS__))(__VA_ARGS__)

In file included from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:7,
from c:\msys32_19\msys32\opt\xtensa-esp32-elf\xtensa-esp32-elf\sys-include\string.h:24,
from c:\msys32_19\msys32\opt\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\8.2.0\cstring:42,
from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/macro_utils.h:18,
from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/pal/inc/sntp.h:16,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/port/esp32/include/lwipopts.h:46,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/opt.h:51,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/sockets.h:42,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error_code.hpp:21,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error.hpp:19,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/error.ipp:20,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/src.hpp:22,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/src/asio.cpp:11:
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:18:1: error: expected primary-expression before ',' token
MOCKABLE_FUNCTION(, STRING_HANDLE, STRING_new);
^~~~~~~~~~~~~~~~~
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:18:1: error: expected primary-expression before 'void'
MOCKABLE_FUNCTION(, STRING_HANDLE, STRING_new);
^~~~~~~~~~~~~~~~~
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:18:1: error: 'IF' was not declared in this scope
MOCKABLE_FUNCTION(, STRING_HANDLE, STRING_new);
^~~~~~~~~~~~~~~~~
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:18:1: note: the macro 'IF' had not yet been defined
In file included from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/pal/inc/sntp.h:16,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/port/esp32/include/lwipopts.h:46,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/opt.h:51,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/sockets.h:42,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error_code.hpp:21,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error.hpp:19,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/error.ipp:20,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/src.hpp:22,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/src/asio.cpp:11:
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/macro_utils.h:12570: note: it was later defined here
#define IF(condition, trueBranch, falseBranch) C2(IF,ISZERO(condition))(trueBranch, falseBranch)

In file included from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:7,
from c:\msys32_19\msys32\opt\xtensa-esp32-elf\xtensa-esp32-elf\sys-include\string.h:24,
from c:\msys32_19\msys32\opt\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\8.2.0\cstring:42,
from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/macro_utils.h:18,
from C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/pal/inc/sntp.h:16,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/port/esp32/include/lwipopts.h:46,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/opt.h:51,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/lwip/lwip/src/include/lwip/sockets.h:42,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error_code.hpp:21,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/error.hpp:19,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/error.ipp:20,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/include/asio/impl/src.hpp:22,
from C:/msys32_19/msys32/home/Jerry/esp40/esp-idf/components/asio/asio/asio/src/asio.cpp:11:
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:19:1: error: expected primary-expression before ',' token
MOCKABLE_FUNCTION(, STRING_HANDLE, STRING_clone, STRING_HANDLE, handle);
^~~~~~~~~~~~~~~~~
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:19:1: error: 'handle' was not declared in this scope
MOCKABLE_FUNCTION(, STRING_HANDLE, STRING_clone, STRING_HANDLE, handle);
^~~~~~~~~~~~~~~~~
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:19:1: note: suggested alternative: 'random'
C:/msys32_19/msys32/home/Jerry/esp40/projy/conso2/components/esp-azure/azure-iot-sdk-c/c-utility/inc/azure_c_shared_utility/strings.h:19:1: error: 'COUNT_ARG' was not declared in this scope
MOCKABLE_FUNCTION(, STRING_HANDLE, STRING_clone, STRING_HANDLE, handle);
^~~~~~~~~~~~~~~~~

[ESP_Piyush]

We have updated https://github.com/espressif/esp-azure a couple of days back so that it can now compile with IDF 4.0. Can you check with that?

[jsam589]

Initial test worked. I got fresh clone of esp-azure and dropped into console example. Built successfully with idf.py under v4.0-beta1. I will need a day for additional testing with make, etc., but the initial results are positive. Thanks!