Help using CLion + CMake + ESP-IDF v3.2.2

[meowsqueak]

(Moderator's note: Moved to a new thread.)

I've spent a bit of time this morning trying out the new CMake build system. I had a few issues with changes between 3.2.2, 3.3-beta3 and master but I was able to resolve those and get a non-trivial ESP32 project, written in C (not C++), with custom components, to build correctly from the command line (i.e. mkdir build; cd build; cmake ..; make).

However I haven't been able to get this to work with CLion - when trying to open the project CMakeLists.txt, it complains about missing Git submodule ("Could NOT find Git (missing: GIT_EXECUTABLE)") even though it's definitely in my path and I've never had an issue with git before. Also, even though I have set up Python and installed requirements.txt, I get "Some Python dependencies must be installed. Check above message for details." yet there is no output above to indicate what the problem is.

I have set PATH and IDF_PATH in CLion's configuration so that should be OK.

I'm pretty keen to see if I can get this working so if anyone has any ideas how to proceed I'd appreciate it. I'm currently using ESP IDF v3.2.2 however I can switch to v3.3-beta3 or even master pretty quickly.

[ESP_Angus]

Hi meowsqueak,

We have a few developers using CLion so I'll ask them to take a look at this thread.

Can you give any more details about your setup? What host OS? Do you get the same error if you try to build the IDF template project or an example? And anything else that might be useful.

[meowsqueak]

I realise that this may be a bit tricky to convey as I'm running into a few different problems.

One issue I have is that the docs here don't exist: https://docs.espressif.com/projects/esp ... index.html - so I'm using a mixture of 3.2 and master to get by.

In my original project, I was using a pyenv-managed virtualenv with python 3.7.3, but I didn't set the PYTHON cmake variable. I just set this now (to the path to my virtualenv's python binary) and that seems to have cleared up the Python issue, so thats OK.

However, I now see two problems. The first only occurs with my custom project that has some components brought in as simple git submodules (with their own CMakeLists.txt file at the root of each). This error is:

Code: Select all

/snap/clion/73/bin/cmake/linux/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G "CodeBlocks - Unix Makefiles" /home/meowsqueak/esp32/projects/myproj
-- Could NOT find Git (missing: GIT_EXECUTABLE) 
CMake Warning at /home/meowsqueak/esp32/esp-idf-v3.3-beta3/tools/cmake/git_submodules.cmake:4 (message):
  Git executable was not found.  Git submodule checks will not be executed.
  If you have any build issues at all, start by adding git executable to the
  PATH and rerun cmake to not see this warning again.
Call Stack (most recent call first):
  /home/meowsqueak/esp32/esp-idf-v3.3-beta3/tools/cmake/idf_functions.cmake:27 (include)
  /home/meowsqueak/esp32/esp-idf-v3.3-beta3/tools/cmake/project.cmake:5 (include)
  CMakeLists.txt:5 (include)

I've also tried with the hello_world example from 3.3-beta3 and I do not see this error there, although that project doesn't use git submodules so perhaps that is why?

The second error happens with both projects (I tried v3.2 here, hence the version number change):

Code: Select all

-- Checking Python dependencies...
Python requirements from /home/meowsqueak/esp32/esp-idf-v3.2/requirements.txt are satisfied.
-- Building empty aws_iot component due to configuration
-- Component names: soc log heap xtensa-debug-module app_trace freertos vfs newlib esp_ringbuf driver esp_event ethernet lwip tcpip_adapter partition_table app_update spi_flash mbedtls micro-ecc bootloader_support nvs_flash pthread smartconfig_ack wpa_supplicant esp32 cxx esp32-rotary-encoder esp32-smbus esp8266_wrapper i2c-master lis3mdl asio jsmn aws_iot bootloader bt coap console nghttp esp-tls esp_adc_cal tcp_transport esp_http_client esp_http_server esp_https_ota esptool_py expat wear_levelling sdmmc fatfs freemodbus idf_test json libsodium mdns mqtt openssl protobuf-c protocomm spiffs ulp wifi_provisioning main
-- Component paths: /home/meowsqueak/esp32/esp-idf-v3.2/components/soc;/home/meowsqueak/esp32/esp-idf-v3.2/components/log;/home/meowsqueak/esp32/esp-idf-v3.2/components/heap;/home/meowsqueak/esp32/esp-idf-v3.2/components/xtensa-debug-module;/home/meowsqueak/esp32/esp-idf-v3.2/components/app_trace;/home/meowsqueak/esp32/esp-idf-v3.2/components/freertos;/home/meowsqueak/esp32/esp-idf-v3.2/components/vfs;/home/meowsqueak/esp32/esp-idf-v3.2/components/newlib;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp_ringbuf;/home/meowsqueak/esp32/esp-idf-v3.2/components/driver;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp_event;/home/meowsqueak/esp32/esp-idf-v3.2/components/ethernet;/home/meowsqueak/esp32/esp-idf-v3.2/components/lwip;/home/meowsqueak/esp32/esp-idf-v3.2/components/tcpip_adapter;/home/meowsqueak/esp32/esp-idf-v3.2/components/partition_table;/home/meowsqueak/esp32/esp-idf-v3.2/components/app_update;/home/meowsqueak/esp32/esp-idf-v3.2/components/spi_flash;/home/meowsqueak/esp32/esp-idf-v3.2/components/mbedtls;/home/meowsqueak/esp32/esp-idf-v3.2/components/micro-ecc;/home/meowsqueak/esp32/esp-idf-v3.2/components/bootloader_support;/home/meowsqueak/esp32/esp-idf-v3.2/components/nvs_flash;/home/meowsqueak/esp32/esp-idf-v3.2/components/pthread;/home/meowsqueak/esp32/esp-idf-v3.2/components/smartconfig_ack;/home/meowsqueak/esp32/esp-idf-v3.2/components/wpa_supplicant;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp32;/home/meowsqueak/esp32/esp-idf-v3.2/components/cxx;/home/meowsqueak/esp32/projects/myproj/components/esp32-rotary-encoder;/home/meowsqueak/esp32/projects/myproj/components/esp32-smbus;/home/meowsqueak/esp32/projects/myproj/components/esp8266_wrapper;/home/meowsqueak/esp32/projects/myproj/components/i2c-master;/home/meowsqueak/esp32/projects/myproj/components/lis3mdl;/home/meowsqueak/esp32/esp-idf-v3.2/components/asio;/home/meowsqueak/esp32/esp-idf-v3.2/components/jsmn;/home/meowsqueak/esp32/esp-idf-v3.2/components/aws_iot;/home/meowsqueak/esp32/esp-idf-v3.2/components/bootloader;/home/meowsqueak/esp32/esp-idf-v3.2/components/bt;/home/meowsqueak/esp32/esp-idf-v3.2/components/coap;/home/meowsqueak/esp32/esp-idf-v3.2/components/console;/home/meowsqueak/esp32/esp-idf-v3.2/components/nghttp;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp-tls;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp_adc_cal;/home/meowsqueak/esp32/esp-idf-v3.2/components/tcp_transport;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp_http_client;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp_http_server;/home/meowsqueak/esp32/esp-idf-v3.2/components/esp_https_ota;/home/meowsqueak/esp32/esp-idf-v3.2/components/esptool_py;/home/meowsqueak/esp32/esp-idf-v3.2/components/expat;/home/meowsqueak/esp32/esp-idf-v3.2/components/wear_levelling;/home/meowsqueak/esp32/esp-idf-v3.2/components/sdmmc;/home/meowsqueak/esp32/esp-idf-v3.2/components/fatfs;/home/meowsqueak/esp32/esp-idf-v3.2/components/freemodbus;/home/meowsqueak/esp32/esp-idf-v3.2/components/idf_test;/home/meowsqueak/esp32/esp-idf-v3.2/components/json;/home/meowsqueak/esp32/esp-idf-v3.2/components/libsodium;/home/meowsqueak/esp32/esp-idf-v3.2/components/mdns;/home/meowsqueak/esp32/esp-idf-v3.2/components/mqtt;/home/meowsqueak/esp32/esp-idf-v3.2/components/openssl;/home/meowsqueak/esp32/esp-idf-v3.2/components/protobuf-c;/home/meowsqueak/esp32/esp-idf-v3.2/components/protocomm;/home/meowsqueak/esp32/esp-idf-v3.2/components/spiffs;/home/meowsqueak/esp32/esp-idf-v3.2/components/ulp;/home/meowsqueak/esp32/esp-idf-v3.2/components/wifi_provisioning;/home/meowsqueak/esp32/projects/myproj/main
CMake Error at /home/meowsqueak/esp32/esp-idf-v3.2/tools/cmake/idf_functions.cmake:162 (message):
  Internal error, toolchain has not been set correctly by project (or an
  invalid CMakeCache.txt file has been generated somehow)
Call Stack (most recent call first):
  /home/meowsqueak/esp32/esp-idf-v3.2/tools/cmake/project.cmake:117 (idf_verify_environment)
  CMakeLists.txt:6 (project)


-- Configuring incomplete, errors occurred!
See also "/home/meowsqueak/esp32/projects/myproj/cmake-build-debug/CMakeFiles/CMakeOutput.log".

The CMakeOutput.log file is lengthy but doesn't seem to contain any failures, errors or warnings.

My top-level CMakeLists.txt looks like this:

Code: Select all

cmake_minimum_required(VERSION 3.5)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(myproj)

I haven't explicitly set a toolchain in the project because I thought that setting IDF_PATH would do that - perhaps that's the problem here?

I'm building on Ubuntu Linux 16.04 with cmake 3.10.3, CLion 2019.1.

[ESP_cermak]

CLion works flawlessly for me with ESP-IDF projects. I'm on macOS, but also tried windows and linux in the past and saw no issues.
The only configuration needed was to setup IDF_PATH under Toolchain->CMake->Environment. Other configuration was to set the build directory to be in line with idf.py, but that's just for convenience of running idf.py in Terminal for flash and monitor targets.

As for the first issue (git submodule), I would suspect the environment (Is your system environment available in CLion? Do you have the checkbox under Toolchain->CMake->Environment "Include system environment variables" checked?)

The second issue might have something to do with CMake version, have you tried to upgrade to the latest? I'm with CMake >= 3.11 (works fine with system CMake as well as with CLion bundled)

I'm also currently using CLion 2019/1, with all IDF version from 3.2 to master, but usually with very simple projects/examples from IDF.

Edit: Doesn't seem to be a CMake version issue, have tested with 3.10 and works as expected on Mac.

[meowsqueak]

@ESP_cermak, thanks for your reply.

I thought I'd try setting it up "from scratch" on my Mac (Mojave, 10.14.5) which I already have ESP IDF v3.2 installed on. I have cmake 3.14.0 installed.

I did a fresh install of CLion 2019.1.4 and opened the CMakeLists.txt file from a freshly copied (from IDF v3.2) getting_started/hello_world example directory. I chose all the defaults for CLion's toolchain, etc (i.e. the default Apple toolchain, didn't choose anything ESP32-related).

I then opened the CMake Environment settings and set IDF_PATH to the path to my IDF v3.2 directory: /Users/meowsqueak/esp32/esp-idf-v3.2/. I am using pyenv on Mac too, so I set PYTHON to /Users/meowsqueak/.pyenv/versions/esp32-3.7.3/bin/python and made sure this virtualenv has the requirements.txt from IDF v3.2 installed (it already had).

I also had to add the path to the Xtensa toolchain to the start of my PATH variable (I have "Include system environment variables" ticked): PATH=/Users/meowsqueak/esp32/xtensa-esp32-elf-1.22.0-80-g6c4433a/bin:[original setting]

When I "Reload CMake Project" in CLion I get the following error:

Code: Select all

/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G "CodeBlocks - Unix Makefiles" /Users/meowsqueak/esp32/projects/hello_world.v3.2
-- Checking Python dependencies...
Python requirements from /Users/meowsqueak/esp32/esp-idf-v3.2/requirements.txt are satisfied.
-- Building empty aws_iot component due to configuration
-- Component names: soc log heap xtensa-debug-module app_trace freertos vfs newlib esp_ringbuf driver esp_event ethernet lwip tcpip_adapter partition_table app_update spi_flash mbedtls micro-ecc bootloader_support nvs_flash pthread smartconfig_ack wpa_supplicant esp32 cxx asio jsmn aws_iot bootloader bt coap console nghttp esp-tls esp_adc_cal tcp_transport esp_http_client esp_http_server esp_https_ota esptool_py expat wear_levelling sdmmc fatfs freemodbus idf_test json libsodium mdns mqtt openssl protobuf-c protocomm spiffs ulp wifi_provisioning main
-- Component paths: /Users/meowsqueak/esp32/esp-idf-v3.2/components/soc;/Users/meowsqueak/esp32/esp-idf-v3.2/components/log;/Users/meowsqueak/esp32/esp-idf-v3.2/components/heap;/Users/meowsqueak/esp32/esp-idf-v3.2/components/xtensa-debug-module;/Users/meowsqueak/esp32/esp-idf-v3.2/components/app_trace;/Users/meowsqueak/esp32/esp-idf-v3.2/components/freertos;/Users/meowsqueak/esp32/esp-idf-v3.2/components/vfs;/Users/meowsqueak/esp32/esp-idf-v3.2/components/newlib;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp_ringbuf;/Users/meowsqueak/esp32/esp-idf-v3.2/components/driver;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp_event;/Users/meowsqueak/esp32/esp-idf-v3.2/components/ethernet;/Users/meowsqueak/esp32/esp-idf-v3.2/components/lwip;/Users/meowsqueak/esp32/esp-idf-v3.2/components/tcpip_adapter;/Users/meowsqueak/esp32/esp-idf-v3.2/components/partition_table;/Users/meowsqueak/esp32/esp-idf-v3.2/components/app_update;/Users/meowsqueak/esp32/esp-idf-v3.2/components/spi_flash;/Users/meowsqueak/esp32/esp-idf-v3.2/components/mbedtls;/Users/meowsqueak/esp32/esp-idf-v3.2/components/micro-ecc;/Users/meowsqueak/esp32/esp-idf-v3.2/components/bootloader_support;/Users/meowsqueak/esp32/esp-idf-v3.2/components/nvs_flash;/Users/meowsqueak/esp32/esp-idf-v3.2/components/pthread;/Users/meowsqueak/esp32/esp-idf-v3.2/components/smartconfig_ack;/Users/meowsqueak/esp32/esp-idf-v3.2/components/wpa_supplicant;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp32;/Users/meowsqueak/esp32/esp-idf-v3.2/components/cxx;/Users/meowsqueak/esp32/esp-idf-v3.2/components/asio;/Users/meowsqueak/esp32/esp-idf-v3.2/components/jsmn;/Users/meowsqueak/esp32/esp-idf-v3.2/components/aws_iot;/Users/meowsqueak/esp32/esp-idf-v3.2/components/bootloader;/Users/meowsqueak/esp32/esp-idf-v3.2/components/bt;/Users/meowsqueak/esp32/esp-idf-v3.2/components/coap;/Users/meowsqueak/esp32/esp-idf-v3.2/components/console;/Users/meowsqueak/esp32/esp-idf-v3.2/components/nghttp;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp-tls;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp_adc_cal;/Users/meowsqueak/esp32/esp-idf-v3.2/components/tcp_transport;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp_http_client;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp_http_server;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esp_https_ota;/Users/meowsqueak/esp32/esp-idf-v3.2/components/esptool_py;/Users/meowsqueak/esp32/esp-idf-v3.2/components/expat;/Users/meowsqueak/esp32/esp-idf-v3.2/components/wear_levelling;/Users/meowsqueak/esp32/esp-idf-v3.2/components/sdmmc;/Users/meowsqueak/esp32/esp-idf-v3.2/components/fatfs;/Users/meowsqueak/esp32/esp-idf-v3.2/components/freemodbus;/Users/meowsqueak/esp32/esp-idf-v3.2/components/idf_test;/Users/meowsqueak/esp32/esp-idf-v3.2/components/json;/Users/meowsqueak/esp32/esp-idf-v3.2/components/libsodium;/Users/meowsqueak/esp32/esp-idf-v3.2/components/mdns;/Users/meowsqueak/esp32/esp-idf-v3.2/components/mqtt;/Users/meowsqueak/esp32/esp-idf-v3.2/components/openssl;/Users/meowsqueak/esp32/esp-idf-v3.2/components/protobuf-c;/Users/meowsqueak/esp32/esp-idf-v3.2/components/protocomm;/Users/meowsqueak/esp32/esp-idf-v3.2/components/spiffs;/Users/meowsqueak/esp32/esp-idf-v3.2/components/ulp;/Users/meowsqueak/esp32/esp-idf-v3.2/components/wifi_provisioning;/Users/meowsqueak/esp32/projects/hello_world.v3.2/main
CMake Error at /Users/meowsqueak/esp32/esp-idf-v3.2/tools/cmake/idf_functions.cmake:162 (message):
  Internal error, toolchain has not been set correctly by project (or an
  invalid CMakeCache.txt file has been generated somehow)
Call Stack (most recent call first):
  /Users/meowsqueak/esp32/esp-idf-v3.2/tools/cmake/project.cmake:117 (idf_verify_environment)
  CMakeLists.txt:6 (project)

I took a look in CMakeError.log:

Code: Select all

Checking whether the ASM compiler is GNU using "--version" did not match "(GNU assembler)|(GCC)|(Free Software Foundation)":
Apple LLVM version 10.0.1 (clang-1001.0.46.3)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Checking whether the ASM compiler is Clang using "--version" did not match "(clang version)":
Apple LLVM version 10.0.1 (clang-1001.0.46.3)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

So I'm seeing the same error on both Mac and Linux with an IDF Example (I'll worry about the git submodule issue once this is sorted).

Do I need to explicitly set up a toolchain in CLion for Xtensa?

Code: Select all

$ pip list
Package          Version
---------------- -------
appnope          0.1.0  
asn1crypto       0.24.0 
backcall         0.1.0  
cffi             1.12.3 
cryptography     2.6.1  
decorator        4.4.0  
future           0.17.1 
ipython          7.5.0  
ipython-genutils 0.2.0  
jedi             0.13.3 
numpy            1.16.4 
parso            0.4.0  
pexpect          4.7.0  
pickleshare      0.7.5  
pip              19.1.1 
prompt-toolkit   2.0.9  
ptyprocess       0.6.0  
pycparser        2.19   
Pygments         2.4.2  
pyserial         3.4    
setuptools       40.8.0 
six              1.12.0 
traitlets        4.3.2  
wcwidth          0.1.7  

[meowsqueak]

Hmm, a breakthrough!

I was using the Environment "table" under Build, Execution, Deployment > CMake. I clicked the little rectangle symbol on the right and was adding IDF_PATH and PYTHON variables there:

Capto_Capture 2019-07-03_15-03-48_.png (46.49 KiB) Viewed 17176 times

However it turns out that this doesn't work on its own for IDF_PATH. Because if I specify -DIDF_PATH=/Users/meowsqueak/esp32/esp-idf-v3.2 in the "CMake options" field it does work:

Capto_Capture 2019-07-03_15-05-28_.png (13.31 KiB) Viewed 17176 times

Once I do this, on my Mac at least (will test on Linux soon), the Hello World project now builds without errors.

But here's the weird thing - IDF_PATH has to be set in both the Environment table and the CMake options field. If it's only in one or the other, it doesn't build. If it's missing from Environment I get:

Code: Select all

CMake Error at CMakeLists.txt:5 (include):
  include could not find load file:

    /tools/cmake/project.cmake

- as if the IDF_PATH prefix isn't defined.

Any idea what's going on here?

[meowsqueak]

Looks like I'm getting the same behaviour on my Linux system - IDF_PATH must be set in both the command line (as -DIDF_PATH=...) and in the Environment table. If it's in neither or only one, I get an error from the IDF .cmake functions as per above.

I naively assume that anything in the Environment table gets converted into "-DVAR=..." options on the cmake command line, so I find this very surprising!

EDIT: I solved my "git submodules" problem - I had changed the system PATH variable to read "/path/to/my/xtensa/toolchain:$PATH" but apparently $PATH won't expand in this case, and that probably meant that almost everything was no longer in my path, including the git binary. I reverted this and added the path at the front of the system value and that must have fixed this issue.

[meowsqueak]

One other thing I've noticed and I can't work out if it's CLion or the CMake build system (that CLion is running frequently) that is doing it, but something is regularly touching (or rebuilding) sdkconfig, which causes the non-cmake system (which I run alongside CLion because I prefer the 'make flash monitor' workflow) to force a re-run of 'make menuconfig', which requires manual intervention, and subsequently a full rebuild. Unfortunately 'make oldconfig' or 'make silentoldconfig' doesn't bypass this like it would with a Linux kernel, but this wouldn't prevent the full rebuild anyway. Any idea why sdkconfig is being modified so often?

[ESP_renz]

Sharing my experience as well. I've had the same error as you did. Having ESP-IDF environment variables (IDF_PATH, addition of toolchain to PATH) in ~/.profile made import of CMake projects seamless for me.

[esp_agramakov]

It is Windows experience, but may help. Only that's needed for work well with CLion

Set Environments in linux way, it is nice to do with ~/.profile as Renz said:
IDF_PATH=C:\Users\dongr\esp\esp-idf
Path= %IDF_PATH%/tools; C:\msys32\usr\bin; C:\msys32\opt\xtensa-esp32-elf\bin; C:\Program Files\Git\cmd; C:\Python37\; C:\Python37\Scripts;


Update project's CMakeLists.txt with added compiller vars

Code: Select all

cmake_minimum_required(VERSION 3.5)

set(CMAKE_C_COMPILER xtensa-esp32-elf-gcc)
set(CMAKE_CXX_COMPILER xtensa-esp32-elf-g++)
set(CMAKE_ASM_COMPILER xtensa-esp32-elf-as)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(blink)

And do not forget to execute from IDF_PATH folder

Code: [Select all] [Expand/Collapse]

1. git submodule update --recursive --init

GeSHi © Codebox Plus Extension