Preview release: CMake-based build system for ESP-IDF

[ESP_Angus]

Since the beginning of ESP-IDF we've had a build system based on GNU Make. It's been effective and reliable for most users.

However, there have been two long term shortcomings:

• Windows users must use the Unix-like MSYS2 environment, which provides full functionality but is un-Windows-like and can be slow.
• Complex GNU Make build systems are difficult to integrate into IDEs. We support Eclipse, but people often have trouble getting their ESP-IDF Eclipse projects to work reliably. Third-party IDE support is also a mixed bag.

To overcome these shortcomings, after significant research and consideration we have developed a new build system based on CMake. The new build system aims to provide a better experience for both Windows users and IDE users, plus some other nice benefits for everyone (such as faster incremental builds in many cases).

The new CMake-based build system is now available as a preview pre-release. Not all build system features are fully supported, and we expect some bugs. Please report bugs via Github or here on the forums (please specify that you are using CMake when reporting bugs). We appreciate your patience while we iron things out.

If there's functionality that you can currently implement in the GNU Make based system which seems to be complex or impossible in the CMake-based system, please also let us know the details and we'll work through this support with you.

We're calling the new build system "CMake-based" because you don't need to run cmake directly if you don't want to. For command line users, a new Python-based tool called "idf.py" provides a wrapper around all other tools. "idf.py menuconfig" and "idf.py build flash monitor" is everything that you need to run.

Getting Started

Documentation is here:
https://docs.espressif.com/projects/esp ... index.html

(At the above link, follow the Getting Started guide for your platform to configure the CMake-based build system. There is also a reference page with details of the new build system.)

The preview branch can be found here:
https://github.com/espressif/esp-idf/tree/feature/cmake

EDIT: Since 2018-09-08 and commit 4eeed31, experimental CMake support is available in the master branch of ESP-IDF.)

The preview IDF master branch contains both the original GNU Make-based build system and the CMake-based build system, side by side.

Windows users: note the new setup instructions and ESP-IDF Tools Installer. MSYS2 is no longer required to use CMake. 64-bit Windows is required, please let us know if this is a showstopper for you.

The Future

The CMake preview branch (feature/cmake) will be updated regularly to track the master branch. We don't plan to merge CMake support into the master branch until after the v3.1 release candidate.

Experimental CMake support is available in the ESP-IDF V3.1 release and the master branch of ESP-IDF (for V3.2 and beyond).

CMake-based build system will be the default build system in ESP-IDF V4.0.

The existing GNU Make build system will not be going away soon. It will continue to be supported for all ESP-IDF v3.x releases. Depending on how the CMake Preview progresses, we may look into exclusively supporting the CMake-based system for ESP-IDF v4.0 v5.0 or in a later version.

Still to come

The following features from the GNU Make based build system are still pending. They will be added to the CMake-based build system branch over the coming weeks:

• IDE setup instructions (generic cmake instructions may work for many IDEs, but there will probably be some rough edges)
• Support for Secure Boot & Flash Encryption
• Support for the ULP processor toolchain
• Chinese translations for the CMake-based Getting Started documentation (currently English only)

[mikemoy]

Getting Started

Documentation is here:
https://docs.espressif.com/projects/esp ... index.html

I don't know if it's me or what, but those instructions are clear as mud to me. IMHO, you should have a single step by step install without all these links jumping around to areas for the old way. Just make it clear from start to building hello world.

[ESP_Angus]

I don't know if it's me or what, but those instructions are clear as mud to me. IMHO, you should have a single step by step install without all these links jumping around to areas for the old way. Just make it clear from start to building hello world.

Thanks for noting this, Mike. Someone else made a similar observation via Github. We'll try and figure out the best way to reorganise the docs.

The "cmake-specific" docs at the link above shouldn't themselves "jump" back to the other build systems, they're intended to be a complete parallel set of pages. Any links from inside one "cmake-based" Getting Started document should go to other cmake-based documents. But it seems like maybe there are some bad links, or perhaps it's unclear because there are two sets of pages in the "table of contents" sidebar which have similar/same names...? Will rethink this.

[mikemoy]

ok cool. So do we just keep checking that link to see if things changed, of is there a better way to know when an update is posted so we can try it ?

[ESP_Angus]

Will reply here and on the Github issue when docs are updated. You can subscribe to either/both of these if you'd like to be emailed when this happens.

[Deouss]

So how does that CMake work?
Let's say I already set up a Toolchain and ESP-IDF from GitHub
I program in C# too so maybe I could simplify stuff a bit.

I don't quite understand - idf.py must have a CMake file in the project folder to prepare the project just like Make does?
And how does that CMake file work - is it similar to Make or we should modify it ourselves?

I want to try it

[ESP_Angus]

Will reply here and on the Github issue when docs are updated. You can subscribe to either/both of these if you'd like to be emailed when this happens.

The docs have been updated, so now the CMake-based Getting Started follows the same flow as the old build system guide. The old "cmake-only" docs pages are gone, you can find the new docs at https://docs.espressif.com/projects/esp ... ure-cmake/ .

[ESP_Angus]

So how does that CMake work?
Let's say I already set up a Toolchain and ESP-IDF from GitHub

If you're using an existing ESP-IDF configuration then you'll need to install some new prerequisites (both build systems can exist side by side). If you follow the link to the documentation above, then go to Getting Started for your platform and you'll see what is required.

I don't quite understand - idf.py must have a CMake file in the project folder to prepare the project just like Make does?
And how does that CMake file work - is it similar to Make or we should modify it ourselves?

That's right. A CMake-based project has a file called "CMakeLists.txt" which is the equivalent of a Make-based project's "Makefile".

There is a new automatic conversion tool to convert simple projects from the GNU Make based build system to CMake. It's documented here: https://docs.espressif.com/projects/esp ... rsion-tool

(I recommend following the Getting Started guide and building one of the examples before trying to use the automatic conversion tool.)

[Deouss]

I setup required CMake environment and toolchain and I am at step of compiling hello-world but..
I am getting these errors:

cannot open source file "stddef.h" (dependency of "sys/cdefs.h")
cannot open source file "stddef.h" (dependency of "sys/reent.h")
cannot open source file "stddef.h" (dependency of "sys/types.h")
cannot open source file "stdbool.h" (dependency of "esp_spi_flash.h")
cannot open source file "stddef.h"
cannot open source file "stdarg.h"

Note: I have all the includepath references. Those header files seem not to exist in current edf

Any clues, advices?

[Deouss]

UPDATE: I resolved the errors. Paths for needed headers were actually in Program Files folder - for some reason.

All compiled after running idf.py build. Compilation under VS Code
I hope I did it right

However it took some longer time to build.
Not sure because I was using VM and nothing was connected through USB.
So confirming that CMake works and I don't know if I can make it compile faster.
Msys32 uses some multi core features - -j8 option or something?

Any help would be greatfull