cmake in esp-idf 4.0-beta verbose

[wevets]

A couple of days ago I downloaded esp-idf v4.0-beta as it seemed like a good time to do it, (I was having some problems with esp-idf 3.3 and would have had to re-install anyway) and things generally work with C programs. I'm running windows 10 cmd.exe as called out in the documentation. Cmd is lacking in some capabilities, such as the ability to send output from a build to both the screen and a file, which I cannot find help for anywhere I've looked on the net. This is made worse by the fact that idf.py puts out a lot of.... how shall I put this delicately...... crap, that may be useful in some contexts but not to me now. Especially irksome is the output of the full include path. This is a problem because cmd doesn't let one scroll very far up into previous output, and so much rolls by so fast it can't be seen and can never be recovered. This is tempered by the fact that cmake on cmd.exe seems to be much faster than make in msys32, which is nice, but problematic when trying to follow things.
Is there anyway to stop this.... crap from coming out? I can't find it and it doesn't seem to documented anywhere I can find it reasonably quickly.
One nice thing however is that idf.py gives good info on how a build is progressing with a counter of build steps and the number of build steps total in the form [833/844], for instance.
But that's a problem. After building a couple of C programs that run on the main processor, I tried to build sample C and ULP assembly programs that came with esp-idf 4.0. As I write this, what appears on my cmd window, after the listing of a large pile of what looks like path strings is the following:

Code: [Select all] [Expand/Collapse]

1. -- The ASM compiler identification is GNU
2. -- Found assembler: C:/Users/Steve/.espressif/tools/esp32ulp-elf/2.28.51.20170517/esp32ulp-elf-binutils/bin/esp32ulp-elf-as.exe
3. -- The C compiler identification is GNU 8.2.0
4. -- Check for working C compiler: C:/Users/Steve/.espressif/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe
5. -- Check for working C compiler: C:/Users/Steve/.espressif/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe -- works
6. -- Detecting C compiler ABI info
7. -- Detecting C compiler ABI info - done
8. -- Detecting C compile features
9. -- Detecting C compile features - done
10. -- Building ULP app ulp_main
11. -- ULP assembler version: 2.28.51.20170517
12. -- Configuring done
13. -- Generating done
14. -- Build files have been written to: C:/Users/Steve/Desktop/esp-idf/examples/system/ulp/build/esp-idf/main/ulp_main
15.  
16. C:\Users\Steve\Desktop\esp-idf\examples\system\ulp\build\esp-idf\main\ulp_main>C:\Users\Steve\.espressif\tools\cmake\3.13.4\bin\cmake.exe -E touch C:/Users/Steve/Desktop/esp-idf/examples/system/ulp/build/esp-idf/main/ulp_main-prefix/src/ulp_main-stamp/ulp_main-configure   || exit /b
17. [833/844] Performing build step for 'ulp_main'

GeSHi © Codebox Plus Extension

The build is not progressing beyond step 833.

I'm trying to build a sample ULP program that came with esp-idf 4.0, pulse_cnt.S and it's C driver The 4.0 documentation says that the assembler build tools are packaged with the idf for 4.0.

So my questions would be:

1. Is the ULP not yet supported in esp-idf 4.0 yet?

2. Is there a way to reduce the output from a build, at least not put out the include path and other "stuff."

3. If I'm on Windows, is there a more fully-featured environment than cmd.exe that I can use with 4.0?

Thanks. Any help is appreciated. If I can't get this to work, I'm going to have to retreat back to 3.3.

wevets

[ESP_Angus]

The build is not progressing beyond step 833.

Do you mean that it hangs entirely? If you Ctrl-C out of it, do you get any additional error that gives a clue as to what was happening? If you look in Task Manager, do you see any process(es) using CPU?

If you "cd build" and then run "ninja -v" to build the project, can you see what command is running when the build stalls? (Warning that the output from this command is extremely verbose.)

1. Is the ULP not yet supported in esp-idf 4.0 yet?

Supported.

2. Is there a way to reduce the output from a build, at least not put out the include path and other "stuff."

Not exactly, but we can take this as a feature request. Most of the output only appears on an initial build when CMake is invoked. Subsequent builds only invoke "ninja" - unless the config or some high level build files are changed - and there will be a lot less output.

3. If I'm on Windows, is there a more fully-featured environment than cmd.exe that I can use with 4.0?

You may find this useful: https://github.com/espressif/idf-eclipse-plugin

We are also working on official VSCode support but this is not available yet.

[wevets]

As you suggested, I set up virgin build conditions for ulp_example_main.c, in the esp-idf\examples\system\ulp directory hanging off \desktop, as installed by the esp-idf v4.0 install file, by deleting the build directory, and started a build. (The file ulp_pulse_cnt.S file is in the ulp directory hanging off the main directory). I'm guessing you know all this, but I want to be crystal clear.

In the cmd shell for esp-idf, as created in the 4.0 install, I ran "idf.py menuconfig", which ran normally, then "idf.py -p COM 3 flash".
This seemed to run normally at about 99% CPU utilization, as reported by Task Manager with appropriate allocation of CPU as reported in Resource Monitor. When the "step count" got to [833/844] there was a hard hang, with the CPU utilization going, over the course of a second or two (perhaps due to reporting lag) to the normal quiescent 6~7% CPU utilization.
Immediately on the hang, an attached file calling itself "esp32ulp_mapgen.py" popped up in a notepad window. I've attached it, but had to change the name of the file by adding a ".txt" file termination because the forum's "add attachments" mechanism rejected a file with a ".py" file extension. This file was written to another directory, ...\desktop\esp-idf\components\ulp. (Weird, but OK.)
When I <ctrl-c>'ed out of the build hang, there was nothing but a new prompt on the cmd line.

I hope this helps diagnosing what went on. I'm reluctant to use the elipse plug-in you pointed me to, because if there's an issue with the underlying build mechanism, running the builds from eclipse is unlikely to go any better, I thinking.

When, as you suggested, I cd'ed to the build directory and ran "ninja -v" I got some new output on the cmd window, but can't recover it all because the cmd windows does not allow much back scroll. (That's why It would be nice to have another build environment. Msys32 was great for this.) Here are the last 40-or-so lines that popped up after running "ninja -v":

Code: [Select all] [Expand/Collapse]

1. ../main/ulp_example_main.c:20:2: error: stray '\' in program
2.  \\#include "ulp_main.h"
3.   ^
4. ../main/ulp_example_main.c:20:3: error: stray '#' in program
5.  \\#include "ulp_main.h"
6.    ^
7. ../main/ulp_example_main.c:20:12: error: expected '=', ',', ';', 'asm' or '__attribute__' before string constant
8.  \\#include "ulp_main.h"
9.             ^~~~~~~~~~~~
10. ../main/ulp_example_main.c: In function 'init_ulp_program':
11. ../main/ulp_example_main.c:46:40: error: 'ulp_main_bin_start' undeclared (first use in this function); did you mean 'ulp_main_bin_end'?
12.      esp_err_t err = ulp_load_binary(0, ulp_main_bin_start,
13.                                         ^~~~~~~~~~~~~~~~~~
14.                                         ulp_main_bin_end
15. ../main/ulp_example_main.c:46:40: note: each undeclared identifier is reported only once for each function it appears in
16. ../main/ulp_example_main.c:63:5: error: 'ulp_debounce_counter' undeclared (first use in this function); did you mean 'update_pulse_count'?
17.      ulp_debounce_counter = 3;
18.      ^~~~~~~~~~~~~~~~~~~~
19.      update_pulse_count
20. ../main/ulp_example_main.c:64:5: error: 'ulp_debounce_max_count' undeclared (first use in this function)
21.      ulp_debounce_max_count = 3;
22.      ^~~~~~~~~~~~~~~~~~~~~~
23. ../main/ulp_example_main.c:65:5: error: 'ulp_next_edge' undeclared (first use in this function)
24.      ulp_next_edge = 0;
25.      ^~~~~~~~~~~~~
26. ../main/ulp_example_main.c:66:5: error: 'ulp_io_number' undeclared (first use in this function); did you mean 'gpio_num'?
27.      ulp_io_number = rtc_gpio_desc[gpio_num].rtc_num; /* map from GPIO# to RTC_IO# */
28.      ^~~~~~~~~~~~~
29.      gpio_num
30. ../main/ulp_example_main.c:67:5: error: 'ulp_edge_count_to_wake_up' undeclared (first use in this function)
31.      ulp_edge_count_to_wake_up = 10;
32.      ^~~~~~~~~~~~~~~~~~~~~~~~~
33. ../main/ulp_example_main.c:90:20: error: 'ulp_entry' undeclared (first use in this function)
34.      err = ulp_run(&ulp_entry - RTC_SLOW_MEM);
35.                     ^~~~~~~~~
36. ../main/ulp_example_main.c: In function 'update_pulse_count':
37. ../main/ulp_example_main.c:108:38: error: 'ulp_edge_count' undeclared (first use in this function); did you mean 'pulse_count'?
38.      uint32_t pulse_count_from_ulp = (ulp_edge_count & UINT16_MAX) / 2;
39.                                       ^~~~~~~~~~~~~~
40.                                       pulse_count
41. ninja: build stopped: subcommand failed.

GeSHi © Codebox Plus Extension

I hope I've answered your questions fully. If you would like anything else from me or want me to run further tests, just ask.

wevets

[wevets]

When I submitted my last reply, I'm not sure the file I tried to attach went with the submission. Here's another attempt. If you get two copies of the same file, ignore one.

wevets

[ESP_Angus]

Hi wevets,

Thanks for the detailed description of the bug. This is the actual bug:

Immediately on the hang, an attached file calling itself "esp32ulp_mapgen.py" popped up in a notepad window.

The build system is executing a .py file with the default system interpreter (which happens to be a Notepad editor) rather than running it in the Python interpreter. When Notepad closes, the build is incomplete because the script hasn't done the things that it was supposed to do.

(This bug is only triggered on Windows and only when the default Windows behaviour for a .py file is not to run it in the same Python interpreter used for IDF, which is why it hasn't been noticed until now.)

You should be able to fix this by adding one line in the file components/ulp/project_include.cmake as shown:

Code: Select all

diff --git a/components/ulp/project_include.cmake b/components/ulp/project_include.cmake
index 95996b32f0..39d2e594c5 100644
--- a/components/ulp/project_include.cmake
+++ b/components/ulp/project_include.cmake
@@ -45,6 +45,7 @@ function(ulp_embed_binary app_name s_sources exp_dep_srcs)
                         -DCOMPONENT_INCLUDES=$<TARGET_PROPERTY:${COMPONENT_TARGET},INTERFACE_INCLUDE_DIRECTORIES>
                         -DIDF_PATH=${idf_path}
                         -DSDKCONFIG=${SDKCONFIG_HEADER}
+                        -DPYTHON=${PYTHON}
             BUILD_COMMAND ${CMAKE_COMMAND} --build ${CMAKE_CURRENT_BINARY_DIR}/${app_name} --target build
             BUILD_BYPRODUCTS ${ulp_artifacts} ${ulp_artifacts_extras} ${ulp_ps_sources}
                             ${CMAKE_CURRENT_BINARY_DIR}/${app_name}/${app_name}

EDIT: Updated suggested change.

This fix will be available in ESP-IDF before the v4.0 final release (it may be too late for beta 2, unfortunately).

[wevets]

Hi.
I added that one line to .../components/ulp/project_include.cmake as you suggested. Things are still not working completely. Again, things work with C-only files. But this is a ULP thing.

The build seems to finish without error, but then I get (the last 5 lines of screen output are shown)

Code: [Select all] [Expand/Collapse]

1. C:/Users/Steve/Desktop/esp-idf/components/soc C:/Users/Steve/Desktop/esp-idf/components/spi_flash C:/Users/Steve/Desktop/esp-idf/components/xtensa
2. -- Configuring done
3. -- Generating done
4. -- Build files have been written to: C:/Users/Steve/Desktop/esp-idf/examples/system/ulp/build/bootloader
5. ninja: build stopped: subcommand failed.
6. ninja failed with exit code 1

GeSHi © Codebox Plus Extension

And then a cmd prompt appears.

wevets

[wevets]

Is there any reply to my last post?
I've gone back v3.3 in the meantime.

wevets