ESP-S2 build time and size

[mfc@speleotrove.com]

I've used many computers over the years (from mainframes to 8-pin AVR ICs). I'm now starting a new project and would like to base it on the Espressif ESP-S2. For the prototype I have purchased a SparkFun ESP-S2 Thing Plus and the hardware seems to be exactly what I need.

However, the support software is problematic. I am using the latest Arduino 2.3.2 IDE. However, to compile and download the 'blinking LED' first example for the ESP32-S2 Thing Plus the build time is 17 seconds (second and subsequent times) and the resulting download is 234 kilobytes(!) -- which takes a similar amount of time to send to the device. With overheads, a simple iteration such as changing (say) one of the delay times takes the best part of a minute to get to the device. This is too slow.

On querying this in the Arduino forums another user reported:

"For AVR (board package version 1.8.3) the time was 1 second [after first compile] and the size of the (hex) file 2.6kB. Upgraded the ESP32 board package to 2.0.6 ... first compile took 70 seconds, second compile took 26 seconds; size of bin file 230kB."

This indicates that the slowness is not the IDE but the Espressif package.

In contrast, compiling and linking my PanGazer application for Windows is 62 seconds on the same PC. That's compiling it all from scratch: 36,000 lines of C and C++. Normally, of course, only one or two source files need to be compiled, which is less than 5 seconds per iteration.

So: is there a setting that will reduce the compile time (and download size) for Espressif ESP-S2 so that a usable time-per-iteration can be achieved?

Many thanks,

Mike Cowlishaw
https://speleotrove.com/mfc/

[ESP_Sprite]

Fwiw, for upload speed, you may be able to increase the baud rate and get more speed like that.

(Note that in general, the underlying ESP-IDF is not that slow. As a reference, on my laptop (a 3rd gen Thinkpad T14s with a 'power-efficient' aka slower than desktop CPUs i7-1280P) under Linux a full build of a sizable app (includes webserver, sntp client, graphics) takes 27 seconds, while a rebuild with one file changed takes 4 seconds. Flashing that project over the USB-serial-JTAG port of an ESP32-C6 takes 7.3 seconds, and this is for a binary that is near one megabyte in size.)

[mfc@speleotrove.com]

I'm currently using 115200 baud for the connection, which is what I thought was recommended.

I agree that the IDE (Arduino or ESP-EDF) is not the problem. It would seem that the problem is that far more is being compiled than is necessary just to toggle a LED. Given the size of the binary (nearly a quarter of a MB), it must be that many libraries are being compiled and included beyond those necessary.

The question remains: how do I prevent these unnecessary compilations/links and so get the iteration time down to something reasonable? I really do want to move from AVR and ARM-Cortex (both of which I have been using for 20+ years) to ESP for my new project, but it will waste too much of my time if I cannot fix this.

Thanks!

Mike

[ESP_Sprite]

I'm currently using 115200 baud for the connection, which is what I thought was recommended.

Note that that's fine for debug output, but for flashing you want to go as fast as you can, 1 to 4 Mbaud if possible.

I agree that the IDE (Arduino or ESP-EDF) is not the problem. It would seem that the problem is that far more is being compiled than is necessary just to toggle a LED. Given the size of the binary (nearly a quarter of a MB), it must be that many libraries are being compiled and included beyond those necessary.

The question remains: how do I prevent these unnecessary compilations/links and so get the iteration time down to something reasonable? I really do want to move from AVR and ARM-Cortex (both of which I have been using for 20+ years) to ESP for my new project, but it will waste too much of my time if I cannot fix this.

I just looked at a 'hello world' example, and I'm afraid that there is not much to trim. ESP-IDF is more of an 'OS' (providing a libc with filesystem support, OTA, FreeRTOS etc) than the bare-metal coding you'd do on an AVR or even ARM-Cortex, and that takes up a good 200K indeed. We could probably decrease that by a fair bit but there's really no incentive for that. Everyone is generally OK with this because that ESP chips are first and foremost used as WiFi/BT/Zigbee/... devices, and for that you need that entire OS stack (and more!) anyway. No one uses an ESP chip to simply blink an LED, so that use case is not optimized.

This means that if your program expands, you get more 'bang for the buck' as more of that 200K is objectively useful. However, in absolute terms, this still means that your iteration cycles are pretty long. I'm not exactly sure how to help that aside from mentioning the standard developer excuse: 'works for me', as in, I posted my iteration times and I generally find those acceptable. You could try changing up things to match that more: use Linux (if you're familiar with that), use ESP-IDF, make sure you're doing all of this on a recent processor, use USB-serial-JTAG on a newer chip.

Alternatively, perhaps some Arduino expert or user will pass by here; potentially there's an easier fix that I am not familiar with as I rarely use Arduino.

[mfc@speleotrove.com]

OK, thanks, that's useful background, etc. In fact I don't need WiFi so maybe I am indeed looking at the wrong chip. I'll take another look at current boards.

Mike

[joglz8]

I do not have an answer to your question, but I have noticed the same.

I just figured it was a compounding of multiple causes that affected me. I'm using older, not so powerful computers (mostly old Gateway lappies running Debian). The Arduino IDE is (my understanding) using an Arduino wrapper around ESP-IDF around libraries, etc., who knows how many layers deep?

I have thought that if I used ESP-IDF it would be faster but at this time I do not desire / need to learn another editor (as Master Foo would say).

[mfc@speleotrove.com]

Indeed. It seems the whole system is structured badly. I expect when I'm developing an application that if I make a one-line change to one module (e.g., to tweak some parameter) then only that module will be compiled and the new .exe will be running 5 seconds (or less) later. Why anyone things that 30+ seconds (excluding download time) is reasonable is beyond me. It is hugely wasteful of people-time.

Suppose there are 100,000 users [there are probably more] recompiling 100 times a day [another low estimate] and the compile takes 30 seconds longer than it needs to [as it does]. That's 83 thousand hours lost every day, because that time is not useful for doing anything else...

[ESP_Sprite]

Indeed. It seems the whole system is structured badly. I expect when I'm developing an application that if I make a one-line change to one module (e.g., to tweak some parameter) then only that module will be compiled and the new .exe will be running 5 seconds (or less) later. Why anyone things that 30+ seconds (excluding download time) is reasonable is beyond me. It is hugely wasteful of people-time.

It's not the compiling step that takes long. Arduino and ESP-IDF generally are smart enough to keep the old object files and not recompile the object files when the corresponding sources aren't touched. It's the linking step: taking all the object files and linking them together to resolve symbols etc. That unfortunately is not something that is trivial to make faster. There have been some attempts to fix that (effectively making the SDK an API with a fixed set of entry points so linking effectively happens only in the user program) but they tend to be finnicky and break easy. There's no silver bullet for it. Good news is that CPU power helps a lot; see my iteration times, and that's not on a super-powerful CPU either.

[mfc@speleotrove.com]

There seems to be a lot of setup, too. I see long pauses during "Detecting libraries used" and "Generating function prototypes". "Compiling Sketch" doesn't show until 17 seconds after clicking on 'Verify'. "Compiling sketch" takes about 5 seconds and then "Compiling core" (using previously compiled file) and "Linking everything together" together take about 8 seconds. Total about 30 seconds. [These times are for a different device .. see below.]

Separately, I've looked again at the many processor boards from SparkFun and, in the Thing Plus line, the Artemis Thing Plus is looking a good fit for me (I may well want to add Bluetooth connection later). Build time with the Arduino IDE is about half that for the ESP-S2 [these are the times I noted in the paragraph above], and the binary is also half the size (116kB).

Also, as a benchmark this board is supported by the Simplicity Studio IDE -- building their 'Blink bare metal' takes 3.2 seconds and the binary is a little over 10kB. Both a decimal order of magnitude better than the Arduino IDE numbers. Sadly I may have to stick with the Arduino IDE just because so many libraries are targeted for it...