Ways to make binary smaller?

[kolban]

Howdy John,
Lets imagine the problem in the small. Imagine you have two C source files ... let us call one "idf.c" and the other "app.c". Now imagine inside idf.c you have the following:

Code: Select all

void myFunc() {
   printf("Hello world!\n");
}

now imagine that in app.c we want to call the function:

Code: Select all

void main() {
   myFunc();
}

when we compile idf.c we end up with idf.o. When we compile app.c we end up with app.o.
We can link idf.o to produce an executable. From that executable, we can ask it for the actual addresses of all the functions it exports ... this will include "myFunc". However, since we have produced an executable, this will be a relocated actual address. We can then flash the executable into the ESP32 and it will just "sit there".

Since we have the list of relocated actual addresses of all the functions from idf.o we can add those to a linker script. We can then link app.o with the linker script which will produce an executable for app. This works because there will be no unresolveds. The linker script satisfied the missing "myFunc". Since we knew the relocated address of myFunc, the code that is generated for the app executable will correctly call the correct address for myFunc on reference.

Scaling this up to the full ESP-IDF "just worked". We can compile the ESP-IDF, create an "app_main" entry point and ask the resulting ELF for ALL the symbols exported from it. We can then take all those symbols and add them to a linker script and compile/link a normal ESP-IDF application and it will just work.

[fly135]

Wow Neil! That's awesome that you did that and it worked. I have no shortage of flash, but the big OTA downloads have been troublesome. I may have to look into this some more.

John A

[hoainguyen265]

Howdy John,
Lets imagine the problem in the small. Imagine you have two C source files ... let us call one "idf.c" and the other "app.c". Now imagine inside idf.c you have the following:

Code: Select all

void myFunc() {
   printf("Hello world!\n");
}

now imagine that in app.c we want to call the function:

Code: Select all

void main() {
   myFunc();
}

when we compile idf.c we end up with idf.o. When we compile app.c we end up with app.o.
We can link idf.o to produce an executable. From that executable, we can ask it for the actual addresses of all the functions it exports ... this will include "myFunc". However, since we have produced an executable, this will be a relocated actual address. We can then flash the executable into the ESP32 and it will just "sit there".

Since we have the list of relocated actual addresses of all the functions from idf.o we can add those to a linker script. We can then link app.o with the linker script which will produce an executable for app. This works because there will be no unresolveds. The linker script satisfied the missing "myFunc". Since we knew the relocated address of myFunc, the code that is generated for the app executable will correctly call the correct address for myFunc on reference.

Scaling this up to the full ESP-IDF "just worked". We can compile the ESP-IDF, create an "app_main" entry point and ask the resulting ELF for ALL the symbols exported from it. We can then take all those symbols and add them to a linker script and compile/link a normal ESP-IDF application and it will just work.

I'm so interesting about your idea. Could you give a simple example?
Thanks.

[Laurent Louf]

Hello all,

Since this post is the top Google result for "reduce binary size esp32", I think it's worth adding a simple configuration change that can greatly reduce your code size. This configuration is in the sdkconfig in "Component config -> ESP32 specific " and the title is "Enable 'nano' formatting options for printf/scanf family" (CONFIG_NEWLIB_NANO_FORMAT). If you are using functions in the printf family, you might want to activate this. The only downside is you loose the support for 64 bits integers in printf functions, and features like positional arguments. So you can probably enable this option without loosing any functionality. In my code, activating this leads to a whopping decrease of 70kbytes in code size.

[Laurent Louf]

So I found myself playing with the configuration of mbedTLS also, and here are the results (the change of configurations are made one of the top of the previous) :

Code: Select all

Archive File DRAM .data & .bss   IRAM Flash code & rodata   Total
libmbedtls.a         92    264      0      94799    18102  113257 (base version)
libmbedtls.a         92    264      0      92646    17451  110453 (Disable PEM parsing : just use DER certificates)
libmbedtls.a         92    264      0      92601    17451  110408 (Disable support TLS 1.0)
libmbedtls.a         92    264      0      91118    17451  108925 (Disable support TLS 1.1)
libmbedtls.a         92    264      0      84624    17127  102107 (TLS protocole role client)

So if you can modify the format of your certificates, are sure your server is using a recent version of TLS (1.2) and your application will, from TLS point of view, only play the role of a client, you can save on the order of 10kbytes. If you dig still a bit further, you can probably disable the support for most of the Elliptic Curve Ciphers and keep only the one your server is using. But I would do that only with complete control over the server.

[kuhatovuk]

Thank you Laurent, newlib nano saved me 50kb.

[Fredckx]

Hi @kolban,

I find your proposed method very attractive! I am relatively new so I want to understand this a bit more before I try this.

1)
The most important, in the end you say: "The obvious downside of this is that (absent an ability to refresh the ESP-IDF) you are stuck with a constant ESP-IDF inside your device." Do you mean that the ESP-IDF part can NEVER be removed from the device?

2)
Do I understand it well that there is an OTA "region" that is large enough to hold my app, so in the order of 1MB?
So roughly half of the space is not used if I don't use OTA?

3)
What if I never have to do OTA? Can I double my app space by adding the OTA space to the app space? Is this reversible?

Thanks
Fred

EDIT: hmm, dived into menuconfig and see that there is a setting Partition table > Single factory app, no OTA. It is already selected in my project, so I'm afraid that the trick with sacrificing OTA space does not hold anymore.... it is already sacrificed ....

[Fredckx]

My app uses esp_wifi and mqtt_client for connection with the outside world. The total app is too large. I already implemented options described here: https://docs.espressif.com/projects/esp ... /size.html.

1) this is not enough
2) I need access to debugging information

I now think of using a less memory hungry to communicate with the outside world. I was thinking about the following:

- Remove WIFI and MQTT functionality from the app
- Add a less memory hungry method to connect with a peer ESP32 board
- Let the peer ESP32 board do communication with the (rest of ) the outside world over WIFI and MQTT

The "less memory hungry method" does not need to be necessarily wireless. I could connect the second board via UART/I2C.
Does anyone have ideas, or even an example of such an approach?

Thanks
Fred

[mikemoy]

The "less memory hungry method" does not need to be necessarily wireless. I could connect the second board via UART/I2C.
Does anyone have ideas, or even an example of such an approach?

Thanks
Fred

Fred, there are ESP modules up to 16MB all in the same size package, only costing $2.99. Are you aware of this?

[ESP_Sprite]

Fred, there are ESP modules up to 16MB all in the same size package, only costing $2.99. Are you aware of this?

Additionally, did you look at the partitioning scheme yet? Most default partitions only allocate 1MiB to the app, while most ESP modules have 4MiB at least; depending on what else you need (ota, spiffs, nvs, ...) you may very well get two to four times as much app space by simply building a custom partition table.