Could you, please, enable LTO (link time optimization) support?
I see in viewtopic.php?f=13&t=361&start=20 that you decided not to implement LTO because you did not see significant code size reduction.
I tried to enable LTO myself in my project and saw more than 4.5% Total image size reduction: 65 KB: from 1386574 to 1319486.
Without LTO my image cannot possibly fit into 4MB flash (when OTA and boot security is enabled).
I had to modify some IDF build process somewhat to make it compile and run
but I am worried that the system may not work correctly since I do not have the full testing procedure used to validate IDF SDK.
So, please, make LTO official.
Thank you
Please, enable LTO (link time optimization) support
Re: Please, enable LTO (link time optimization) support
Hi jhnlmn,jhnlmn wrote: ↑Tue Jun 29, 2021 6:44 amCould you, please, enable LTO (link time optimization) support?
I see in viewtopic.php?f=13&t=361&start=20 that you decided not to implement LTO because you did not see significant code size reduction.
We would really like to have LTO enabled as well. I think you may have misread that thread, though. These things were said back in 2019:
- We didn't see anything close to a 50% size reduction (this is true, it's not magic!) We got around 10% reduction in our experiments, but the firmware didn't always work! Although in those cases we also recompiled the Wi-Fi libraries with LTO enabled, the libraries on GitHub don't have this which is probably why you only see 4.5%.
-
- unfortunately this still true, Xtensa arch GCC does some things that don't currently work with LTO, and ESP-IDF makes heavy use of linker features for things like different memory regions. Our current blocker is "dangerous relocation" errors when linking the firmware, depending on the linking order. But most likely there will be other issues found after that one is solved.ESP-IDF relies heavily on particular linker behaviours so this is complex to implement without breaking existing functionality in some cases.
In the meantime, to reduce the binary size then you may find some of these techniques useful: https://docs.espressif.com/projects/esp ... /size.html
Re: Please, enable LTO (link time optimization) support
> ESP_Angus wrote:
> We got around 10% reduction in our experiments, but the firmware didn't always work!
Yes, when I built all .obj files with LTO I got as much as 5%, but that did not work at all.
Then I gradually restored some object files from non-LTO build until it started working:
in library libnewlib.a:
heap.c.obj
abort.c.obj
in library libesp_system.a:
panic_handler.c.obj
in library libsoc.a:
spi_flash_hal_iram.c.obj
esp32/rtc_clk.c.obj
rtc_time.c.obj
soc_memory_layout.c.obj
in library libspi_flash.a
- all files should be non-lto
in library libfreertos.a:
xtensa/port.c.obj
xtensa/xtensa_init.c.obj
xtensa/xtensa_intr.c.obj
xtensa/xtensa_overlay_os_hook.c.obj
xtensa/xtensa_vector_defaults.S.obj
croutine.c.obj
event_groups.c.obj
FreeRTOS-openocd.c.obj
list.c.obj
queue.c.obj
tasks.c.obj
Again, the .obj above should be non-lto while all others - LTO.
If I replace entire library like libsoc.a by non-LTO version, then size improvement disappears.
So, it is important to rebuild the library with some objects from LTO and others from non-LTO.
I guess that it may be possible to further reduce non-LTO code to only few routines within those modules above, but it will take time.
> Our current blocker is "dangerous relocation" errors when linking the firmware
I fixed "dangerous relocation" by adding -mlongcalls to ASM:
SET(CMAKE_ASM_FLAGS "-mlongcalls")
So, now my FW boots and runs, WiFi connects,
but I am worried that there may be some hidden intermittent failures which will take time to find.
I assume that you have some comprehensive test suite that you run to validate entire SDK,
so I was hoping that you will validate the LTO build.
Thank you
> We got around 10% reduction in our experiments, but the firmware didn't always work!
Yes, when I built all .obj files with LTO I got as much as 5%, but that did not work at all.
Then I gradually restored some object files from non-LTO build until it started working:
in library libnewlib.a:
heap.c.obj
abort.c.obj
in library libesp_system.a:
panic_handler.c.obj
in library libsoc.a:
spi_flash_hal_iram.c.obj
esp32/rtc_clk.c.obj
rtc_time.c.obj
soc_memory_layout.c.obj
in library libspi_flash.a
- all files should be non-lto
in library libfreertos.a:
xtensa/port.c.obj
xtensa/xtensa_init.c.obj
xtensa/xtensa_intr.c.obj
xtensa/xtensa_overlay_os_hook.c.obj
xtensa/xtensa_vector_defaults.S.obj
croutine.c.obj
event_groups.c.obj
FreeRTOS-openocd.c.obj
list.c.obj
queue.c.obj
tasks.c.obj
Again, the .obj above should be non-lto while all others - LTO.
If I replace entire library like libsoc.a by non-LTO version, then size improvement disappears.
So, it is important to rebuild the library with some objects from LTO and others from non-LTO.
I guess that it may be possible to further reduce non-LTO code to only few routines within those modules above, but it will take time.
> Our current blocker is "dangerous relocation" errors when linking the firmware
I fixed "dangerous relocation" by adding -mlongcalls to ASM:
SET(CMAKE_ASM_FLAGS "-mlongcalls")
So, now my FW boots and runs, WiFi connects,
but I am worried that there may be some hidden intermittent failures which will take time to find.
I assume that you have some comprehensive test suite that you run to validate entire SDK,
so I was hoping that you will validate the LTO build.
Thank you
Who is online
Users browsing this forum: No registered users and 196 guests