Splitting ESP32 app over multiple partitions

[themadsens]

I am contemplating using the ESP32 for a Lora appliance ie. with very low network bandwidth. However it would be nice to still be able to update the main application part over the network (OTA).

For this to work I would like to compile/link the lower layers of the SW into a separate partition and then link to a (stub) library representing this partition from the main application.

I have been unable to find any references for this use case. Has anyone done something similar? or have any pointers to relevant material?

[themadsens]

Answering myself ..

Found the solution:

Do it the same way entrypoints from the internal ROM. Ie. w. PROVIDE() in .ld files. Fi. looking at tzset:

> ag tzset

esp-idf/components/newlib/include/time.h
82:_VOID _EXFUN(tzset, (_VOID));
83:_VOID _EXFUN(_tzset_r, (struct _reent *));

esp-idf/components/esp32/ld/esp32.spiram.rom-functions-iram.ld
49: *lib_a-tzset_r.o(.literal .text .literal.* .text.*)
50: *lib_a-tzset.o(.literal .text .literal.* .text.*)

esp-idf/components/esp32/ld/esp32.rom.spiram_incompatible_fns.ld
153:PROVIDE ( tzset = 0x40001a1c );
154:PROVIDE ( _tzset_r = 0x40001a28 );

[ESP_Angus]

The themadsens,

Sorry noone replied to this when you first posted. Another longer thread discussing the same idea is here:
viewtopic.php?f=13&t=5950

The short answer is that this is technically possible but there is no easy or supported way to do this.

A few details:

• The addresses in software PROVIDE statements like the ones shown above will have to be for linked symbol addresses, not unlinked addresses (the ROM binary has been linked, already). So some "system binary" linking step would be required which includes all of the public API calls used by the "user" firmware, and then from this binary the PROVIDE statements can be generated to link the "user" binary against it.
• This "system" linking step can't include all of ESP-IDF, as the binary would be enormous (ESP-IDF apps currently only link in the parts of ESP-IDF which are referenced by the final application.)
• However, if the "system" linking step only includes parts of ESP-IDF which are currently referenced by the app firmware, an OTA update which called a new (previously uncalled) ESP-IDF API function would require a new "system" binary to be distributed as well, or it would have parts of the "system" binary linked to it somehow. Or there would need to be a mechanism to specify public APIs calls that are not called now but may be called in a future update, to pre-emptively link these into the "system".
• Any change in project sdkconfig would also require a new "system" binary to be built and linked.

[themadsens]

Just saw this .. Thanks for your reply

We ended up writing our APP layer in Lua which lives in SPIFFS files. This way we can achieve much more granular OTA updates and smaller sizes (besides writing highlevel stuff in Lua is just so much nicer

Anyway regarding your bullets:
The "system" partition would be the normal idf project with a normal app_main() calling into the "app" partition on a hardwired address .
The main.c file would then contain "noop" references to all the entrypoint required by the "app" partition.
This could be done as a three step process:

1. Link the "app" partition and collect the errors/undefineds
2. Use this list to extract addresses from the "system" partion, already built with the normal build system
3. Link the "app" partition again with the generated .ld file

/Flemming

[ESP_Angus]

Hi Flemming,

We ended up writing our APP layer in Lua which lives in SPIFFS files. This way we can achieve much more granular OTA updates and smaller sizes (besides writing highlevel stuff in Lua is just so much nicer

Very nice solution! I should have thought to mention using a high level interpreted language like this when
I replied, as well.

Angus