Max .rodata 4MB? How to use 12MB in 16MB WROVER for fast .rodata? FATFS is slow.

[jcsbanks]

Instead of of using FATFS for large binaries (typically up to 1MB, but multiple), I'm embedding them in .rodata as they go with the application and can then be OTA udpated. However, I noticed a comment in the datasheet that suggested 4MB max for read only data.

Since I have an almost 2MB factory partition and only now have one OTA partition as OTA is only done whilst running factory, I have nearly 14MB available for app and data, of which I hoped to use 12MB for data.

It is really nice to get a pointer to the data I need as it performs so much faster than FATFS.

If they have to be 32 bit aligned to somehow get access through an instruction cache that is fine, most of them are zipped and copied to PSRAM for use.

[jcsbanks]

I just tried adding a 4MB file to existing 1.3MB and as suspected, rodata oversize:

`.flash.rodata' will not fit in region `drom0_0_seg'

[jcsbanks]

In component_wrapper.mk there is:

Code: Select all

OBJCOPY_EMBED_ARGS := --input-target binary --output-target elf32-xtensa-le --binary-architecture xtensa --rename-section .data=.rodata.embedded

Tried changing to to .text.flash and .flash.text but still overflows so think there is more to it.

If a small tweak can get some of these into .text with 32 bit alignment I think it would do what I want and still give a single atomic OTA partition.

iram0_2_seg is the one that overflows when I rename .data to .text, but looking through the ld files it seems this is flash and not RAM, and the reference manual has it as the correct address for the range I want to use.

[jcsbanks]

esp32.ld:

Code: Select all

  /* Even though the segment name is iram, it is actually mapped to flash
  */
  iram0_2_seg (RX) :                 org = 0x400D0018, len = 0x330000-0x18

Raising this 0x330000 allows larger binary to build without the overflow warning along with:

Code: Select all

OBJCOPY_EMBED_ARGS := --input-target binary --output-target elf32-xtensa-le --binary-architecture xtensa --rename-section .data=.text

in component_wrapper.mk

Bit puzzled by the 0x330000 value for the length of iram0_2_seg region which I expected would be about 0xB3E000‬?

I think I need to understand the MMU better.

[WiFive]

https://github.com/espressif/esp-idf/issues/1184

[jcsbanks]

Thanks. A 6.5MB build with some these tweaks did OTA update in about 50 seconds, but when rebooting it went back into factory and I did not have serial connected to see why.

I also tried tweaking OTA to update FATFS, but even then cannot get a method working to produce the image and even reading from FATFS is slow, and writing is terrible.

I would love a single large OTA with matching app and data with some way to access the contents. Even if I have to bank switch on 64KB boundaries with a home spun pseudo file system, it will feel less like transferring buckets of water through straws.

A large partition that boots from which I can work out addresses to use partition and mmap commands could work.

[jcsbanks]

Investigating whether it can go in a noload section so it doesn't overflow the sub 4MB flash mapping done by the second stage bootloader (for .text and .rodata it seems), but still be part of the partition and the OTA, then mmap it.

[jcsbanks]

Added a memory region (I gave it the address 0x40400000 and length 0x80000) and section that goes to it, but it doesn't get included in the .bin if it has NOLOAD tag.

If it does get included, the second stage bootloader is too clever and tries to load it and fails:

E (701) bootloader_flash: bootloader_mmap excess size 4004c8
E (705) esp_image: bootloader_mmap(0x414efc, 0x4004c8) failed
E (711) boot: OTA app partition slot 0 is not bootable[/code]

Also tried 4MB size. If I cannot get the linker to add the section to the binary but not allocate it in the bootloader, looks like the 2nd stage bootloader could be edited.

[jcsbanks]

Curveball is that the 2nd stage bootloader wants to load all the segments to do checksums/hashes of them. If you don't load all the segments, it cannot do that as it is written, and it looks like it does it on all the segments in the partition at once. It will boot the OTA app with the extra unloaded data in the partition if you bypass the checks, but obviously that is no use except for testing.