bootloader_flash: bootloader_mmap excess size

[PeterR]

The error reportis :

Code: Select all

I (123) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x3aefc8 (3862472) map
E (132) bootloader_flash: bootloader_mmap excess size 3aefc8
E (139) esp_image: bootloader_mmap(0x10020, 0x3aefc8) failed

I have a large (3Mbyte) website to serve.
SPIFFS is far too slow for the task viewtopic.php?f=2&t=12514.

I used lwip's ROM makefsdata instead. makefsdata embeds a file system as character arrays within the program image. 50x times faster, awesume!
However as I approach 4Mbytes I get the titled run time error. The error is actually raised around 3.8M which I do not understand.

A similar problem is documented here https://esp32.com/viewtopic.php?t=10394. The author is trying to achieve the same result as me and eventually gets my run time error report. The author also starts from the position that FATFS is too slow (I have not yet tried FATFS on ESP but also not sure how to 'build' and load FATFS).

There is much which I do not understand. The error report details suggest that the size limit is << 4MB but the window should be 4MB. Why do I fail before 4MB and so at what point of memory usage should I worry (that my app size is getting too large)? Clearly I cannot develop an application close to fundamental limits....

The main question/objective is however:
Q) How do I achieve a large (3MB+ webpages plus 2MB+ app) fast website on the ESP? SPIFFS is way too slow and if FATFS how do I create the image? I can segment if needed.

I had presumed makefsdata but (as discussed) I hit this run time size problems. How do I load/access a large data section in FLASH? I have PSRAM but will be using bluetooth so 2MB+ application size.

[WiFive]

Not sure why but segment size is limited to 3.1mb by bootloader code.

[PeterR]

Thanks and darn, 3.1MB - that's bad. That has me on the hairy arse end of disaster....
Do you have a reference?

What is the 16MB use/case then? You have 16MB but cannot use it? I don't understand the rules.

Also how are SPIFFS/FAT etc located and addressed? SPIFFS can do >4MB. I started out with my website in SPIFFS. Slow but all there. Also I note that I may use 11MB of code space (aside: what/how do you use the remaining 5MB?).
How then do I build a data binary and locate in code and/or use whatever mechanism is being used by FAT/SPIFFS file system (assuming that this addressing mechanism is not the reason behind SPIFFS/FAT performance suck) such that I can use the lean and mean lwip ROM filesystem.
I believe that the lwip file system has way less code than SPIFFS/FAT so I will eventually be faster but then I do not understand why SPIFFS is so slow.

EDIT (& without bluetooth and no wifi):

Code: Select all

 DRAM .data size:   11512 bytes
 DRAM .bss  size:   27856 bytes
Used static DRAM:   39368 bytes ( 141368 available, 21.8% used)
Used static IRAM:   55257 bytes (  75815 available, 42.2% used)
      Flash code:  606070 bytes
    Flash rodata: 2326768 bytes
Total image size:~2999607 bytes (.bin may be padded larger)

Just to clarify. Am I 100K away from the 3.1MB (you're fired) limit? I will try myself tommorrow, but darn, not seen that detail in any up front wiki/document.

[WiFive]

The 3.1mb is per segment not the whole app. It's actually a bug but even if you get around it you may hit the problem referenced in the other thread you linked.

The rules are supposed to be 4mb mapped to drom and 11.1mb mapped to irom.

Spiffs does not use mmapped reads. Fatfs only uses them on encrypted partitions and only mmaps them just in time, not the whole filesystem.

[PeterR]

Thanks.

Spiffs does not use mmapped reads. Fatfs only uses them on encrypted partitions and only mmaps them ]just in time, not the whole filesystem.

Not sure I understand the distinction and/or use and implications of using mmapped reads.

Clearly mapping

just in time

will give a performance hit.
What does a mmap read achieve and how do mmap read's differ in performance/resources vrs unmappend reads? Should I care for a static read once (http but need to serve quick) file system?

Looking at the SPIFFS generator then I can easily write an lwip binary creator. I just don't want to spend that time then to find I have SPIFFS performance!
EDIT: Because of some mmapped related issue.