Memory Layout BUG Overlap on BOOT

[Baldhead]

Hi,

I am trying to use external spiram in my application as much as i can( wifi, lwip, https server, one socket uses ~ 40kB RAM with SSL, and so on), because i need internal ram memory for other purposes.

The i2s hardware module has a bug that i cannot use dma transfers with sizes less than 32 bits, soh i need to copy a "big" 8 bit buffer in slices to send to a lcd display.
So, if i use two buffers with 3840 pixels(2 bytes per pixel -> 7680 bytes), i need 7680*2 bytes = 15360 bytes only for graphics library write in then, taking out other bytes that lib uses for scratch.
I need more 2 dma buffers with 32 bits. So i need 7680*4 =30720 bytes per buffer. 2 buffers = 30720*2 = 61440 bytes only in two dma buffers.
61440 bytes in two dma buffers + 15360 bytes in two graphics lib buffers = 76800 bytes.

I would like to work with at least twice that 153600 bytes.
Before adding other features to the project i could do this.

When i enable External SPIRAM on idf.py menuconfig:
Component config → mbedTLS → Memory allocation strategy → External SPIRAM.

The compilation was successful, then i flash the program inside esp32, but when i boot the esp32 the program doesnt work.
Then i execute the command "idf.py -p COM7 monitor", and the following error appears:
memory_layout: SOC_RESERVE_MEMORY_REGION region range 0x3f800000 - 0x40000000 overlaps with 0x3ffae000 - 0x3ffae6e0


Summarizing the subject:

* How to transfer some kilobytes from heap dynamic internal memory to statically allocated internal memory on esp-idf ?

* Can i put these items in the spi ram ?
if i can. What do i need to do ?
items: wifi, lwip, https server, all needed sockets that uses ~ 40kB RAM with SSL.

* How to solve this problem ?
memory_layout: SOC_RESERVE_MEMORY_REGION region range 0x3f800000 - 0x40000000 overlaps with 0x3ffae000 - 0x3ffae6e0


Notes:
* esp-idf version: v4.2-dev-1235-g71dc5eb27
* ESP32-DevKitC-VB
* win10 pro

Thank's.

[Baldhead]

In idf.py menuconfig i changed back to internal ram:
Component config → mbedTLS → Memory allocation strategy → internal ram.

Even after switching back to internal memory, the boot error remained:
memory_layout: SOC_RESERVE_MEMORY_REGION region range 0x3f800000 - 0x40000000 overlaps with 0x3ffae000 - 0x3ffae6e0

I tried to delete the build folder, recompile, flash, but unsuccessfully with the same error on boot.

I needed to delete build folder and all sdkconfig files, execute "idf.py menuconfig" and edit all needed fields, recompile, flash and the program worked again.

[Baldhead]

No one ???

[WiFive]

Set CONFIG_SPIRAM_SIZE to auto

[Baldhead]

Now worked.

I don't remember if CONFIG_SPIRAM_SIZE to auto was configured before.

But esp-idf is still using the internal memory instead of the external one for wifi, lwip, httpd and probably on https sockets(~40 Kbytes each socket).

Any suggestion ?

OBS: i checked "Try to allocate memories of Wi-Fi and LWIP in SPIRAM firstly. If failed, allocate internal memory".

Thank's.

[Baldhead]

?????????????????

[ESP_Angus]

Hi Baldhead,

For sake of completeness, WiFive has submitted a fix for the underlying PSRAM issue here, although not yet merged. Thanks WiFive.

To make sure I understand the residual problem: You have enabled the config item ""Try to allocate memories of Wi-Fi and LWIP in SPIRAM firstly. If failed, allocate internal memory" but despite this:

esp-idf is still using the internal memory instead of the external one for wifi, lwip, httpd and probably on https sockets(~40 Kbytes each socket).

Will try to provide some context about what can be changed:

• For WiFi (and LWIP to some extent), there is some memory which has to be allocated internally as it's accessed via DMA. So there are limits to what the config settings can achieve.
• Regarding https sockets, have you set the Component config → mbedTLS → Memory allocation strategy setting back to External? This is the setting that will be used to determine TLS buffer allocation.
• For HTTPD and for other non-specific buffers which may be allocated by other code, you can tune the following value: Maximum malloc() size, in bytes, to always put in internal memory. The default is 16KB, suggest trying 2KB or 4KB. Setting too low may cause severe performance degradation.

My colleague Mahavir also recently wrote up a blog post about optimising memory usage in ESP-IDF, that you may find useful.

[bobolink]

And the takeaway from this exchange is that there is such a thing as the “ESP Journal”

https://medium.com/the-esp-journal

[ESP_Angus]

Good point bobolink. Will find some places to promote this more.