Confusing Heap availability in ESP-IDF (dual-core, freertos, no Bluetooth, no wifi)

[hassan789]

tl;dr -- you can skip down to summary

Hi All. I am trying to do some image processing on the ESP32 chip, however, I am unable to use too much heap ram. I would like to use 300kB+ if possible (more the better) in dual-core mode. I am NOT using bluetooth, and I am not using WiFi. I am using the latest master code from github: 4048db35b6a1ac4216150480dcacdc4230f0baa2

Running the the example "hello_world_main.c" code, the maximum heap I can use is 167kB. If I try to use any more heap than that, the chip crashes. You can recreate this by simply adding the following line of code, and assigning 1 element in app_main():

Code: Select all

volatile uint8_t buff[168*1024] = {0};	//168kB on heap, code will crash!!??

While running 167kB or less, it runs good, with the following info:

Code: Select all

I (175) cpu_start: App cpu up.
I (196) heap_init: Initializing. RAM available for dynamic allocation:
I (203) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (209) heap_init: At 3FFDC5A0 len 00003A60 (14 KiB): DRAM
I (215) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (221) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (228) heap_init: At 40088484 len 00017B7C (94 KiB): IRAM

Also, running "make size" results in the following:

Code: Select all

$ make size
Total sizes:
 DRAM .data size:    8580 bytes
 DRAM .bss  size:  173080 bytes
Used static DRAM:  181660 bytes ( 146020 available, 55.4% used)
Used static IRAM:   33924 bytes (  97148 available, 25.9% used)
      Flash code:   62908 bytes
    Flash rodata:   18284 bytes
Total image size:~ 123696 bytes (.bin may be padded larger)

I have read the following as well:

Code: Select all

http://esp-idf.readthedocs.io/en/latest/api-guides/general-notes.html#dram-data-ram

According to this guide, I should have 256kB of heap available.

Last piece of the puzzle was the linker script, esp32.ld. Acording to the linker script, we have 320kB of DRAM.

Code: Select all

  /* Shared data RAM, excluding memory reserved for ROM bss/data/stack.
     Enabling Bluetooth & Trace Memory features in menuconfig will decrease
     the amount of RAM available.
  */
  dram0_0_seg (RW) :                 org = 0x3FFB0000 + CONFIG_BT_RESERVE_DRAM,
                                     len = 0x50000 - CONFIG_TRACEMEM_RESERVE_DRAM - CONFIG_BT_RESERVE_DRAM

  /* Flash mapped constant data */
  drom0_0_seg (R) :                  org = 0x3F400018, len = 0x400000-0x18

In summary:
1. Running IDF in dual code mode, with "hello world main" example. I can only allocate 167kB of heap.
However, IDF tells me I still have "(111 KiB): D/IRAM" of heap available.
2. If I run "make size", I still have 146kB of DRAM free
3. According to online readthedocs documentation, I should have 256kB of DRAM.
4. According to the linker scripts I have 320kB of DRAM

Questions:
1. How can I allocate more than 167kB of heap/DRAM in "hello world main" example?
2. Can I access the full 320kB? If no, what is using it?
3. Is there any other documentation I should be reading?


Thanks

[ESP_igrr]

The buffer in you example is allocated statically, not on the heap. You can get heap memory at run-time, by calling 'malloc'.

The maximum size of static data is limited to approximately 160kB, because at that offset from the beginning of DRAM memory space there is a region used by startup code as stack. This region is subsequently given to the application and can be used for heap.
We are tracking a task to allow larger amounts of statically allocated memory in applications, but currently it is not high on the priority list.

An alternative would be to allocate memory at run time from heap.

[hassan789]

Thanks ESP_igrr,
I was able to allocate 280kB of ram, via malloc.
Looks like however, non-contiguous memory is not supported at this time, since I could not malloc all 280kB at once.

Had to do it in this exact order of 3 chunks:
160kB
110kB
10kB

Next steps is to study the heap_caps API:

Code: Select all

http://esp-idf.readthedocs.io/en/latest/api-reference/system/mem_alloc.html