BUG?! in Memory Usage

[atsteich]

In Principle there should be ~300 Kbyte ram available to save data on an esp32 chip. But: As soon as i allocate more memory than ~94400Kbyte to log e.g. some data, the chip crashes before entering setup(). As long the allocated memory is smaller, the chips reports free heap of ~200Kbyte. It is not a problem of the size of the arrays, since it is also not possible to allocate more memory in smaller blocks. (not even in very small blocks, I tried...)

SO, what is going on there?? Is the usable Ram (for data) only 100K? Found nothing about that in any spec sheet... I found a bug report here: https://github.com/espressif/arduino-esp32/issues/1163 that sounds pretty similar but there was no solution found...

Can anybody try to reproduce? I really don't get what the Problem here is...

[ESP_igrr]

Not all of the available heap memory is contiguous. You can call heap_caps_get_largest_free_block to determine the how much can be allocated in one malloc call.

https://docs.espressif.com/projects/esp ... k8uint32_t

[atsteich]

Since it is not even possible to allocate memory for single integer values, that this is not the problem. What is the variable t_caps when calling heap_caps_get_largest_free_block(uint32_t caps)?

[atsteich]

As i wrote, it is not even possible to allocate memory for a single int as soon as the total used variables need more memory than (in my case) exactly 94492 bytes. So the remaining 200K ram consists of non contiguous blocks smaller than 4 bytes? Sounds not plausible.... Also what exactly is the variable "t_caps" that the function size_t heap_caps_get_largest_free_block(uint32_t caps) takes?

[ESP_Angus]

As i wrote, it is not even possible to allocate memory for a single int as soon as the total used variables need more memory than (in my case) exactly 94492 bytes. So the remaining 200K ram consists of non contiguous blocks smaller than 4 bytes?

Each allocation has 4 bytes of heap metadata overhead, so malloc(4) will make free heap go down by 8 bytes (in the default heap configuration used in Arduino). I'm not quite sure what you're describing, but maybe this helps explain it?

Sounds not plausible.... Also what exactly is the variable "t_caps" that the function size_t heap_caps_get_largest_free_block(uint32_t caps) takes?

It can be a bitmask of various capability values, because the ESP32 contains different types of RAM which can be used for different purposes. An extended description with all the capability values is here:
https://docs.espressif.com/projects/esp ... alloc.html

To see the amount of free heap which can be allocated using regular malloc(), you can call either xPortGetFreeHeapSize() or heap_caps_get_free_size(MALLOC_CAP_8BIT) (these two are equivalent and deal with 8-bit accessible "DRAM" only).

To see the largest single block of contiguous free heap available to malloc(), you can call heap_caps_get_largest_free_block(MALLOC_CAP_8_BIT)

Some other heap information functions are also available, see https://docs.espressif.com/projects/esp ... nformation

Bear in mind that static memory usage (.data, .bss) also take from the available DRAM, and WiFi and Bluetooth both make allocations from heap as well.

During boot a short summary of the total heap memory in the system (after static memory is assigned to the app but before any heap allocations are made) is logged on the console:

Code: Select all

I (229) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (235) heap_init: At 3FFB32D8 len 0002CD28 (179 KiB): DRAM
I (241) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (247) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (254) heap_init: At 40088C00 len 00017400 (93 KiB): IRAM

(MALLOC_CAP_8BIT corresponds to the DRAM and D/IRAM regions in this log.)

Note that after these values are logged other parts of the system will start allocating from heap, especially if the app initializes WiFi/BT/etc, but also for RTOS task stack memory, etc. So not all this memory will be available.

[chegewara]

Just like Angus said. The problem is that you are trying to allocate global variables in arduino.
https://github.com/espressif/arduino-esp32/issues/1163

Maybe my explanation is not 100% accurate in this issue and someone can correct me.

[atsteich]

Sorry for my double reply, and thank you very much for your help! I think there is still anything strange going on, Iet me explain more detailed:

The following code

Code: [Select all] [Expand/Collapse]

1. #include <Arduino.h>
2. #include <Esp.h>
3.  
4. int* var1 = (int*) calloc(24000, sizeof(int));
5.  
6. void setup() {
7.     Serial.begin(115200);
8.     for(int i = 0; i < 24000; i++){
9.             var1[i] = i;
10.         }
11.     Serial.printf("biggest free block: %i\n", heap_caps_get_largest_free_block(MALLOC_CAP_8BIT));
12. }
13.  
14. void loop() {
15.     Serial.println("main");
16.     delay(2000);
17. }
18.  

GeSHi © Codebox Plus Extension

fails, and results in a rebooting loop (not even entering setup() ). And of course this could be explained (as you said) by the fact, that the ram is split up in some blocks (3 blocks with ~114000 bytes?). Running an empty script, the largest available block reported is in my case 113792 bytes. Although var1 has a total size of "only" 96000 bytes, I could imagine that some memory of the block is used while booting, and released before entering setup() ... So, this could be ok.


So, coming to something I cannot understand:
The following code

Code: [Select all] [Expand/Collapse]

1. #include <Arduino.h>
2. #include <Esp.h>
3.  
4. int * var1 = (int*) calloc (23000, sizeof(int));
5.  
6. void setup() {
7.     Serial.begin(115200);
8.     var1[0] = 1;
9.     Serial.printf("biggest free block: %i\n", heap_caps_get_largest_free_block(MALLOC_CAP_8BIT));
10. }
11.  
12. void loop() {
13.     Serial.println("main");
14.     delay(2000);
15. }

GeSHi © Codebox Plus Extension

works fine, and in setup() the chip reports still a largest free block size of 113792. So in principle it should be possible to allocate another memory block like the one for var1 right? But this fail and results in an reboot loop. Also trying to allocate a smaller block for e.g. only 2000 integer values fails, although there should be a free memory block of more than 100kKByte. This is strange, I do not understand why this happens.

So, declaring global Variables with a total size bigger than ~94 KByte fail. I have tried with multiple chips, and flashing the chips with Arduino IDE, as well as PlatformIO. I also tried to allocate the memory in smaller blocks, nothing worked. As soon as the total allocated GLOBAL memory gets bigger than ~9400 KByte, the chips end up in a reboot loop.

What DOES work, is declaring the variables in a local scope:

Code: [Select all] [Expand/Collapse]

1.     #include <Arduino.h>
2.     #include <Esp.h>
3.  
4.  
5.     void setup() {
6.         Serial.begin(115200);
7.         //filling the memory with some values...
8.         int* var1 = (int*) calloc(23000, sizeof(int));
9.         int* var2 = (int*) calloc(23000, sizeof(int));
10.         for(int i = 0; i < 23000; i++){
11.             var1[i] = i;
12.             var2[i] = i;
13.         }
14.         delay(500);
15.         Serial.printf("biggest free block: %i\n", heap_caps_get_largest_free_block(MALLOC_CAP_8BIT));
16.     }
17.  
18.     void loop() {
19.         Serial.println("main");
20.         delay(2000);
21.     }

GeSHi © Codebox Plus Extension

removing var2, the chips still reports a largest block of 113792, and i can allocate (in this case) the full empty block (I tried allocating 112 KByte, and it worked). So this is working as expected. A bit strange, (but still explainable) is, that var1 still must not be bigger than ~94 KByte. I think this could be due to memory needed while booting.

In summary, it looks like one can only allocate memory of one of the 3 ram block in global scope... Is there any suitable reason for this, or is this a bug? I know, there are simple workarounds, but if one can allocate global memory, why only a part of it?

Best, Christoph