Managing RAM for large arrays
Posted: Wed Jun 26, 2024 9:43 am
Hi
I want to use large float arrays in a code, related to AI. So before that I want to test how much of such arrays I can load in RAM.
I wrote a code to test but I'm quite disappointed in the results.
Here is the code : it dynamically allocates a temporary array to place it in flash memory, using Preferences. Then I delete this array, since I don't need it anymore.
Then, as I need to use 2 arrays at the same time, I allocate 2 arrays of the same size. I load the data from flash into the first array, do some maths to compute the second one.
My idea is to increase the value of N until I get out of available RAM. It works until N = 14330. Here is the output:
What is wrong ? How can I use larger array sizes?
Thanks for your help.
I want to use large float arrays in a code, related to AI. So before that I want to test how much of such arrays I can load in RAM.
I wrote a code to test but I'm quite disappointed in the results.
Here is the code : it dynamically allocates a temporary array to place it in flash memory, using Preferences. Then I delete this array, since I don't need it anymore.
Then, as I need to use 2 arrays at the same time, I allocate 2 arrays of the same size. I load the data from flash into the first array, do some maths to compute the second one.
Code: Select all
#define N 14320
#include <Preferences.h>
Preferences donnees;
void setup() {
Serial.begin(115200);
donnees.begin("Network", false, "network"); // namespace, readonly, partition name
Serial.println("Initializing TEMP array");
// for (int i = 0; i < N; ++i) printf("temp[%d] = %f\n", i, temp[i]);
float* temp;
temp = (float*)malloc(N * sizeof(float));
for (int i = 0; i < N; ++i) {
temp[i] = i * i / 5.0;
}
printf("temp[1] = %f\n", temp[1]);
printf("temp[N-1] = %f\n", temp[N - 1]);
printf("Free RAM: %d\n", esp_get_free_heap_size());
Serial.println("Moving TEMP to flash");
donnees.putBytes("data", temp, N * sizeof(float));
Serial.println("Erasing TEMP");
free(temp);
printf("temp[0] = %f\n", temp[0]);
printf("Free RAM: %d\n", esp_get_free_heap_size());
Serial.println("Creating IN and OUT arrays");
float* in;
in = (float*)malloc(N * sizeof(float));
float* out;
out = (float*)malloc(N * sizeof(float));
printf("Free RAM: %d\n", esp_get_free_heap_size());
Serial.println("Reading IN from flash");
donnees.getBytes("data", in, N * sizeof(float));
Serial.println("Testing IN:");
printf("in[1] = %f\n", in[1]);
printf("in[N-1] = %f\n", in[N - 1]);
// for (int i = 0; i < N; ++i) printf("in[%d] = %f\n", i, in[i]);
Serial.println("Computing OUT");
for (int i = 0; i < N; ++i) out[i] = in[i] * 2.0;
Serial.println("Testing OUT:");
printf("out[1] = %f\n", out[1]);
printf("out[N-1] = %f\n", out[N - 1]);
Serial.println("Moving OUT to IN");
in = out;
Serial.println("Testing IN:");
printf("in[1] = %f\n", in[1]);
printf("in[N-1] = %f\n", in[N - 1]);
}
void loop() {
// put your main code here, to run repeatedly:
}
Then if I increase N to 14450 it throws an error:temp[1] = 0.200000
temp[N-1] = 41064048.000000
Free RAM: 159936
Moving TEMP to flash
Erasing TEMP
temp[0] = 0.000000
Free RAM: 217272
Creating IN and OUT
Free RAM: 102600
Reading IN from flash
Testing IN:
in[1] = 0.200000
in[N-1] = 41064048.000000
Computing OUT
Testing OUT:
out[1] = 0.400000
out[N-1] = 82128096.000000
Moving OUT to IN
Testing IN:
in[1] = 0.400000
in[N-1] = 82128096.000000
It seems that the second array 'out' wasn't correctly allocated. The value of the free RAM is equal after the allocation of the temp array, and after the allocation of the 2 arrays of the same size:Initializing TEMP
temp[1] = 0.200000
temp[N-1] = 41178760.000000
Free RAM: 159712
Moving TEMP to flash
Erasing TEMP
temp[0] = 0.000000
Free RAM: 217128
Creating IN and OUT
Free RAM: 159712
Reading IN from flash
Testing IN:
in[1] = 0.200000
in[N-1] = 41178760.000000
Computing OUT
Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception was unhandled.
Core 1 register dump:
PC : 0x400d15de PS : 0x00060a30 A0 : 0x800d31f9 A1 : 0x3ffb2250
A2 : 0x3ffc1040 A3 : 0x00000000 A4 : 0x3ffe46c4 A5 : 0x00000004
A6 : 0x0000e034 A7 : 0x3f4001d9 A8 : 0x0000380e A9 : 0x00000000
A10 : 0x3ffe46c4 A11 : 0x0000000d A12 : 0x40000000 A13 : 0x4183a2b4
A14 : 0x38000000 A15 : 0x00000000 SAR : 0x00000004 EXCCAUSE: 0x0000001d
EXCVADDR: 0x00000000 LBEG : 0x400d15d2 LEND : 0x400d15e1 LCOUNT : 0x0000380d
Backtrace: 0x400d15db:0x3ffb2250 0x400d31f6:0x3ffb2290
But for a total RAM size of 500 kB, I was expecting to reach N values up to 500 / 2 / 4 so close to 60000.Free RAM: 159712
What is wrong ? How can I use larger array sizes?
Thanks for your help.