pvPortMallocCaps with MALLOC_CAP_32BI and memset

[commarmi76]

Hello,

I ported lame (libmp3lame) mp3 encode to esp32. I'm using pvPortMallocCaps with MALLOC_CAP_32BIT to get memory for two big int arrays. I wonder if I can use memset to initialize this arrays because according the documentation, memory allocated with MALLOC_CAP_32BIT needs to be accessed in 32bits words.
Is there someone that knows the answer ?

PD: The port is in https://github.com/commarmi76/libmp3lame_for_esp32 and apparently works

Thanks

[ESP_Angus]

Very exciting port!

Memset (and the similar newlib functions) should use only 32-bit word operations (for performance) as long as the argument buffer is at a 32-bit aligned address & is a 32-bit multiple of length. So if you're memsetting the entire buffer, this will be safe.

You can read the source here: https://github.com/espressif/newlib-esp ... g/memset.c

Out of interest, why are you using MALLOC_CAP_32BITS? Do you need to allocate from IRAM to avoid running out of DRAM?

[commarmi76]

@ESP_Angus:
You are right, I use MALLOC_CAP_32BITS because Lame creates a lot of buffers but two at the begining are specially big (48k echa one). Fortunately these are int buffers. You can see memory info at the start of the lame_test.c:

ESP32 SDK version:v2.0-rc1-590-g0ea4c3c0, RAM left 180060
get_lame_version(): 3.99.5
pre lame_init() free mem8bit: 179892 mem32bit: 284324
post lame_init() free mem8bit: 91724 mem32bit: 196156
....
post lame_init() free mem8bit: 21136 mem32bit: 29552


As you can see lame is a memory eater.

Br.

[commarmi76]

Very exciting port!

Memset (and the similar newlib functions) should use only 32-bit word operations (for performance) as long as the argument buffer is at a 32-bit aligned address & is a 32-bit multiple of length. So if you're memsetting the entire buffer, this will be safe.

You can read the source here: https://github.com/espressif/newlib-esp ... g/memset.c

Out of interest, why are you using MALLOC_CAP_32BITS? Do you need to allocate from IRAM to avoid running out of DRAM?

other two questions:

- Can I use MALLOC_CAP_32BITS to create array of float ? I'm not sure if floats are accesed only in 32 bits way.

- When I try to create an array (float [64][64]) ) but I have no enough memory, the system don't crash but when I access to the array I write other parts of the memory. Maybe I have something wrong configured, did someone experienced something similary ?

thks.

[ESP_Angus]

- Can I use MALLOC_CAP_32BITS to create array of float ? I'm not sure if floats are accesed only in 32 bits way.

I'm fairly sure this will work.

- When I try to create an array (float [64][64]) ) but I have no enough memory, the system don't crash but when I access to the array I write other parts of the memory. Maybe I have something wrong configured, did someone experienced something similary ?

How are you creating this array, is it on the stack or in static memory or via malloc? Can you post a code snippet?

[commarmi76]

Hello,

you can see here: https://github.com/commarmi76/libmp3lam ... el.c#L1820
If I don't set CONFIG_MAIN_TASK_STACK_SIZE with enough memory this line don't crash but when the code reaches the memset below the array declaration, it overwrite (with 0) a global array defined here: https://github.com/commarmi76/libmp3lam ... pvt.c#L172
It sounds crazy, but it happens.

Thanks

[f.h-f.s.]

if your stack isn't big enough, your variable cant be allocated or will overflow your task stack. You have a overflow there, you declare a big two dimensional array in a function that overwrites that array. Right?

[commarmi76]

if your stack isn't big enough, your variable cant be allocated or will overflow your task stack. You have a overflow there, you declare a big two dimensional array in a function that overwrites that array. Right?

Yes. Is there any method to check this kind of overflow implemented in freertos ?
I got crazy trying to debug this issue.

[ESP_Sprite]

We have stack canaries, as well as the option to put a watchpoint on the end of the stack; see menuconfig for these things. Because of pretty fundamental reasons, this only can catch writes to the last 64 or so bytes of the stack, so if you allocate an array on the stack and don't write to all entries, you can still manage to corrupt memory. In the general case, this usually helps catching those issues however.

[f.h-f.s.]

You can also use malloc and free instead of declaring a array.
overflow canaries are definitely a must have.