How to reduce background dynamic allocation in ESP-IDF sdmmc driver when reading from SD card

[adyrapeaux]

I'm developing an application on ESP32-S3 N8R2 (ESP32-S3-WROOM-1) that pulls in 500+ KB/s of data from an SPI-connected device, compresses it to ~ 150 KB/s, stores that on an SD card, and reads back the file to stream the contents to an http server, all at once.

I have an issue wherein the sdmmc driver makes hundreds of individual one-sd-card-block dynamic allocations per second when reading the card. At times, because I'm tight on available system memory at 33 kB available during runtime, this causes an allocation fail for the driver which then crashes. I could recover but I'd lose the data written to file in the SD card as it wouldn't be closed properly so info wouldn't be written to FAT. That's unacceptable, even if the rest of the system doesn't crash.

A bit of context:

To make things run on time and avoid the occasional wait on malloc I've tried allocating as much as I can statically, once, at boot time. As a result of optimizing wifi transfer speeds and placing certain functions e.g. SPI master functions in IRAM rather than flash, I have realtively little free RAM available, but I make no dynamic allocation calls in my application (that I have control over).

The remaining available heap memory is about 33 kB during runtime. I monitor this value once every 8 seconds and while it fluctuates slightly by several kB, over time it stays stable, but only when I just write to the SD card (and not when reading/writing).

I have an issue with the sdmmc driver in ESP-IDF: this driver calls malloc to make dynamic allocations of size [one SD card block] (or 512 bytes in my case) when the input memory to write to when reading from SD card is not DMA-compatible. The number of calls for writing the SD card is quite small and doesn't cause any issue, however there are many, many such calls when reading the SD card. Every now and then the SD card decides to do some housekeeping (likely page erase), and when this happens the calls pile up and available heap can decrease significantly, enough that on occasion the sdmmc driver's allocation call fails, the driver crashes, and I lose data (e.g. file isn't closed properly so even if I can recover from the error that's unacceptable for my application).

Is it possible to know what's happening under the hood for this driver to request so many one-block memory allocations (and whether I can assign a static, DMA-compatible address for that?) when reading from the SD card? We're talking hundreds of calls a second.

Additional information:
I don't have a working debugger but I did a bit of diving in the driver code and I've managed to isolate the call as a result of executing sdmmc_read_sectors in sdmmc_cmd.c with a non DMA-capable memory address for writing to (from the SD card).

I am using SPIRAM for a substantial part of my program but have specifically prevented the normal system from using malloc to allocate memory in SPIRAM, which I know to be DMA-incompatible in sdk config.

Calls for reading/writing to the SD card that I use in my program all make use of the same statically-allocated DMA-compatible buffer (64 kB). I either write or read from the SD card so I use the same 64 kB buffer for both, in sequence, in the same thread function, so I guarantee that buffer can't be accessed from two places at once. For inter-thread communication I pass addresses pointing to larger SPIRAM buffers to other threads via queues - that works great and I don't have any issues there.

I'm wondering how to reduce/eliminate the instances where the sdmmc driver is passed these DMA-incompatible addresses. Printing out the offending addresses has let me determine that the address never changes during runtime, currently it's 0x3dfe0bc8 associated to SD card writing (3 calls in quick succession every 5 seconds or so, not an issue), and 0x3dfe23fc associated with SD card reading (hundreds of calls a second, I overload the console if I print the statements).

I can test with no SD card reading (just the writing) quite easily, but can't do the same the other way around. When doing just SD card writing, everything works like a charm. When doing both writing and reading, I crash within a couple to about 10 minutes of booting the system (actively transferring data from the SPI-connected device).

Please advise, happy to provide additional details. I've attached the part of my program that makes use of the sdmmc driver. I'm aware it wouldn't pass muster for production since I'm using esp_vfs_fat_sdmmc_mount() rather than going through the proper steps for setting up the sd card. It's worked great for me so far, let me know if my issue is linked to using that function.

Kind Regards,

adyrapeaux

EDIT: I was mistaken about the size of the dynamically allocated block. One SD card block is actually 512 bytes, and not 32 as in the original post.

[adyrapeaux]

Hello all,

For what it's worth I managed to get it working by simply increasing the amount of available system RAM during runtime from 33 kB to about 50 kB - trimming down some static allocation from other threads in my application. Since then I haven't had any crashes. When reading and writing to the SD card the amount of free heap falls to ~ 20 kB on occasion, but always recovers to 50 kB afterwards (sometimes over several seconds). No fluctuations of that size happen when only writing to the SD card (just a couple of kB, that I've been able to observe).

If this is intended behaviour it would be useful to include a comment in the documentation for SD card interfacing with ESP32 that the driver requires ~30 kB of free heap to run when reading from an SD card.