SD_MMC file access time

[copych]

Code: [Select all] [Expand/Collapse]

1. SD_MMC.open(filename)

GeSHi © Codebox Plus Extension

Is there a way to speed up file open operation? I can imagine some kind of caching start sectors location or some sort of keeping indexed directory structure on the app's side or whatever. Or maybe some tuning is available (I'd really prefer to stay in Arduino for compatibility reasons).
I have a code that uses SD_MMC driver for accessing files on a microSD card. The app is time-critical. On the card there are folders and files, hundreds of them. The problem is access time. First files (I mean default sorting) in the folder are about 5ms till first read, but the 200th file would be already ~20ms to open, and so on, which is not acceptable for my goal.
ESP is only performing reading operations, but quite intensively.
So, please, is there a way in ESP32 Arduino to have lower level access to the filesystem or even to the SD, so I could use something instead of SD_MMC.open(filename) for random access? But I wouldn't want to write my custom fatfs )))
Please note, that this must be an ordinary FAT card, not a custom partitioned storage. It must contain files readable/writable on a PC.

[lbernstone]

Only thing I can recommend would be to try using the SdFat library. It may do some additional caching of the file tables that would give you faster access.

[copych]

Oh, thank you so much for the answer and for the link! Meanwhile I wrote a raw-reader class for the FAT32 SD, and now facing a problem with caching, cause data is quite heavy in terms of ESP32.
Please, advise how to make it better:
I use ESP32 / ESP32S3 boards with PSRAM.
When building an index of the directory with both long and short file names, I use std::vector (I need the names at least for some time, cause next I have to parse them and bind indexes to events, and several passes may be needed). And at some point I receive rst:0x8 (TG1WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT), which I suppose is connected with heap allocation problems during vector growing (wrong?). At least, increasing stack or adding taskYIELD() don't help.
So how can I use PSRAM to allocate std::vector? Should I ps_malloc() the initial vector, and then it'd be growing in PSRAM? Or am I getting it wrong?

[lbernstone]

https://github.com/lbernstone/psram_cus ... ocator.ino

That WDT reset is more likely due to slow SPI access blocking the cpu. Make sure you put some delay in between heavy periods of disk reading/writing. If you already have this separated as a task, then lower the priority of your task to use yield.

[copych]

Thank you so much.

[copych]

That WDT reset is more likely due to slow SPI access blocking the cpu. Make sure you put some delay in between heavy periods of disk reading/writing. If you already have this separated as a task, then lower the priority of your task to use yield.

I have tried to place delay(10) or even delay(100) inbetween intense readings, but no luck. Also, please, specify what SPI access do you mean: PSRAM or SD? Actually I thought that SD_MMC is not SPI, and PSRAM is not engaged as a heap in Arduino IDE, so I don't use it in this part of the app yet (please, forgive my ignorance).

[lbernstone]

Ok, it is the sdio bumping the watchdog, not spi. sdio is black box, so you may just have to increase the watchdog timer.