[SOLVED] Question about reading and writing to the same sd card simultaneously

[zydizz]

Hi all,

I have a code that takes measurements from a microphone 8 times in a second, those 8 measurements get averaged and a final value for a second is calculated. Then the calculations get appended as a line to a text file in the sd card. In its current state the code runs for some defined duration then sends the final form of the text file to a server.

The thing I want to achieve is that the measurement never stops (until like a physical interrupt). But after some defined duration (say like 15 minutes) the device sends the data from the last 15 minutes to the server. BUT as this sending process is taking place the measurements must still continue and write their results into a file in the sd card.

How should I approach this problem? Utilizing both cores of the esp32? Interrupts?

[lbernstone]

There's not really any benefit to having this simultaneous, since there is only one pipeline to the hardware, and lots of downside. So, rather than shoehorning into "simultaneous", work on making the process asynchronous, and then put a mutex on the sd access. This means you need to have a storage buffer for your incoming data that is big enough to handle any possible delays, and a process for writing what may be more than 8 pieces of data. A mutex example will be added to the core soon- https://github.com/espressif/arduino-es ... c48b78a5d0
Another way to handle this, if data collection is the primary function of the device, would be to drop the extra $0.25 and get an esp32 with psram. This gives you an extra 4MB of data space to work with, so you could easily keep the whole "file" as a string in active memory, write it out to flash periodically, and deliver it to your permanent storage as needed. You can see an example of this methodology at https://github.com/lbernstone/rrdtool_E ... sramDB.ino

[zydizz]

There's not really any benefit to having this simultaneous, since there is only one pipeline to the hardware, and lots of downside. So, rather than shoehorning into "simultaneous", work on making the process asynchronous, and then put a mutex on the sd access. This means you need to have a storage buffer for your incoming data that is big enough to handle any possible delays, and a process for writing what may be more than 8 pieces of data. A mutex example will be added to the core soon- https://github.com/espressif/arduino-es ... c48b78a5d0
Another way to handle this, if data collection is the primary function of the device, would be to drop the extra $0.25 and get an esp32 with psram. This gives you an extra 4MB of data space to work with, so you could easily keep the whole "file" as a string in active memory, write it out to flash periodically, and deliver it to your permanent storage as needed. You can see an example of this methodology at https://github.com/lbernstone/rrdtool_E ... sramDB.ino

Thank you so much for answering, both of these methods did come to my mind. I guess my board does have 4mb psram. That approach would be far easier (and logical due to the strict timing requirements of the measurement task) to implement in my current code. Also, thank you so much for the examples you have provided.

By simultaneous I meant that the main task should NOT be intrrupted by any means. Because it takes measurements in 0.125 second intervals FIXED. That is why i find that psram approach better, because a minute of calculations is about 20kb's of data.

Thanks again!