ESP32 Forum

- You have a data logger writing data to a file every 100ms.
- You send a fsync and fflush every 5 seconds
- After every hour you close the log and open a new one.

If there is a power down, could you loose more than the last 5 seconds of log?
Are the writes on the FAT sector atomic? or could the SD card need formatting if the power down happens in the middle of a write?

Thanks

Typically SD cards don't offer atomicity guarantees, so if the power down happens while FAT sector is being written, the filesystem may be corrupted (not just the last 5 seconds of the data).

You can search for "raspberry pi SD card corruption" to get the idea of the problems, although Raspberry typically has SD cards formatted not as FATFS. E.g. https://www.raspberrypi.org/forums/view ... &start=175

Just stick a 1-20F cap on your power rail depending on all what your power needs are. Use the built in A/D to detect the power fail, and close the file system right away. A cap like this will give you 10's of seconds of running time when the power fails.

mikemoy wrote: ↑

Fri May 31, 2019 1:10 pm

Just stick a 1-20F cap on your power rail depending on all what your power needs are. Use the built in A/D to detect the power fail, and close the file system right away. A cap like this will give you 10's of seconds of running time when the power fails.

This is what I had done on the previous version. But to reduce size and cost I was looking to remove the supercaps. Also we had one product where the supercap leaked

I did some testing to see how fast I could close the filesystem with a comparator to trigger an interrupt rather than polling with ADC.
But calling esp_vfs_fat_sdmmc_unmount() from interrupt causes a crash.

The next thing I tried was using an event bit from the interrupt to wake a thread that would close all opened files and unmount. This was taking about 50ms but also caused some crashes if it powered down in the middle of writes:

> [0;31mE (64520) spi_master: spi_bus_remove_device(414): Have unfinished transactions[0m
> [0;31mE (64530) spi_master: spi_bus_free(256): not all CSses freed[0m
> C:/msys32/home/gunar.kroeger/esp-idf/components/newlib/locks.c:113 (_lock_close)- assert failed!
> abort() was called at PC 0x40085697 on core 0

Is there a way to close filesystem from an interrupt? Or has anyone other ideas?
PS: I don't care about loosing the last 5 seconds, so if there is a way to unmount without fclose it could be faster

Thanks

But calling esp_vfs_fat_sdmmc_unmount() from interrupt causes a crash.

Have a look at the GPIO example. This is the way you can trigger something to happen outside an IRQ.

https://github.com/espressif/esp-idf/bl ... ple_main.c

mikemoy wrote: ↑

Fri May 31, 2019 3:16 pm

Have a look at the GPIO example. This is the way you can trigger something to happen outside an IRQ.

https://github.com/espressif/esp-idf/bl ... ple_main.c

yes, from what I understand it does the same as using event queue.
But it is an extra delay.

If doing it outside IRQ is the only way:
what is the sequence I should do after the xEventGroupWaitBits/xQueueReceive to safely close the FATFS?
- should I delete all tasks that could be writing to the SD card at the moment?
- should I discard the cache of all the fwrites that have not been fflushed and fsynced? How could I do that?
- should I call esp_vfs_fat_unregister_path after esp_vfs_fat_sdmmc_unmount?

yes, from what I understand it does the same as using event queue.
But it is an extra delay.

The delay is so minimal, your talking in the very low microseconds. For instance I am using the same method to run a BLDC motor. The 3 hall sensors trigger an IRQ to which the waiting task does the work. It will be far faster than you need.

I have not done any work with the filesystem so to that end I cannot speak. I would check out the examples. I am sure what your looking for is in there.

I think that I will abandon FATFS. It looks like it really isn't design for robustness.
We can't risk having to format the card and loose all logged data if the FATFS gets corrupted.
Even if I get the unmount right during a power-down, there can be an unforeseen bug that causes a crash forcing the esp to restart.

The solution I'm going to study is to write directly to the sectors using SD/SDIO/MMC Driver (driver/sdmmc_host.h) without a Filesystem. This should have the disadvantage of not being able to read the data from e.g. windows directly but hopefully will be less error prone.

Still open for suggestions for other ways to solve this (:

I think that I will abandon FATFS. It looks like it really isn't design for robustness.

Don't give up so quickly.

From looking at the examples I don't understand what is the issue your really having.
From the examples it looks like just a few commands are needed:
Taken from this example.
https://github.com/espressif/esp-idf/bl ... ple_main.c

fclose(f);
esp_vfs_fat_spiflash_unmount(base_path, s_wl_handle);


IMHO, I would be to concerned writing that much data to a uSD card over and over again. I'd nab me a Wrover module with it extra ram, and write all your data to ram, then at the end of your hour, just do one write to the card and close it.

As for the cap leaking goes, sounds like you might be using some cheap Chinese brand. We have been using them for over 14 years and never had one product come back with a bad one.

I'll stop chiming in and let someone else advise.

Well I'm just giving if I find better alternatives haha,
Thanks for the suggestions mikemoy. Good supercaps are indeed expensive ):

I've found this alternative https://www.freertos.org/FreeRTOS-Plus/ ... stem.shtml
wonder how hard it would be to implement for the esp. I shall give it a try