Best Practice to Implement a Circular Logger on Internal Flash

[mfatih]

Hello Everyone,

I want to create a logger on internal flash of ESP32. The data is fixed sized. Some approaches come to my mind listed below:

1- Using NVS library. Disadvantage of this method is needing a key for every data which is unnecessary.

2- Using SPIFFS. This looks heavy to me for this functionality. Also every log data needs a file name and needs additional space on the flash memory. File open/read/write operations may consume more CPU comparing to another method.

3- Writing data by address directly to the flash seems optimum method to me. But I don't know how to do that.

Thanks in advance for any suggestion.

[nicolasludwig]

Hey mfatih, do you have a solution for this? I'm trying to use NVS as a ring logger, but I'm facing some problems. I'd be glad if you can share your results!

[ESP_haberturdeur]

If you want circular buffer, going directly to the flash would probably be easiest.
However it might be beneficial to go through FATFS, or other FS simply for host compatibility.

What to look out for:

• Don't use raw flash directly, use idf provided Wear_leveling component, or implement some other wear leveling scheme yourself.

• If you plan to redirect the output of esp_log.h, take care to not log from inside the circular log buffer implementation and be aware, that the flash layer can possibly log as well, because this can cause infinite recursion. I would recommend using some thread-local flag to stop the recursion.

• Be wary of power loss, use some mechanism to ensure the logs stay in a consistent state no matter what (at the cost of possibly loosing last log message). Depending on your scheme this will probably be the biggest problem, after the first (and all other) wrap around.