Persistent time counter using NVS without burning a hole in the FLASH?

[akasaka_spk]

Once again I'm designing weird features into my clock OS/platform.

This time I want the settings menu to display a runtime counter on vacuum tube based displays, to be able to tell how many hours the VFD/plasma display has spent actively showing something and/or with the heater turned on.

Of course, just storing a uint64 in NVS will be wasteful and also slow, because I presume NVS erases the whole block of flash just to change a single setting.

So what I came up with is:

1. Allocate 512 bytes of NVS for each counter. Of which 8 bytes will be our hours count (uint64), 8 bytes can be the checksum (e.g. CRC), and the rest is filled with 1's (0xFF, a.k.a. erased).
2. Every power on, read the array and count the number of 0 bits. For ease of programming the number of 0 bits is kept in RAM, so we don't have to count them every time.
3. Every second, go to the array and flip one of the 1's into a 0. Since Flash does not need erasing to change 1 to 0, unlike the opposite, it can be done very fast (microseconds).
4. Once we have changed exactly 3600 of the 1's into 0's, that mean an hour has passed. We increment the hour counter allocated in the 8 bytes, and at the same time rewrite the remaining bytes back to 0xFF. This could also be a good place to relocate the data block for wear leveling.
5. When the user requests the total runtime of the display, we can calculate it as (8 byte value) hrs + (number of 0 bits / 60) minutes + (number of 0 bits % 60) seconds and show it in the service menu.

The questions that I have are:

1. Will this actually wear out the flash memory less than just updating an uint64?
2. How do I approximate the lifetime of the flash with this algorithm, if I'm not exactly doing a whole cell cycle each second? Can I just consider this as one cycle per hour? (thus 100k hrs without wear leveling?)
3. Does the NVS library allow to even do bit-level manipulation of the byte arrays, or does it yeet the whole block and rewrite it again no matter what? Can I perhaps mmap an NVS setting to do this myself, and only unmap and let the NVS library take care of it for wear leveling and all that on the hour mark?

[ESP_Sprite]

Once again I'm designing weird features into my clock OS/platform.

This time I want the settings menu to display a runtime counter on vacuum tube based displays, to be able to tell how many hours the VFD/plasma display has spent actively showing something and/or with the heater turned on.

Of course, just storing a uint64 in NVS will be wasteful and also slow, because I presume NVS erases the whole block of flash just to change a single setting.

You're assuming wrong. How it does work is documented.

So what I came up with is:

(Cut: setting a bit for each second)

You could, but you'd have to use a raw partition and not NVS for the bit-twiddling, because as documented above, NVS works differently.

Honestly, what I'd do is decrease your granularity a bit: is it that important that the hour count will change by a certain fraction of an hour if the clock looses power? If you for instance go for a still-probably-way-too-accurate minute count, you'd write a new NVS entry (consisting of 32 bytes) every minute. Assuming a small 16K NVS partition, that means it would fill up with old entries every (16K/32=)512 minutes, or almost every 10 hours. The flash we use in ESP32 chips with internal flash can handle 100K cycles, but say you got a lower-quality that can only do 10K cycles; that would still net you 5120000 minutes or 9 years of continuous writing. If you need more, you could allocate a larger NVS partition and get a proportionally longer lifetime.

[MicroController]

hours count (uint64)

Probably a waste of storage space as vacuum tubes and their enclosing universes tend to break down a lot earlier than 2^64 hours after first deployment

[akasaka_spk]

Probably a waste of storage space as vacuum tubes and their enclosing universes tend to break down a lot earlier than 2^64 hours after first deployment

You can never know! Especially if using only 450 out of the 512 byte block why bother saving space

... write a new NVS entry (consisting of 32 bytes) every minute.... it would fill up with old entries every (16K/32=)512 minutes, or almost every 10 hours.... that would still net you 5120000 minutes or 9 years of continuous writing

Interesting, I actually have a 20K NVS in the project, and I don't think a lot of it is actually full. Maybe 10 minute precision is even good enough. Plus some timestamp saving to account for boot times e.g. during OTA HTTP updates.

That being said, I'm using WROVER-IBs from Taobao so I would say the quality might be even less. They are bad enough to the point that I have to pop off the shielding and fix tombstoned caps around the crystal and power circuitry, gah!

And I am hoping to get at least 15 years out of those, already working on the 2038 fix 'cause arduino-esp32 seems to not have the latest IDF yet

[MicroController]

Maybe also an option:
"The DS1307 serial real-time clock (RTC) is a low- power, full binary-coded decimal (BCD) clock/calendar plus 56 bytes of NV SRAM."

[akasaka_spk]

Yes, that is also something I'm considering for the next revision of the PCB. Who knew a clock could use an RTC, huh!

[MicroController]

Who knew a clock could use an RTC, huh!

Maybe @Espressif could consider providing a Vbat pin with automatic switch-over for retention power of RTC RAM and timer on future chips... (Potentially also enable the ULP to run uninterrupted via that supply, or even the option to wake-up the ULP upon main power loss?)

[ESP_Sprite]

Maybe @Espressif could consider providing a Vbat pin with automatic switch-over for retention power of RTC RAM and timer on future chips... (Potentially also enable the ULP to run uninterrupted via that supply, or even the option to wake-up the ULP upon main power loss?)

I think the P4 has a pin like that, although I'm not 100% sure what power domains (aside from retention for the RTC timer) it serves.