Data Storage

[Jerameyas]

I am experiencing issues with the NVS storage on my ESP32-S2-Solo-U device. According to the built-in partition table, the NVS storage is available from address 0x9000 to 0xEFFF. However, I am unable to retain the data stored in the address range from 0x9000 to 0xA500. I need assistance to understand and resolve this issue.

Specifically, I would like to know:

What could be causing the data retention issue in the specified address range?
Where exactly is the NVS data stored—Is it in the internal flash or elsewhere?
Is there a possibility that my program code is overwriting these locations?
To further investigate, I would like to extract the hex file from the device with the existing code to analyze the memory usage and data storage. I am flashing the code via UART.

Could you please provide guidance on how to extract the hex file from the device via UART?

[ESP_Sprite]

Explain a bit more please: how are you storing data and how do you conclude it's disappearing?

[Jerameyas]

I've been experiencing issues with data retention within the NVS storage range on my ESP32-S2-Solo-U device. According to the built-in partition table, the NVS storage spans from address 0x9000 to 0xEFFF. However, any data stored in the address range from 0x9000 to 0xA500 is not retained after a restart.

Here's a detailed explanation of my situation:
Data Storage Method:
I am writing data to the specified locations using the esp_flash_write function. After writing the data, I restart the device and then read back the data from these locations to verify retention.

Issue Observed:
Despite being within the NVS storage range, the data appears corrupted or lost when I attempt to read it after a restart. This behavior contradicts the expected functionality of the NVS storage, which should retain data even after a power cycle.
Steps Taken for Analysis:
To further investigate, I have extracted the bin file from the device using esptool and converted it to a hex file. Here’s the memory layout I found:

0x0000 to 0x1000 (4KB): Free space
0x1000 to 0x64F0 (21.23KB): Bootloader
0x8000 to 0x8080 (128 bytes): Partition Table
0x9000 to 0xCA10 (14.52KB): Unknown Data
0xE100 to 0xEF00 (3.5KB): User saved data
0x10000 to 0xEE6F0 (891.75KB): Firmware code
0xEE6F0 to 0x3FFFF0 (3.07MB): Free space
I would like to understand what this unknown data (0x9000 to 0xCA10) is, given that it is within the NVS storage range as per the partition table. When I attempt to store data in this range, it gets corrupted upon restart.

Questions:
Unknown Data in NVS Storage: What could this unknown data be, and why is it occupying space within the NVS storage range?
Data Corruption: Why is the data I store in this range getting corrupted on restart?
Best Practices: Are there any specific guidelines or best practices I should follow to ensure data integrity in the NVS storage?

Any insights or guidance on these issues would be greatly appreciated.

[ESP_Sprite]

You're misunderstanding the use of the NVS partition. NVS is an api on top of the flash that's a key-value store that handles wear leveling, atomic writes, etc. It has its own formatting of the flash and you should only interface with it through the NVS API. If you directly write to that partition as well, you'll mess things up. If you simply want to have a place where you can use the raw flash API to read and write, you should add your own partition with your own data type to the partition table and use that.

[Jerameyas]

Thank you for your response. I understand that the NVS partition is meant to be accessed via the NVS API and that direct writes can cause issues. However, I observed some curious behaviour that I hope you can help me understand.
I retained data upon restart when I saved data in the 0xE000 to 0xEF00 locations, but not in the 0x9000 to 0xA500 range. Since both locations fall within the NVS storage partition, this discrepancy adds to my curiosity.

Could you provide insights into the following:
Data Retention Behaviour: Why is data retention successful in the 0xE000 to 0xEF00 range but not in the 0x9000 to 0xA500 range, given that both are within the same NVS storage partition?

Understanding Data Occupancy: What could be the nature of the data occupying the space from 0x9000 to 0xCA10? Is it safe to ignore or reallocate this space for other uses? Is it because of this data that the saved data between 0x9000 to 0xA500 got corrupted?

I appreciate guidance on these points to ensure reliable data storage and retention on my ESP32-S2-Solo-U device.

[ESP_Sprite]

No, sorry. I gave you the practical answer: you use the NVS library for NVS partitions, and you allocate your own data partition for anything else. Assuming any type of behaviour from the internals of the NVS library is dangerous, and I have no desire to try to figure out why you abusing it fails in a certain way.