flashing custom bootloader makes nvs non-persistant

[Hrox2504]

Hi everyone,

I have come across a very weird behaviour (or potentially bug?) regarding the bootloader. I have been stuck on this the last few days. I would be very grateful for any help.

I was able to break down the situation to the following: I have a platformIO project that uses the ESP-Arduino framework set up for an ESP32S3 (16MB). The code (see below) simply increments an uint32_t count every second and saves that value to non-volatile-storage via the <Preferences.h> library. When the application starts, it loads the value, i.e. continues counting where it stopped when switched off:

Code: [Select all] [Expand/Collapse]

1. #include <Arduino.h>
2. #include <Preferences.h>
3.  
4. Preferences preferences;
5. uint32_t count;
6.  
7. void setup() {
8.     Serial.begin(115200);
9.     delay(2000);
10.     Serial.println("Calling preferences.begin()");
11.     bool status = preferences.begin("my_storage", false);
12.     if (status) Serial.println("    returned true");
13.     else        Serial.println("    returned false");
14.    
15.     Serial.println("Attempting to load count");
16.     if (0 == preferences.getBytes("count", &count, sizeof(count))) {
17.         Serial.println("no count set -> starting at 0");
18.         count = 0;
19.     }
20. }
21.  
22. void loop() {
23.     Serial.printf("Count: %d\n", count);
24.     count++;
25.     if (0 == preferences.putBytes("count", &count, sizeof(count))) {
26.         Serial.println("    did not put any bytes");
27.     }
28.     delay(1000);
29. }

GeSHi © Codebox Plus Extension

When I flash this code using '$pio run -t upload' everything works as expected.

Next I use a custom partition table (partitions.csv):
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
test, app, test, 0x20000, 0x180000
app0, app, ota_0, , 0x300000
coredump, data, coredump, , 64k

and add 'board_build.partitions = partitions.csv' in the platformio.ini. Flashing with '$pio run -t upload' and still everything works as expected. Note, that the test partition is simply not used.

The next step is the bad one: I now switch to an esp-idf project set up for the esp32s3, the same partition table and otherwise default settings. All I do is '$idf.py bootloader && idf.py bootloader-flash' onto the already-platformio-flashed esp32s3. Now the app still boots and runs, but on restart it does not find the 'count' key in nvs and creates a new one every time (see serial output below):

Code: Select all

Calling preferences.begin()
    returned true
Attempting to load count
[  2734][E][Preferences.cpp:503] getBytesLength(): nvs_get_blob len fail: count NOT_FOUND
no count set -> starting at 0
Count: 0
Count: 1
Count: 2
Count: 3
Count: 4

What I do not understand, is, why and how the bootloader affects the nvs behaviour. For anyone interested, the goal is to install a second app in the test partition that works as updater and can be booted by holding the power button. The main app is written in the platformIO ecosystem and therefore we are somewhat stuck to platformIO. Unfortunately, it does expose the "boot test partition on GPIOn input" functionality. None of this logic is implemented in this minimal example however.

I have also tested doing all the flashing using the esp-idf directly by using the parttool.py to flash the firmware.bin producted by platformIO.
$idf.py bootloader && idf.py bootloader-flash
$idf.py partition-table && idf.py partition-table-flash
$parttool.py ...

Best,
Hrox

[lbernstone]

I remember there were problems like this with the S3 when the OPI models first came out, so it likely has to do with the flash speed settings. I'd recommend you use the lib-builder to generate the bootloader so that you are starting from the same sdkconfig as the arduino-esp32 framework, and just change the CONFIG entries that are critical to your process.

[Hrox2504]

Thank you for your answer! I have never worked with the lib-builder before, but will look into it.