Saving preferences doesn't work

[Gilbert]

Hi,

I'm new to ESP32 and Arduino, but I know a few bits about programing Atmel micros.
In my ESP32 project I'm trying to retrieve some data from NVS using the preferences library. If the data (or rather the key for the data) doesn't exist, I set a default value and save that value to preferences.
When I reboot, I expect the key and the associated data to exist on NVS and to load without problems. But that doesn't happen.

Here's my code (I left out the statements which are irrelevant to using preferences):

Code: Select all

char * registration;
char * pin;
char * uuid;

  prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace
  if(prefs.getString("Reg", registration, 8) == 0) {            // get bike registration ID
    registration = registration_DEFAULT;                        // assign default if not defined
  }

  if(prefs.getString("PIN", pin, 8) == 0) {                     // get PIN code if it exists
    Serial.println("Assigning default PIN: 123456");
    pin = "123456";                                             // assign default if not defined
   
   if(prefs.putString("PIN", pin) == strlen(pin)) {            // save to NVS
     Serial.println("Success saving PIN to NVS");
   }
  }
  else {
    Serial.print("Loaded PIN from NVS - PIN = ");
    Serial.println(pin);
  }

 
  if(prefs.getString("UUID", uuid, 40) == 0) {                  // get device UUID if it exists
    Serial.println("Assigning default UUID: 00000000-0000-0000-0000-000000000000");
    uuid = "00000000-0000-0000-0000-000000000000";              // assign Nil/Empty UUID if not defined
    
    if(prefs.putString("UUID", uuid) == strlen(uuid)) {
       Serial.println("Success saving UUID to NVS");
    }
  }
  else {
    Serial.print("Loaded UUID from NVS - UUID = ");
    Serial.println(uuid);
  }  
  prefs.end();                                                  // close Auth namespace.

Here is the corresponding output on Serial Monitor:

ets Jun 8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5816
entry 0x400806ac
Assigning default PIN: 123456
Success saving PIN to NVS
Assigning default UUID: 00000000-0000-0000-0000-000000000000
Success saving UUID to NVS
Registration = MABC123
Success to initialize controller
Success to initialize bluedroid
Success to enable bluedroid
Success to initialize SerialBT using bt_name
Bluetooth interface enabled, please pair with your smartphone and connect
Device address = FC:F5:C4:01:C0:6A

After rebooting, I expect that serial monitor shows like "Loaded PIN from NVS - PIN = ...", etc, but that doesn't happen,
which tells me that my default values didn't make it to NVS.

[pipi61]

Use another variable to check valid NVS content.
read this, if variable != 0x5A (or any fix value) then write this variable to 0x5A in NVS, and set all variable to default.
The NVS content don't guarantee the default empty value 0x0!

[Gilbert]

Hi pipi61,

I don't see why the actual value (zero or not) would make a diference, but I tried your suggestion.
Code:

Code: Select all

  prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace
  if(prefs.getString("Reg", registration, 8) == 0) {            // get bike registration ID
    Serial.println("Assigning default registration: MABC123");
    registration = "MABC123";                                   // assign default if not defined
    prefs.putString("Reg", registration);   
    Serial.println("saved registration item to NVS");  
  }
  else {
    Serial.print("Loaded registration from NVS - registration = ");
    Serial.println(registration);
  }

The first time I get this:

...
Assigning default registration: MABC123
saved registration item to NVS
...

After rebooting ESP32 I keep getting the same output, which cofirms dat my data is not saved.
If this is not what you meant in your reply, maybe you can suggest a code snippet in line with what I'm trying to achieve?

tried it in a slightly different way, but the result is the same:

Code: Select all

  prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace
  prefs.getString("Reg", registration, 8);                      // get bike registration ID
  if(registration != "MABC123") {
    Serial.println("Assigning default registration: MABC123");
    registration = "MABC123";                                   // assign default if not defined
    prefs.putString("Reg", registration);   
    Serial.println("saved registration item to NVS");  
  }
  else {
    Serial.print("Loaded registration from NVS - registration = ");
    Serial.println(registration);
  }

[lbernstone]

You aren't allocating any space in the char* to hold the result. Try initializing with an fixed length array instead of a pointer:

Code: Select all

char pin[8];

[Gilbert]

You aren't allocating any space in the char* to hold the result. Try initializing with an fixed length array instead of a pointer:

Yes, that solves the problem. Thank you.

A few notes on the side:

• I notice that NVS is not wiped when I upload a new sketch. The data from my previous sketch are still there. Is there a way to configure the upload to wipe NVS or do I have to do this in my sketch?

• The preferences feature is not very robust. When I define my string as char ptr and initialize it to an empty string (char * test = ""), then prefs will cause ESP32 to crash when I try to retrieve something from NVS into that char ptr.
  Exception decoder results from the crash:

  Code: Select all

  Guru Meditation Error: Core  1 panic'ed (LoadStoreError). Exception was unhandled.
  Core 1 register dump:
  PC      : 0x4000c2a3  PS      : 0x00060b30  A0      : 0x800d7465  A1      : 0x3ffb1e00  
  A2      : 0x3f400fd5  A3      : 0x3ffb1e4d  A4      : 0x00000007  A5      : 0x3f400fd5  
  A6      : 0x0000004d  A7      : 0x3ffb1e2c  A8      : 0x800d6e04  A9      : 0x3ffb1dd0  
  A10     : 0x00000000  A11     : 0x3ffb1e4c  A12     : 0x00000020  A13     : 0x0000ff00  
  A14     : 0x00ff0000  A15     : 0xff000000  SAR     : 0x0000001c  EXCCAUSE: 0x00000003  
  EXCVADDR: 0x3f400fd5  LBEG    : 0x40001609  LEND    : 0x4000160d  LCOUNT  : 0x00000000  
  
  Backtrace: 0x4000c2a3:0x3ffb1e00 0x400d7462:0x3ffb1e10 0x400d621e:0x3ffb1e90 0x400d53b2:0x3ffb1ef0 0x400d57e5:0x3ffb1f30 0x400d0dfd:0x3ffb1f50 0x400d0caa:0x3ffb1f80 0x400d1dfb:0x3ffb1fb0 0x40088215:0x3ffb1fd0
  

  Code: Select all

  PC: 0x4000c2a3
  EXCVADDR: 0x3f400fd5
  
  Decoding stack results
  0x400d7462: nvs::Page::readItem(unsigned char, nvs::ItemType, char const*, void*, unsigned int, unsigned char, nvs::VerOffset) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_page.cpp line 294
  0x400d621e: nvs::Storage::readItem(unsigned char, nvs::ItemType, char const*, void*, unsigned int) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_storage.cpp line 465
  0x400d53b2: nvs_get_str_or_blob(nvs_handle, nvs::ItemType, char const*, void*, size_t*) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_api.cpp line 515
  0x400d57e5: nvs_get_str(nvs_handle, char const*, char*, size_t*) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_api.cpp line 520
  0x400d0dfd: Preferences::getString(char const*, char*, unsigned int) at C:\Users\Gilbert\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\Preferences\src\Preferences.cpp line 421
  0x400d0caa: setup() at C:\Users\Gilbert\Documents\Arduino\ESP32\Code\sketch_PrefsTest/sketch_PrefsTest.ino line 22
  0x400d1dfb: loopTask(void*) at C:\Users\Gilbert\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32\main.cpp line 14
  0x40088215: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143
  
  

• When I (deliberately for this test) define the size of my char[] to be less than the length of the actual string assigned to that char[] (in my code that would be "registration"), then Preferences::getString(const char* key, char* value, const size_t maxLen) will retrieve the data and store it in the (undersized) array - I suspect this results in some kind of buffer overflow ? -. As a next step I change the string to something else and try to save the changed string back to NVS. The save function fails (returns 0, which is the length of the saved string). Kinda strange because the putchar function looks at the length of the new string and (I suppose) doesn't know what the size of the original char[] is?
  

  Code: Select all

  #include <Preferences.h>                                        // ESP32 variant of Arduino EEPROM library
  
  Preferences prefs;                                              // define preferences  
  
  char registration[5];
  
  void setup() {
    // put your setup code here, to run once:
  
  #include <Preferences.h>                                        // ESP32 variant of Arduino EEPROM library
  
  Preferences prefs;                                              // define preferences  
  
  char registration[5];						// undersized array
  
  void setup() {
    // put your setup code here, to run once:
  
    Serial.begin(115200);                                          // Arduino serial port for debugging @ 115.2 kBaud
    while (!Serial.available());                                   // wait for serial port user input
    char input = Serial.read();                                   // read the first char
  
    while (Serial.available()) {
      Serial.read();                                              // flush serial input buffer
    }
    
    prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace, R/W
  
    Serial.print("Initial value registration  = ");
    Serial.println(registration);  
    Serial.print("strlen = ");
    Serial.println(strlen(registration));
      
    size_t result1 = prefs.getString("Reg", registration, 8);
    Serial.print("result1 = ");
    Serial.println(result1, DEC);
    
    if(result1 > 0) {                                             // if data found on NVS
      Serial.print("Retrieved registration from NVS - registration = ");
      Serial.println(registration);
      Serial.print("strlen = ");
      Serial.println(strlen(registration));
    }
    else {
      Serial.println("registration not found on NVS");
    }
    
    if(strcmp(registration, "MABC123") == 0) {                    // if string is "MABC123"
        strncpy(registration, "MXYZ321", 10);                     // change to "MXYZ321"      
    }
    else {
      strncpy(registration, "MABC123", 10);                       // change to "MXYZ321"         
    }
  
    Serial.print("Changed registration to ");
    Serial.println(registration);  
    Serial.print("strlen = ");
    Serial.println(strlen(registration));
    
    size_t savedSize = prefs.putString("Reg", registration);                         // save the changed string
    Serial.print("saved nr of chars = ");
    Serial.println(savedSize, DEC);
    
    prefs.end();                                                  // close Auth namespace.
  }
  

  Output to serial:
  

  Code: Select all

  Initial value registration  = 
  strlen = 0
  result1 = 8
  Retrieved registration from NVS - registration = MABC123
  strlen = 7
  Changed registration to MXYZ321
  strlen = 7
  saved nr of chars = 0
  ets Jun  8 2016 00:22:57
  
  rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
  configsip: 0, SPIWP:0xee
  clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
  mode:DIO, clock div:1
  load:0x3fff0018,len:4
  load:0x3fff001c,len:1044
  load:0x40078000,len:8896
  load:0x40080400,len:5816
  entry 0x400806ac
  Initial value registration  = 
  strlen = 0
  result1 = 8
  Retrieved registration from NVS - registration = MABC123
  strlen = 7
  Changed registration to MXYZ321
  strlen = 7
  saved nr of chars = 0
  

[lbernstone]

Preferences is just a thin wrapper over NVS. NVS is non-volatile storage. As such, it is intended to persist, and not be regularly wiped. It stores some system calibration information, along with your data, so if you want to clear a namespace, you should clear just that namespace. If you want to wipe the device, use esptool. https://github.com/espressif/esptool#er ... ase-region
The get and putString functions in Arduino were built with the idea that most people will use dynamic String variables for them, so there is not a mechanism to determine the size of the stored string. The getBytes function exposes the mechanism from NVS where passing a NULL value for length will return the needed size. There is also simply a getBytesLength function you can use for that. If you won't know the maximum length of a stored string, use getBytes instead.

[Gilbert]

Got it. Thank you for the assistance.
Python is not my cup of tea .
I'll add a little piece of code to wipe the old namespaces before I create a final release of my code.

[boarchuz]

The preferences feature is not very robust. When I define my string as char ptr and initialize it to an empty string (char * test = ""), then prefs will cause ESP32 to crash when I try to retrieve something from NVS into that char ptr.

Did you try Ibernstone's solution?:
"You aren't allocating any space in the char* to hold the result. Try initializing with an fixed length array instead of a pointer."

When I (deliberately for this test) define the size of my char[] to be less than the length of the actual string assigned to that char[] (in my code that would be "registration"), then Preferences::getString(const char* key, char* value, const size_t maxLen) will retrieve the data and store it in the (undersized) array

You are fetching up to 8 bytes:

Code: Select all

size_t result1 = prefs.getString("Reg", registration, 8);

(And later with strncpy too.)
Unfortunately, it doesn't crash there, but all bets are off after this point.

[Gilbert]

Did you try Ibernstone's solution?:

Yes, and it works for me.

You are fetching up to 8 bytes:

Yes, and that's odd. When the char * is not initialized, it's crashing, when the char[] is too small it fails silently.

Anyway, lessons learned. I know now that I have to be very careful when using preferences.