ESP32 Forum

Posted: **Tue Oct 06, 2020 3:02 pm**

Hi,

I'm new to ESP32 and Arduino, but I know a few bits about programing Atmel micros.
In my ESP32 project I'm trying to retrieve some data from NVS using the preferences library. If the data (or rather the key for the data) doesn't exist, I set a default value and save that value to preferences.
When I reboot, I expect the key and the associated data to exist on NVS and to load without problems. But that doesn't happen.

Here's my code (I left out the statements which are irrelevant to using preferences):

Code: Select all

char * registration;
char * pin;
char * uuid;

  prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace
  if(prefs.getString("Reg", registration, 8) == 0) {            // get bike registration ID
    registration = registration_DEFAULT;                        // assign default if not defined
  }

  if(prefs.getString("PIN", pin, 8) == 0) {                     // get PIN code if it exists
    Serial.println("Assigning default PIN: 123456");
    pin = "123456";                                             // assign default if not defined
   
   if(prefs.putString("PIN", pin) == strlen(pin)) {            // save to NVS
     Serial.println("Success saving PIN to NVS");
   }
  }
  else {
    Serial.print("Loaded PIN from NVS - PIN = ");
    Serial.println(pin);
  }

 
  if(prefs.getString("UUID", uuid, 40) == 0) {                  // get device UUID if it exists
    Serial.println("Assigning default UUID: 00000000-0000-0000-0000-000000000000");
    uuid = "00000000-0000-0000-0000-000000000000";              // assign Nil/Empty UUID if not defined
    
    if(prefs.putString("UUID", uuid) == strlen(uuid)) {
       Serial.println("Success saving UUID to NVS");
    }
  }
  else {
    Serial.print("Loaded UUID from NVS - UUID = ");
    Serial.println(uuid);
  }  
  prefs.end();                                                  // close Auth namespace.

Here is the corresponding output on Serial Monitor:

ets Jun 8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5816
entry 0x400806ac
Assigning default PIN: 123456
Success saving PIN to NVS
Assigning default UUID: 00000000-0000-0000-0000-000000000000
Success saving UUID to NVS
Registration = MABC123
Success to initialize controller
Success to initialize bluedroid
Success to enable bluedroid
Success to initialize SerialBT using bt_name
Bluetooth interface enabled, please pair with your smartphone and connect
Device address = FC:F5:C4:01:C0:6A

After rebooting, I expect that serial monitor shows like "Loaded PIN from NVS - PIN = ...", etc, but that doesn't happen,
which tells me that my default values didn't make it to NVS.

Posted: **Tue Oct 06, 2020 3:37 pm**

Use another variable to check valid NVS content.
read this, if variable != 0x5A (or any fix value) then write this variable to 0x5A in NVS, and set all variable to default.
The NVS content don't guarantee the default empty value 0x0!

Posted: **Tue Oct 06, 2020 4:36 pm**

Hi pipi61,

I don't see why the actual value (zero or not) would make a diference, but I tried your suggestion.
Code:

Code: Select all

  prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace
  if(prefs.getString("Reg", registration, 8) == 0) {            // get bike registration ID
    Serial.println("Assigning default registration: MABC123");
    registration = "MABC123";                                   // assign default if not defined
    prefs.putString("Reg", registration);   
    Serial.println("saved registration item to NVS");  
  }
  else {
    Serial.print("Loaded registration from NVS - registration = ");
    Serial.println(registration);
  }

The first time I get this:

...
Assigning default registration: MABC123
saved registration item to NVS
...

After rebooting ESP32 I keep getting the same output, which cofirms dat my data is not saved.
If this is not what you meant in your reply, maybe you can suggest a code snippet in line with what I'm trying to achieve?

tried it in a slightly different way, but the result is the same:

Code: Select all

  prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace
  prefs.getString("Reg", registration, 8);                      // get bike registration ID
  if(registration != "MABC123") {
    Serial.println("Assigning default registration: MABC123");
    registration = "MABC123";                                   // assign default if not defined
    prefs.putString("Reg", registration);   
    Serial.println("saved registration item to NVS");  
  }
  else {
    Serial.print("Loaded registration from NVS - registration = ");
    Serial.println(registration);
  }

Posted: **Tue Oct 06, 2020 5:16 pm**

You aren't allocating any space in the char* to hold the result. Try initializing with an fixed length array instead of a pointer:

Code: Select all

char pin[8];

Posted: **Wed Oct 07, 2020 9:55 am**

You aren't allocating any space in the char* to hold the result. Try initializing with an fixed length array instead of a pointer:

Yes, that solves the problem. Thank you.

A few notes on the side:

I notice that NVS is not wiped when I upload a new sketch. The data from my previous sketch are still there. Is there a way to configure the upload to wipe NVS or do I have to do this in my sketch?

The preferences feature is not very robust. When I define my string as char ptr and initialize it to an empty string (char * test = ""), then prefs will cause ESP32 to crash when I try to retrieve something from NVS into that char ptr.
Exception decoder results from the crash:

Code: Select all

Guru Meditation Error: Core  1 panic'ed (LoadStoreError). Exception was unhandled.
Core 1 register dump:
PC      : 0x4000c2a3  PS      : 0x00060b30  A0      : 0x800d7465  A1      : 0x3ffb1e00  
A2      : 0x3f400fd5  A3      : 0x3ffb1e4d  A4      : 0x00000007  A5      : 0x3f400fd5  
A6      : 0x0000004d  A7      : 0x3ffb1e2c  A8      : 0x800d6e04  A9      : 0x3ffb1dd0  
A10     : 0x00000000  A11     : 0x3ffb1e4c  A12     : 0x00000020  A13     : 0x0000ff00  
A14     : 0x00ff0000  A15     : 0xff000000  SAR     : 0x0000001c  EXCCAUSE: 0x00000003  
EXCVADDR: 0x3f400fd5  LBEG    : 0x40001609  LEND    : 0x4000160d  LCOUNT  : 0x00000000  

Backtrace: 0x4000c2a3:0x3ffb1e00 0x400d7462:0x3ffb1e10 0x400d621e:0x3ffb1e90 0x400d53b2:0x3ffb1ef0 0x400d57e5:0x3ffb1f30 0x400d0dfd:0x3ffb1f50 0x400d0caa:0x3ffb1f80 0x400d1dfb:0x3ffb1fb0 0x40088215:0x3ffb1fd0

Code: Select all

PC: 0x4000c2a3
EXCVADDR: 0x3f400fd5

Decoding stack results
0x400d7462: nvs::Page::readItem(unsigned char, nvs::ItemType, char const*, void*, unsigned int, unsigned char, nvs::VerOffset) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_page.cpp line 294
0x400d621e: nvs::Storage::readItem(unsigned char, nvs::ItemType, char const*, void*, unsigned int) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_storage.cpp line 465
0x400d53b2: nvs_get_str_or_blob(nvs_handle, nvs::ItemType, char const*, void*, size_t*) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_api.cpp line 515
0x400d57e5: nvs_get_str(nvs_handle, char const*, char*, size_t*) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/nvs_flash/src/nvs_api.cpp line 520
0x400d0dfd: Preferences::getString(char const*, char*, unsigned int) at C:\Users\Gilbert\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\Preferences\src\Preferences.cpp line 421
0x400d0caa: setup() at C:\Users\Gilbert\Documents\Arduino\ESP32\Code\sketch_PrefsTest/sketch_PrefsTest.ino line 22
0x400d1dfb: loopTask(void*) at C:\Users\Gilbert\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32\main.cpp line 14
0x40088215: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143

When I (deliberately for this test) define the size of my char[] to be less than the length of the actual string assigned to that char[] (in my code that would be "registration"), then Preferences::getString(const char* key, char* value, const size_t maxLen) will retrieve the data and store it in the (undersized) array - I suspect this results in some kind of buffer overflow ? -. As a next step I change the string to something else and try to save the changed string back to NVS. The save function fails (returns 0, which is the length of the saved string). Kinda strange because the putchar function looks at the length of the new string and (I suppose) doesn't know what the size of the original char[] is?

Code: Select all

#include <Preferences.h>                                        // ESP32 variant of Arduino EEPROM library

Preferences prefs;                                              // define preferences  

char registration[5];

void setup() {
  // put your setup code here, to run once:

#include <Preferences.h>                                        // ESP32 variant of Arduino EEPROM library

Preferences prefs;                                              // define preferences  

char registration[5];						// undersized array

void setup() {
  // put your setup code here, to run once:

  Serial.begin(115200);                                          // Arduino serial port for debugging @ 115.2 kBaud
  while (!Serial.available());                                   // wait for serial port user input
  char input = Serial.read();                                   // read the first char

  while (Serial.available()) {
    Serial.read();                                              // flush serial input buffer
  }
  
  prefs.begin("Auth", false);                                   // ESP32 Flash storage, Auth namespace, R/W

  Serial.print("Initial value registration  = ");
  Serial.println(registration);  
  Serial.print("strlen = ");
  Serial.println(strlen(registration));
    
  size_t result1 = prefs.getString("Reg", registration, 8);
  Serial.print("result1 = ");
  Serial.println(result1, DEC);
  
  if(result1 > 0) {                                             // if data found on NVS
    Serial.print("Retrieved registration from NVS - registration = ");
    Serial.println(registration);
    Serial.print("strlen = ");
    Serial.println(strlen(registration));
  }
  else {
    Serial.println("registration not found on NVS");
  }
  
  if(strcmp(registration, "MABC123") == 0) {                    // if string is "MABC123"
      strncpy(registration, "MXYZ321", 10);                     // change to "MXYZ321"      
  }
  else {
    strncpy(registration, "MABC123", 10);                       // change to "MXYZ321"         
  }

  Serial.print("Changed registration to ");
  Serial.println(registration);  
  Serial.print("strlen = ");
  Serial.println(strlen(registration));
  
  size_t savedSize = prefs.putString("Reg", registration);                         // save the changed string
  Serial.print("saved nr of chars = ");
  Serial.println(savedSize, DEC);
  
  prefs.end();                                                  // close Auth namespace.
}

Output to serial:

Code: Select all

Initial value registration  = 
strlen = 0
result1 = 8
Retrieved registration from NVS - registration = MABC123
strlen = 7
Changed registration to MXYZ321
strlen = 7
saved nr of chars = 0
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5816
entry 0x400806ac
Initial value registration  = 
strlen = 0
result1 = 8
Retrieved registration from NVS - registration = MABC123
strlen = 7
Changed registration to MXYZ321
strlen = 7
saved nr of chars = 0

Posted: **Wed Oct 07, 2020 1:49 pm**

Preferences is just a thin wrapper over NVS. NVS is non-volatile storage. As such, it is intended to persist, and not be regularly wiped. It stores some system calibration information, along with your data, so if you want to clear a namespace, you should clear just that namespace. If you want to wipe the device, use esptool. https://github.com/espressif/esptool#er ... ase-region
The get and putString functions in Arduino were built with the idea that most people will use dynamic String variables for them, so there is not a mechanism to determine the size of the stored string. The getBytes function exposes the mechanism from NVS where passing a NULL value for length will return the needed size. There is also simply a getBytesLength function you can use for that. If you won't know the maximum length of a stored string, use getBytes instead.

Posted: **Wed Oct 07, 2020 3:15 pm**

Got it. Thank you for the assistance.
Python is not my cup of tea

.
I'll add a little piece of code to wipe the old namespaces before I create a final release of my code.

Posted: **Wed Oct 07, 2020 4:57 pm**

Gilbert wrote: ↑
Wed Oct 07, 2020 9:55 am

The preferences feature is not very robust. When I define my string as char ptr and initialize it to an empty string (char * test = ""), then prefs will cause ESP32 to crash when I try to retrieve something from NVS into that char ptr.

Did you try Ibernstone's solution?:
"You aren't allocating any space in the char* to hold the result. Try initializing with an fixed length array instead of a pointer."

Gilbert wrote: ↑
Wed Oct 07, 2020 9:55 am

When I (deliberately for this test) define the size of my char[] to be less than the length of the actual string assigned to that char[] (in my code that would be "registration"), then Preferences::getString(const char* key, char* value, const size_t maxLen) will retrieve the data and store it in the (undersized) array

You are fetching up to 8 bytes:

Code: Select all

size_t result1 = prefs.getString("Reg", registration, 8);

(And later with strncpy too.)
Unfortunately, it doesn't crash there, but all bets are off after this point.

Posted: **Thu Oct 08, 2020 6:43 am**

Did you try Ibernstone's solution?:

Yes, and it works for me.

You are fetching up to 8 bytes:

Yes, and that's odd. When the char * is not initialized, it's crashing, when the char[] is too small it fails silently.

Anyway, lessons learned. I know now that I have to be very careful when using preferences.

ESP32 Forum

Saving preferences doesn't work

Saving preferences doesn't work

Re: Saving preferences doesn't work

Re: Saving preferences doesn't work

Re: Saving preferences doesn't work

Re: Saving preferences doesn't work

Re: Saving preferences doesn't work

Re: Saving preferences doesn't work

Re: Saving preferences doesn't work

Re: Saving preferences doesn't work