NVS Blob operations leaking memory

[tommybazar]

After spending some time looking for a slow, but persistent memory leakage problem in our code, I found that nvs_set_blob allocates some overhead memory that is not freed on nvs_erase_key, nor on nvs_erase_all nor on handle close. Other nvs_setXXX functions also allocate this memory, but erasing the given key does free it.

nvs_set_str is does something similar, but it does free about half of the memory it allocates...

Below is a small program demonstrating this.

We're using release v3.0 from January 16th (commit e7dc749). None of the commits from last month seem to have touched the NVS.

Code: Select all

#include <iostream>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <nvs.h>
#include <sstream>
#include "esp_system.h"
#include <esp_log.h>
#include <nvs_flash.h>

#define MAIN_TAG "main"

std::string makeFileName(uint32_t index) {
  std::ostringstream outStr;
  outStr << "testname_" << index;
  return outStr.str();
}

extern "C" void app_main() {

  uint8_t buffer[255] = {0xFF};
  uint8_t buffer2[255] = {0x00};

  nvs_flash_init();

  esp_err_t error = 0;
  nvs_handle nvsHandle;
  nvs_open("test2", NVS_READWRITE, &nvsHandle);

  // Set blobs
  for (int i = 0; i < 100; i++) {
    error = nvs_set_blob(nvsHandle, makeFileName(i).c_str(), buffer, 255);
    nvs_commit(nvsHandle);
    ESP_LOGI(MAIN_TAG, "set blob @%d, error = %d, heap = %d", i, error, esp_get_free_heap_size());
    usleep(10);
  }

  // Get blobs to prove they're really there
  for (int i = 0; i < 100; i++) {
    uint32_t length = 256;
    error = nvs_get_blob(nvsHandle, makeFileName(i).c_str(), buffer2, &length);
    nvs_commit(nvsHandle);
    ESP_LOGI(MAIN_TAG, "get blob @%d, error = %d, heap = %d", i, error, esp_get_free_heap_size());
    usleep(10);
  }

  // Erase blobs to see them not giving back memory
  for (int i = 0; i < 100; i++) {
    error = nvs_erase_key(nvsHandle, makeFileName(i).c_str());
    nvs_commit(nvsHandle);
    ESP_LOGI(MAIN_TAG, "rem blob @%d, error = %d, heap = %d", i, error, esp_get_free_heap_size());
    usleep(10);
  }

  // Get blobs with error to prove they're really NOT there
  for (int i = 0; i < 100; i++) {
    uint32_t length = 0;
    error = nvs_get_blob(nvsHandle, makeFileName(i).c_str(), buffer2, &length);
    nvs_commit(nvsHandle);
    ESP_LOGI(MAIN_TAG, "get blob @%d, error = %d, heap = %d", i, error, esp_get_free_heap_size());
    usleep(10);
  }

  nvs_erase_all(nvsHandle);
  ESP_LOGI(MAIN_TAG, "2 Heap after delete all %d", esp_get_free_heap_size());
  nvs_close(nvsHandle);
  ESP_LOGI(MAIN_TAG, "2 Heap after close %d", esp_get_free_heap_size());
  nvs_open("test2", NVS_READWRITE, &nvsHandle);
  ESP_LOGI(MAIN_TAG, "2 Heap after open %d", esp_get_free_heap_size());

  // Add ints to see them consume heap.
  for (int i = 0; i < 100; i++) {
    error = nvs_set_i32(nvsHandle, makeFileName(i).c_str(), 1);
    nvs_commit(nvsHandle);
    ESP_LOGI(MAIN_TAG, "set int @%d, error = %d, heap = %d", i, error, esp_get_free_heap_size());
    usleep(10);
  }

  // Remove them to see them give it back.
  for (int i = 0; i < 100; i++) {
    error = nvs_erase_key(nvsHandle, makeFileName(i).c_str());
    nvs_commit(nvsHandle);
    ESP_LOGI(MAIN_TAG, "rem int @%d, error = %d, heap = %d", i, error, esp_get_free_heap_size());
    usleep(10);
  }

  nvs_close(nvsHandle);
  usleep(10000000);
}

Cheers,
Tomas.

[ESP_Sprite]

Can you do us a favour and make an issue for this on the Github project? You can do this here. This way we can track progress. Keep in mind that we may take a while because of Chinese new years.

[ESP_Angus]

For anyone following on at home, the discussion continues here:
https://github.com/espressif/esp-idf/issues/1642