Accessing NVS/FAT from task where stack is in external RAM

[johboh]

Hi!
I'm using ESP32-S2 with an external 2MB PSRAM.
In my program I have various tasks, all of which where I allocate the task stack in external RAM (using CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY and xTaskCreateStatic() with EXT_RAM_BSS_ATTR StackType_t _main_event_stack[MAIN_EVENT_TASK_STACK]).

In the main task, I setup NVS as well as mount a FAT file system in flash.

Using ESP-IDF v4.4.4, I can read and write to NVS and the the FAT file system from the various tasks.
But with v.5.1.2, I can only do this from the main thread. If I try to read a file from FAT in a thread with an external allocated task stack, it just reboot. I'm unable to catch any stack trace as the USB/CDC just disconnect. I also tried setting CONFIG_ESP_SYSTEM_PANIC_PRINT_HALT to but to no avail. It also says in the doc that core dumps doesn't work with external ram allocated task stacks.

For me, its a bit unclear from the documentation if reading and writing to flash from a task with external stack is allowed, or it I'm just in luck with v4.4.4.

So my question is, should this be allowed, and if so, what might I be missing with v5.1.2?

Regards,
Johan

[pacucha42]

Hi @johboh,
that sounds weird - accessing file-systems should be independent on specific task. Could you share your code, please? The critical parts would be sufficient, if you cannot share whole application (or if it's too big). I'll take a look asap

Thank you for the report!

[johboh]

I made an example where I can reproduce the issue:

Code: [Select all] [Expand/Collapse]

1. #include "esp_log.h"
2. #include "nvs.h"
3. #include "nvs_flash.h"
4. #include <freertos/FreeRTOS.h>
5. #include <freertos/task.h>
6. #include <string>
7.  
8. #define TAG "MAIN"
9. #define NVS_STORAGE "storage"
10. #define NVS_READ_WRITE_TASK_STACK 8192
11.  
12. nvs_handle_t _nvs_handle;
13.  
14. // ################# NVS setup, read and write ####################
15.  
16. bool setupNvs() {
17.   ESP_LOGI(TAG, "Initialize NVS");
18.   // Initialize NVS
19.   esp_err_t err = nvs_flash_init();
20.   if (err == ESP_ERR_NVS_NO_FREE_PAGES || err == ESP_ERR_NVS_NEW_VERSION_FOUND) {
21.     ESP_LOGE(TAG, "Erasing nvs (%s)", esp_err_to_name(err));
22.     ESP_ERROR_CHECK(nvs_flash_erase());
23.     err = nvs_flash_init();
24.   }
25.   ESP_ERROR_CHECK(err);
26.  
27.   err = nvs_open(NVS_STORAGE, NVS_READWRITE, &_nvs_handle);
28.   if (err != ESP_OK) {
29.     ESP_LOGE(TAG, "Failed to open NVS on storage (%s)", esp_err_to_name(err));
30.     return false;
31.   }
32.  
33.   return true;
34. }
35.  
36. bool commit() {
37.   esp_err_t err = nvs_commit(_nvs_handle);
38.   if (err != ESP_OK) {
39.     ESP_LOGE(TAG, "Failed to commit NVS (%s)", esp_err_to_name(err));
40.     return false;
41.   }
42.   return true;
43. }
44.  
45. uint8_t getU8(const std::string &key) {
46.   uint8_t r = 0;
47.   esp_err_t err = nvs_get_u8(_nvs_handle, key.c_str(), &r);
48.   if (err != ESP_OK) {
49.     ESP_LOGE(TAG, "Failed to get u8 from NVS with key %s (%s)", key.c_str(), esp_err_to_name(err));
50.     return 0;
51.   }
52.   return r;
53. }
54. void setU8(const std::string &key, const uint8_t value) {
55.   esp_err_t err = nvs_set_u8(_nvs_handle, key.c_str(), value);
56.   if (err != ESP_OK) {
57.     ESP_LOGE(TAG, "Failed to set string to NVS with key %s (%s)", key.c_str(), esp_err_to_name(err));
58.   } else {
59.     commit();
60.   }
61. }
62.  
63. // ################# Task reading and writing NVS from external task stack ####################
64.  
65. StaticTask_t _task_buffer;
66. EXT_RAM_ATTR StackType_t _stack[NVS_READ_WRITE_TASK_STACK];
67.  
68. void nvs_task(void *pvParams) {
69.   char a = 100;
70.   while (1) {
71.     ESP_LOGI(TAG, "About to write to NVS from external task");
72.     vTaskDelay(1000 / portTICK_PERIOD_MS);
73.     setU8("key", ++a); // on 5.1.2, this make the device restart
74.     ESP_LOGI(TAG, "About to read from NVS from external task");
75.     vTaskDelay(1000 / portTICK_PERIOD_MS);
76.     auto r = getU8("key");
77.     ESP_LOGI(TAG, "Got value: %d", r);
78.     vTaskDelay(1000 / portTICK_PERIOD_MS);
79.   }
80. }
81.  
82. // ################# Main ####################
83.  
84. extern "C" {
85. void app_main();
86. }
87.  
88. void app_main(void) {
89.  
90.   vTaskDelay(2000 / portTICK_PERIOD_MS);
91.  
92.   if (!setupNvs()) {
93.     ESP_LOGE(TAG, "Failed to setup NVS");
94.     while (1) {
95.       vTaskDelay(1000 / portTICK_PERIOD_MS);
96.     }
97.   }
98.  
99.   // Setup OK! create task.
100.   xTaskCreateStatic(&nvs_task, "nvs_task", NVS_READ_WRITE_TASK_STACK, NULL, 15, _stack, &_task_buffer);
101.  
102.   while (1) {
103.     vTaskDelay(4000 / portTICK_PERIOD_MS);
104.     ESP_LOGI(TAG, "Hello from main, I'm fine still.");
105.   }
106. }
107.  

GeSHi © Codebox Plus Extension

sdkconfig.defaults looks like this:

Code: Select all

CONFIG_ESPTOOLPY_FLASHMODE_QIO=y
CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y
CONFIG_ESPTOOLPY_AFTER_NORESET=y
CONFIG_ESP_CONSOLE_USB_CDC=y
CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y
CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY=y
CONFIG_SPIRAM_USE_CAPS_ALLOC=y
CONFIG_ESP32S2_SPIRAM_SUPPORT=y

The output when using v4.4.4 (don't get any boot info here for some reason):

Code: Select all

I (I (4494) MAIN: Initialize NVS: generic
I (4514) MAIN: About to write to NVS from external task
I (5524) MAIN: About to read from NVS from external task
I (6534) MAIN: Got value: 101

And when using v5.1.2:

Code: Select all

I (36) boot: ESP-IDF v5.1.2-dirty 2nd stage bootloader
I (37) boot: compile time Jan 23 2024 18:18:03
I (37) boot: chip revision: v0.0
I (37) qio_mode: Enabling default flash chip QIO
I (37) boot.esp32s2: SPI Speed      : 80MHz
I (38) boot.esp32s2: SPI Mode       : QIO
I (38) boot.esp32s2: SPI Flash Size : 4MB
I (39) boot: Enabling RNG early entropy source...
I (39) boot: Partition Table:
I (39) boot: ## Label            Usage          Type ST Offset   Length
I (40) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (41) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (43) boot:  2 factory          factory app      00 00 00010000 00100000
I (44) boot: End of partition table
I (44) esp_image: segment 0: paddr=00010020 vaddr=3f000020 size=0b9bch ( 47548) map
I (54) esp_image: segment 1: paddr=0001b9e4 vaddr=3ffbfb90 size=01984h (  6532) load
I (56) esp_image: segment 2: paddr=0001d370 vaddr=40024000 size=02ca8h ( 11432) load
I (60) esp_image: segment 3: paddr=00020020 vaddr=40080020 size=18650h ( 99920) map
I (77) esp_image: segment 4: paddr=00038678 vaddr=40026ca8 size=08ee8h ( 36584) load
I (93) boot: Loaded app from partition at offset 0x10000
I (94) boot: Disabling RNG early entropy source...
I (526) main_task: Started on CPU0
I (526) main_task: Calling app_main()
I (2526) MAIN: Initialize NVS
I (2536) MAIN: About to write to NVS from external task
None
Waiting for the device to reconnect..

[MicroController]

So my question is, should this be allowed

Interesting phrasing
I wouldn't expect this to work at all. External RAM is accessed through the cache, and the cache is disabled when/while writing to flash, making (a stack in) external RAM temporarily inaccessible.
I guess you could be lucky if the code/compiler chooses not to access the stack at some point during the write operation, but this could change at any time with any modification of the code/compiler/optimization settings/...

[johboh]

Interesting phrasing

Yes sorry, what I meant to ask was if accessing NVS/FAT from task where stack is in external RAM is something that is expected to work or not, given that it do work on 4.4.4 and not on 5.1.2. I have a project with several task with stack in external RAM, all of which are reading/writing NVS and flash FAT, and its running 24/7 without any issues. This project is also under active development so its recompiled from time to time. I have verified that the the memory is in fact allocated in external RAM. But the same project (and the example above) does not work on 5.1.2. As it worked in 4.4.4 it lead me to believe that it would also work in 5.1.2 (now when I migrated my project to v5 from v4).

If this should not work at all, I'm curious to know why it works, and consistently so, in 4.4.4.

[MicroController]

I'm curious to know why it works, and consistently so, in 4.4.4.

Very valid question. I don't know the answer, but a reason could be that, e.g., the flash access code in the IDF (and/or the gcc version...) changed in a way that makes the compiler now access the stack in places it previously didn't.

Maybe someone knowledgabe can chime in and answer the original question as to whether flash writes from external-RAM-stack tasks is supported in any way.

[ESP_rrtandler]

The limitations of external stack are described for IDF 5.1 in this section of documentation https://docs.espressif.com/projects/esp ... strictions
As NVS uses flash during write, there might be a collision between disabled flash cache and need to access stack located in external RAM.