readdir stackoverflow with SPIFFS

[davdav]

Hi Everybody,

I have a problem usign SPIFFS components. My scenario:

1.I have a partition which base_path is "/U"
2.I create some file in folders (I know, spiffs doesn't support folders but follow me) like:
/U/UT/foo1
/U/UT/foo2
/U/UE/bar

3.I do: opendir("/U") and readdir() returns me:
/UT/foo1
/UT/foo2
/UE/bar

4.I do: opendir("/U/UT") and readdir() returns me:
foo1
foo2

5.I do: opendir("/U/UE") and readdir() returns me:
bar

Everything is OK and I'm happy. Now, consider I have 100 (or 1000 files) "foo" in "folder" /U/UT:
/U/UT/foo1
/U/UT/foo2
...
/U/UT/foo100

If I do as in point 3. and 4. everything works. If I do as in point 5. the program crash for stack overflow.

Code: Select all

***ERROR*** A stack overflow in task main has been detected.
abort() was called at PC 0x400871f0 on core 0

Backtrace: 0x40087034:0x3ffb43e0 0x400871d7:0x3ffb4400 0x400871f0:0x3ffb4420 0x400840e7:0x3ffb4440 0x40085ea4:0x3ffb4460 0x40085e5a:0x00000000

Entering gdb stub now.
$T0b#e6


 xtensa-esp32-elf-gdb ./build/test_spiffs_crash.elf -b 115200 -ex 'target remote COM6'
GNU gdb (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a5) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-host_pc-mingw32 --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/test_spiffs_crash.elf...done.
Remote debugging using COM6
0x40087034 in invoke_abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:140
140             *((int *) 0) = 0;
(gdb) bt
#0  0x40087034 in invoke_abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:140
#1  0x400871da in abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:149
#2  0x400871f3 in vApplicationStackOverflowHook (xTask=0x3ffb6e94,
    pcTaskName=0x3ffb6ecc "main")
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:120
#3  0x400840ea in vTaskSwitchContext ()
    at C:/msys32/home/Davide/esp/esp-idf/components/freertos/tasks.c:2818
#4  0x40085ea7 in _frxt_dispatch ()
    at C:/msys32/home/Davide/esp/esp-idf/components/freertos\portasm.S:406
(gdb) list
135     #endif
136         while (1) {
137             if (esp_cpu_in_ocd_debug_mode()) {
138                 __asm__ ("break 0,0");
139             }
140             *((int *) 0) = 0;
141         }
142     }
143
144     void abort()

I tried to increase stack size to 10K but it doesn't work; there should be a problem with readdir function in SPIFFS when there are too many files. I have created a test project here:

https://github.com/davctv/test_spiffs_crash.git

which in practice do what I explained above and where you can find sdkconfig file I used.

Maybe I will open also an issue @loboris github account but if someone has any insight on this it will more than welcome.

Thanks

[Ritesh]

Hi Everybody,

I have a problem usign SPIFFS components. My scenario:

1.I have a partition which base_path is "/U"
2.I create some file in folders (I know, spiffs doesn't support folders but follow me) like:
/U/UT/foo1
/U/UT/foo2
/U/UE/bar

3.I do: opendir("/U") and readdir() returns me:
/UT/foo1
/UT/foo2
/UE/bar

4.I do: opendir("/U/UT") and readdir() returns me:
foo1
foo2

5.I do: opendir("/U/UE") and readdir() returns me:
bar

Everything is OK and I'm happy. Now, consider I have 100 (or 1000 files) "foo" in "folder" /U/UT:
/U/UT/foo1
/U/UT/foo2
...
/U/UT/foo100

If I do as in point 3. and 4. everything works. If I do as in point 5. the program crash for stack overflow.

Code: Select all

***ERROR*** A stack overflow in task main has been detected.
abort() was called at PC 0x400871f0 on core 0

Backtrace: 0x40087034:0x3ffb43e0 0x400871d7:0x3ffb4400 0x400871f0:0x3ffb4420 0x400840e7:0x3ffb4440 0x40085ea4:0x3ffb4460 0x40085e5a:0x00000000

Entering gdb stub now.
$T0b#e6


 xtensa-esp32-elf-gdb ./build/test_spiffs_crash.elf -b 115200 -ex 'target remote COM6'
GNU gdb (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a5) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-host_pc-mingw32 --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/test_spiffs_crash.elf...done.
Remote debugging using COM6
0x40087034 in invoke_abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:140
140             *((int *) 0) = 0;
(gdb) bt
#0  0x40087034 in invoke_abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:140
#1  0x400871da in abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:149
#2  0x400871f3 in vApplicationStackOverflowHook (xTask=0x3ffb6e94,
    pcTaskName=0x3ffb6ecc "main")
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:120
#3  0x400840ea in vTaskSwitchContext ()
    at C:/msys32/home/Davide/esp/esp-idf/components/freertos/tasks.c:2818
#4  0x40085ea7 in _frxt_dispatch ()
    at C:/msys32/home/Davide/esp/esp-idf/components/freertos\portasm.S:406
(gdb) list
135     #endif
136         while (1) {
137             if (esp_cpu_in_ocd_debug_mode()) {
138                 __asm__ ("break 0,0");
139             }
140             *((int *) 0) = 0;
141         }
142     }
143
144     void abort()

I tried to increase stack size to 10K but it doesn't work; there should be a problem with readdir function in SPIFFS when there are too many files. I have created a test project here:

https://github.com/davctv/test_spiffs_crash.git

which in practice do what I explained above and where you can find sdkconfig file I used.

Maybe I will open also an issue @loboris github account but if someone has any insight on this it will more than welcome.

Thanks

Hi,

Would you please provide me few information like on which ESP32 IDF you are working and which SPIFFS component you have used to test your SPIFFS Application example? I mean you have used standard SPIFFS component which is integrated into ESP32 IDF 3.0 or different one like loboris SPIFFS component?

So that it will be helpful for us to test same thing here like you have tested as we have tested it with few files and folders but big number of files like you have tested.

[davdav]

Hello Ritesh,

My esp-idf version is

Code: Select all

git describe
v3.2-dev-209-gbe81d2c

And I'm using SPIFFS included in esp-idf (not loboris github) which should be v0.3.7 of spiffs components.

Thanks.

[ESP_igrr]

There is an open PR on GitHub related to this issue: https://github.com/espressif/esp-idf/pull/2212. We haven't had time yet to check the issue and proposed fix, but you can try pulling the PR branch.

[davdav]

Hello ESP_igrr,

Thanks for the "good news" because it is very important for our project.

Do you have a time schedule to merge the proposed PR in master branch?

I will give a try for sure. I just have to understand how to download this specific commit ( I'm just a git newbye...)

Thanks!

[Ritesh]

Hello ESP_igrr,

Thanks for the "good news" because it is very important for our project.

Do you have a time schedule to merge the proposed PR in master branch?

I will give a try for sure. I just have to understand how to download this specific commit ( I'm just a git newbye...)

Thanks!

Hi,

If you have any urgency then you can right now take those changes from below link as well which are not major changes to fix this issue.

https://github.com/espressif/esp-idf/pu ... 65c5743ce4

So, That you can directly pick up those changes into your working SPIFFS component and can go further for your project.

[davdav]

Hello @ESP_igrr and @Ritesh

I have tested the PR and to me it seems to work, so I think you can internally proceed with your testing and merge.

Thanks

[Ritesh]

Hello @ESP_igrr and @Ritesh

I have tested the PR and to me it seems to work, so I think you can internally proceed with your testing and merge.

Thanks

Hi,

I am ESP32 IDF user like you to use it for application development purpose. So, You need to ask to Espressif SDK team to merge it into master running repository after verifying it properly.

[davdav]

Hello @Ritesh,

PR is in "merge_pending" status

https://github.com/espressif/esp-idf/pull/2212

so I guess it will included soon in master.

Thanks

[Ritesh]

Hello @Ritesh,

PR is in "merge_pending" status

https://github.com/espressif/esp-idf/pull/2212

so I guess it will included soon in master.

Thanks

Good to hear that..

Thanks for update regarding same.