Hang and reboot during init when using core dump to flash and external PSRAM

[jcolebaker]

We have an ESP32 module with 2 MiB external PSRAM.

I have been storing core dumps to to flash, which was working well:

Code: Select all

CONFIG_ESP_COREDUMP_ENABLE=y
CONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=y
...

I have now enabled the PSRAM:

Code: Select all

CONFIG_ESP32_SPIRAM_SUPPORT=y
CONFIG_SPIRAM_TYPE_AUTO=y
CONFIG_SPIRAM=y
CONFIG_SPIRAM_BOOT_INIT=y
CONFIG_SPIRAM_USE_CAPS_ALLOC=y
CONFIG_SPIRAM_MEMTEST=y
CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y
CONFIG_SPIRAM_BANKSWITCH_ENABLE=n
... Plus various other options

When I boot with these options, the boot process hangs for about 8 seconds during the ESP core dump initialization, then the device reboots with a watchdog timeout:

Code: Select all

I (1353) spiram: Adding pool of 2047K of external SPI memory to heap allocator
I (1362) spi_flash: detected chip: generic
I (1366) spi_flash: flash io: dio
I (1379) esp_core_dump_flash: Init core dump to flash
I (1393) esp_core_dump_flash: Found partition 'coredump' @ 50000 65536 bytes
ets Jul 29 2019 12:21:46

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)

If I disable the core dump to flash option (CONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=n), everything works well and the PSRAM can be used (but we lose the ability to recover core dump info from flash...).

What am I missing? Is it possible to use esp_core_dump_flash with external SPI RAM enabled?

[jcolebaker]

I've now found a work-around for this issue, but I'm not sure of the ultimate cause.

The problem is with code in the "espcoredump" component which calculates the checksum of the core dump when using flash storage ("esp_core_dump_image_check" function in "components\espcoredump\src\core_dump_flash.c"). This code is VERY slow, and takes ~10 seconds where it used to take maybe 200 mS without external RAM enabled. The delay was causing a watchdog timeout (and would be much too slow for our device boot up anyway).

The work-around is as follows:

* DISABLE core "dump check at boot" option: CONFIG_ESP_COREDUMP_CHECK_BOOT=n
* When attempting to check for a core dump from user code, I also call "esp_core_dump_image_check". I had to move this to a low-priority task so that it could happen in the background after app startup.
* Patched the ESP-IDF SDK to make the "coredump cache size" 128 (was 32): #define COREDUMP_CACHE_SIZE 128 in "components\espcoredump\include_core_dump\esp_core_dump_types.h". This made the checksum calculation significantly faster, but it was still quite slow.

I don't know why this particular flash read is so slow. I have other code which reads and writes significant data to/from flash partitions and it doesn't seem to be noticeably affected by enabling PSRAM.

[ESP_adokitkat]

Hello. Thank you for reporting this problem. I have created an issue about this in our internal issue tracker.