ESP32-WROVER tests & issues

[loboris]

Recently I've got the ESP32-WROVER-KIT v3 with ESP32-WROVER module and I would like ask for some clarifications about few issues I have.

1.
SPI Flash cannot be set to work with 80MHz clock. The flash chip is detected as GigaDevice, Manufacturer ID 0xc8 chip ID 0x6016.
QUIO and DIO modes can be selected with 40MHz and works.
If 80Mhz clock is configured, boot fails with the message:
E (83) qio_mode: Failed to set QIE bit, not enabling QIO mode (only if QUIO mode is configured)
E (102) esp_image: image at 0x1000 has invalid magic byte
E (121) boot: failed to load bootloader header!

I've checked all GigaDevice 32Mb spi falsh data sheets and the chip should work with 80MHz clock.
As a consequence, SPIRAM can also be configured only for 40MHz operation.

2.
spiram works as expected. I've measured the speed with simple memset and memcmp functions and got the following results:

Code: Select all

======= PSRAM Test =======
Allocating 2 memory buffers (2 * 1048576 bytes) with "malloc"

  free heap (before alloc): 4483068
   free heap (after alloc): 2385900
               memset time: 270 ms; 7.407407 MB/sec
               memcmp time: 200 ms; 5.000000 MB/sec

The only issue I have is with realloc function. For example, If I allocate 3MB (3*1024*1024) buffer and then try to expand it to 3MB+256B ((3*1024*1024+256), realloc fails. Obviously, realloc never tries to expand existing memory block (there is 1MB free block), but always tries to allocate the new memory block and free the old one. I suppose it is not a bug, the realloc is probably written that way, but it would be useful if it could expand the allocated buffer by some amount.

3.
Some libraries do not work if SPIRAM is enabled and malloc() can also allocate in SPI SRAM is selected with default parameters.

For example:
If using wear leveling fat fs, it fails with the message:
Guru Meditation Error: Core 1 panic'ed (Cache disabled but cached memory region accessed)
If using SD Card (sdmmc library), mount fails with:
W (1616) vfs_fat_sdmmc: failed to mount card (13)

Both issues can be solved by setting Always put malloc()s smaller than this size, in bytes, in internal RAM to some larger value, for example 32768.

[ESP_Sprite]

I also ran into the 80MHz QIO mode problem. The issue is essentially that the SPI clock and cs lines (iirc) are made available on pins as well, but the parasitic capacitance that results from that kills the SPI signals at 80MHz. If you have a soldering iron, the fix is simple: on the V3 board, you can remove R140-145, R146 and R147 (all 0 ohm resistors, if you ever need to replace them) and the thing should be stable at QIO and 80MHz. At the moment, for some reason I can't get the SPI flash to work reliably on 80MHz yet; we're in the process of investigating why this is. For now, please set it to 40MHz mode (option is somewhere in menuconfig, under 'esp32-specific options').

Realloc at the moment indeed follows the simple-but-stupid allocate-copy-free option. We may switch to a slightly more intelligent solution when we switch heap allocators, something a colleague of mine is working on. That does not guarantee realocating 3MiB of memory always works, however; fragmentation may make in-place reallocation impossible.

Wrt the driver issues: we're aware that not all drivers may work yet. In the process of merging the psram branch into main, we'll also bring these drivers up to date with the allocation methods required. Setting the option you mentioned works, indeed, but be aware that if the system runs out of internal memory, naive mallocs smaller than this value will still be redirected to external memory. If you need a certain driver to work earlier than the merge, post it here and I'll see if I can check what it does wrong.

[loboris]

ESP_Sprite: Thank you for the fast response.

I've tried desoldering the resistors, and it works.

Code: Select all

I (66) qio_mode: Enabling QIO for flash chip GD
I (82) boot: SPI Speed      : 80MHz
I (95) boot: SPI Mode       : QIO
I (107) boot: SPI Flash Size : 4MB

Setting SPIRAM clock to 80MHz also works now:

Code: Select all

======= PSRAM Test =======
Allocating 2 memory buffers (2 * 1048576 bytes) with "malloc"

  free heap (before alloc): 4487308
   free heap (after alloc): 2390140
               memset time: 140 ms; 14.285714 MB/sec
               memcmp time: 130 ms; 7.692307 MB/sec

I've tested some complex applications with 80MHz clock (for ~30 minutes) and found no issues. I'll test more for reliability.

Another 'interesting' issue:
Some applications I've tested with SPIRAM enabled (mostly those with WiFi active) works only if Run FreeRTOS only on first core is set.

[ESP_Sprite]

WRT SPI mem @80MHz: in my app, it works for simpler memory operations, but it's possible to do some operations that seem to trigger bad reads... haven't figured out how it malfunctions yet, it may also be a race condition of some kind that only triggers if the ESP is quick enough.

WRT crashes: There's a known issue we found yesterday that just got solved today that seems to rear its ugly head mostly in the psram branch for some reason. Can you pull and try again? If not, how do those programs crash?

[loboris]

In my test program with FreeRTOS running on both cores , the crash occurs on write to file (using wear leveling fs). Mounting, file reading and file deleting works. The crash message is:

Code: Select all

Guru Meditation Error: Core  0 panic'ed (Cache disabled but cached memory region accessed)

With FreeRTOS running only on firs core it works.

After pulling the latest commit the crash still occurs.

The same crash occurs not only when running on 80MHz, but also when running at 40MHz.

[ESP_Sprite]

Ah, check. The issue most likely is that somewhere in the routines that write to flash, a buffer or a variable in SRAM is referred. Writing to flash disables the cache, which also makes the SPI SRAM inaccessible (because it uses the same cache); reading from it, then, leads to this error.

[loboris]

After more testing I can confirm that ESP_WROVER cannot work reliably at 80MHz SPIFLASH clock.
I've tested with my TFT demo program and it always crashes after some time. Sometimes it crashes after start, sometimes it crashes after 2~3 hours run.
It doesn't matter if SPIRAM is enabled or not or if SPIRAM runs at 40MHz or 80MHz.
It hapens not only if compiled with feature/psram_malloc branch, but also if compiled with master branch (SPIRAM certainly not initialized).

I hope it is because of ESP-WROVER-KIT PCB design, not because of ESP-WROVER module design.

[ESP_Sprite]

Is that with the resistors I indicated removed?

[loboris]

Is that with the resistors I indicated removed?

Yes, of course.

[ESP_Sprite]

Hmm. I'll see if I can poke the PCB guys to look into this. Do you have a copy of the offending program anywhere so we can also replicate this here at Espressif?