Caching issues with mmap'ed flash

[jens.alfke]

I'm calling esp_partition_mmap() to memory-map a flash partition, and then esp_partition_erase_range() and esp_partition_write() to write to it. Most of the time (95%?) this works fine, but intermittently I see errors where, after a write, part of the mapped memory is still filled with FF instead of the data I wrote.

I thought this might be a race condition with the flash controller (maybe the write takes longer than it indicates and sometimes it hasn't finished by the time the read occurs) so I added a 100ms delay after the write; that seems to help somewhat, but it's not a complete fix.

My only other hypothesis is that this is due to CPU cache coherency, but the spi_flash code looks like it's careful to disable/enable caches. I'm not familiar with the ESP32's cache system, but I assume it's doing the equivalent of a memory-barrier.

Any ideas?

Here's some rough code showing the kind of thing I do:

Code: Select all

void *mapped;
esp_partition_mmap(partition, 0, partition_size, SPI_FLASH_MMAP_DATA, &mapped, &handle);
...
esp_partition_erase_range(partition, 0, 4096);
esp_partition_write(partition, 0, data, dataLength);
...
assert(memcmp(data, mapped, dataLength) == 0);    // sometimes fails

[WiFive]

It might be due to stale cache after erase if the cache is not automatically flushed.

[jens.alfke]

Yeah, that's what I'm thinking too; as I wrote above:

My only other hypothesis is that this is due to CPU cache coherency, but the spi_flash code looks like it's careful to disable/enable caches.

However I have no idea what additional cache might be at fault, nor how to flush it.

[WiFive]

Well spi_flash_write and spi_flash_erase will call spi_flash_mark_modified_region but it does not actually flush the cache until spi_flash_ensure_unmodified_region is called. So I don't know if there is a rule that you are supposed to unmap and remap memory if you are going to write/erase it.

[jens.alfke]

Looking through the spi_flash sources I found calls to a Cache_Flush() function; this looked promising, so I added

Code: Select all

Cache_Flush(0); Cache_Flush(1);

to my code just after calling esp_partition_write(). That appears to have fixed the problem.

[WiFive]

I believe those should only be called inside flash guarded section.

[jens.alfke]

FYI, I've filed an IDF issue on Github.