Slowness for write operations on flash with esp-idf v3.1.5

[rayf15]

Hi guys, this is my first post here...

First of all, kudos to ESP32/ESP8266 community for the great work!!!

I'm trying to "upgrade" my project from esp-idf_v3.1.1 to esp-idf_v.3.1.5... No problem at all for the required firmware adaptions (so far none...). But I'm facing a weird slowness for writing operations on flash with newest v3.1.5: my application saves log data to flash every 30s so this is a crucial aspect for my firmware.

Below results are from flash operations carried out with the _same_ firmware compiled against the above two esp-idf versions:

esp_partition_write() -> duration 1-2ms with esp-idf_v3.1.1 and duration 6ms with esp-idf_v.3.1.5
esp_partition_erase() -> duration 1-2ms with esp-idf_v3.1.1 and duration 50-60ms (!!!) with esp-idf_v.3.1.5

To be noted that test program for above results is an essential firmware that executes only operations on flash and nothing more (i.e. no WiFi, BT, etc...).

Any hints about these different behaviors?

Thanks in advance for the support!

rayf15

[ESP_Angus]

Hi rayf,

Thanks for the kind words and detailed report.

The only relevant thing I can think of a new configuration option added in v3.1.5 to delay the CPU when performing long erase operations, to avoid problems reported with starvation of other tasks. It's this option:
https://docs.espressif.com/projects/esp ... RING_ERASE

But this defaults to disabled, so unless it's been manually enabled it shouldn't be doing anything.

A couple of other questions:

- How many sectors are you erasing each time?
- Do you see the same erase time on the exact same flash chip running the two firmware versions? Flash chips vary a bit one to the next, and they age a bit each time they are erased meaning the erase times will slowly increase over their lifespan (the datasheet for the specific chip gives some upper limit for the erase time.)

[rayf15]

Hi ESP_Angus,
thanks for you quick reply!!!

I confirm that for v3.1.5 test the "yield during erase operations" option has left disabled.

Here you are the answers to your further questions:

1) esp_partition_erase_range() is called with third parameter = 4096, so a single flash sector size
2) The test results have been carried out with the same hardware sample (a brand new evaluation board ESP32_DevKitC_V4)

In addition (and in my first post not explained): I also tried to compile the firmware against esp-idf_v3.2.2 and there are again slowness issues...

rayf15

[rayf15]

Update:
I tried to compile my simple application using also esp-idf_v3.1.2, esp-idf_v3.1.3 and esp-idf_v3.1.4.
With each esp-idf version listed above flash operations are quick as with esp-idf_v3.1.1.
So there should be something added with esp-idf_v3.1.5 that causes flash write/erase operation slowness...
Didn't anybody really notice this behavior?
I need to upgrade the esp-idf version to benefit of wifi binary libs update...
So far, maybe it worths update only the binary files from https://github.com/espressif/esp32-wifi-lib, release/3.1...

Is this last approach feasable or is it intrinsically error prone?

Thank you again!

rayf15

[ESP_Angus]

Hi ray,

Thanks for the additional details about the erase sizes and devices in use. We're looking into the flash erase slowness issue, will get back to you ASAP.

It's strongly recommended not to "mix and match" the binary libraries between IDF versions. It might work but it is completely unsupported.


Angus

[rayf15]

Hi all,
is there any news about this topic?
Unfortunately I'm stuck in understanding this issue... I absolutely need a esp-idf version upgrade for our application...
This upgrade is important because using esp-idf_v3.1.1 there are a few cases where my application (configured in WIFI_MODE_APSTA mode ) can't connect as station to access points having WPA2_PSK authentication requirements, whereas a simple test application (same AP/STA mode) written and compiled with latest Arduino for ESP32 (core version 1.0.2 based on IDF 3.2) can connect without any problem to the above problematic access points.
I really think (and hope) that upgrade of WiFi libs coming with esp-idf_v3.1.5 could really solve this situations.
To be noted that when my application connects to an access point, everything works perfectly... so I realized that there could be problems with the association mechanism with a little few third parts access point (but surely I'm not an IEEE 802.11 expert).

Thank you again!

rayf15

[ESP_Angus]

Hi ray,

One question: is the task with the slow write operations pinned to a core? If you change to pin it to either core, does the slowness go away?

(This is still a bug, just trying to determine if it's the same bug that you're seeing.)

Angus

[rayf15]

Hi ESP_Angus,

the minimal test application creates only an additional single task from app_main() using xTaskCreate(), so there is no preference for the core involved. The priority of this task is (ESP_TASK_MAIN_PRIO + 1).

I further investigated if there could be some different initializations for external flash access in esp-idf/components/bootloader and esp-idf/components/bootloader_support but nothing...

rayf15

[ESP_Angus]

Sorry I realised I didn't word my question properly. It sounds like in your test app the core is unpinned. If you change it to either xTaskCreatePinnedToCore(... , 0) or xTaskCreatePinnedToCore(... , 1) then does the slowness go away?

the minimal test application creates only an additional single task from app_main() using xTaskCreate(), so there is no preference for the core involved. The priority of this task is (ESP_TASK_MAIN_PRIO + 1).

[rayf15]

Hi ESP_Angus, hi all,
we finally solved the enigma... esp-idf_v3.1.5 added this fix to FreeRTOS's tick count error while erasing/writing to flash (https://github.com/espressif/esp-idf/co ... 812a81ca11).
Therefore the _real_ durations of flash operations are similar with esp-idf_v3.1.5 and esp-idf_v3.1.1, but in the latter case the measure method is intrinsically wrong!!!
In effect, further measurements using hardware timer functions like esp_timer_get_time() confirmed the expectations.

Thank you for the support!

rayf15