spiffs.begin(true) hangs....sometimes

[dovoto]

We just completed a manufacturing test run of a wroom 32D (16MB) based device and are seeing a strange issue with spiffs.

About 1% of time (14 of the 1200 test boards we ran) the application hangs on spiffs.begin(true). The code, hardware, and start up conditions of each device are identical.

We have 4 primary partitions:
factory at 0x10000, this is used to talk to our automated test equipment and is pretty small.
two 4MB OTA slots (0x210000,61...)
And a spiffs partition at the end that takes up most of the rest.

The spiffs.begin is called in the setup() of the factory ATE firmware which does some very limited testing with our programmer interface before we load the release firmware in to an OTA slot. It is here that the spiffs format operation hangs on 1% of the devices (the rest seem to function as expected and the format is quick).

I only wait about 1 minute for the format to complete (the successful boards take well under this and any longer and its quicker to just flash a blank SPIFFS file). I am about to let a couple set and see if it just takes an extra long time.

Is this something anyone else has seen?

Do some flash chips just take a long time to format?

It seems odd that these chips might be bad in some way since I do a successful full erase, program, verify cycle on the first 2MB or so of the chip and the factory code seems to run fine until the SPIFFs call (which is close to the beginning of setup()).

[dovoto]

Updated info:

So I made some modifications to wait up until 3 minutes for the spiffs format. Its about 6MB of empty space so I am not entirely sure what a normal time but it seems to be:

99% of the devices take 10-15 seconds
1% of the devices take 1-3 minutes.

This is deterministic (the devices seem to take the same amount of time, even if I have formatted them previously).

I erase the first 100k of the spiffs partition before I call the format routine but that seems to have no effect.

Difficult to think of a reason one chip would take longer than another to format but continuing to investigate.