SPIFFS performance degrades as partition size increase

[PeterR]

ESP-IDF: v4.0-dev-562-g2b301f53e

HTTP loading 17 files, in total 800,000 bytes on a file system which has approximately 100 files and 3MB total storage, then:

Partition Size=5MB
Loading completes in 11 seconds.

Partion Size=13MB
Loading completes in 25 seconds

NOTE: No change in number or size of files. Just partition size.

HTTP loading completes very much faster at 1 or 2 MB partition sizes. I had less total number of files and (oviously) a smaller image size in these tests.

The 5MB/13MB results clearly demonstrate however that performance degrades as partition size increases.
Extrapolating then I might expect 1MB to load in 2 seconds with a 1MB partition.
This fits with my earlier design proving tests when using an EVB (I was limited to small partitions back then).

Q1) Has SPIFFS been tested on larger devices?
Q2) Suggested alternative? The webpages are read only so conversion to an 'C' array and index table would be fine. Rather not re-invent the wheel.
If so what's the fastest way to read a block of FLASH?

EDIT:
Confirmed that my 2MB partition filesystem (50 files, 1.2MB total) loads all 50 files in 15 seconds but when the partition is sized at 13MB loads in 90 seconds.

EDIT: lwip includes a file system which I have used before. As I recall the lwip filesystem reads files directly from program memory using a pointer.
So, again, what is the fastest FLASH reading, should I write my own FLASH block reader or is strcpy() as good as it gets?
Secondly, advise on integrating the lwip filesystem into the standard api, fopen(), fread() etc.

[bnp117]

I've ported LittleFS to the esp and it seems to have much better performance than spiffs. I made the API to be identical to spiffs (just replace all your `spiffs` references with `littlefs`, so testing it out in your project should be pretty easy. I haven't played around with folders in it since I made it to replace spiffs in my project, but they probably work. Any enhancements, pull requests, etc would be appreciated!

https://github.com/joltwallet/esp_littlefs

[jcsbanks]

mmap and get a pointer.

[PeterR]

Thanks both.
Think I will go with lwip filesystem which is quite similar to mmap but also can do gziping & HTTP headers.

To answer my own question; to register the file system then I just need to use esp_vfs_register()
I will measure performance shortly and post results.

I don't understand why SPIFFS should be so slow and/or why performance should degrade so much just by changing partition size.
I'm not going to poke (as hopefully I have my answer) but there does seems a bug there.

[PeterR]

lwip's read only file system is around 70x faster than SPIFFS.