Help with optimizing file uploads to SD

[reactive001]

Hi All,

I was hoping for a bit of wisdom on optimizing some code for accepting a HTTP POST and writing the data to an SD card. The code for the webserver can be found in full here - https://github.com/funkBuild/esphttpd/b ... ttpd.c#L83

To set the scene-

• I have an SD card connected using 4-bit @ 40Mhz with a FAT filesystem mounted under VFS. Write benchmarks comes back at 1400kB/s for 4k buffer sizes so that seems to be all working well.
• CPU is running at 240Mhz, ESP IDF 3.2 (latest revision as of a day ago)
• Running the iPerf example gives a solid 30Mbit/s to my laptop (on a wired connection) so there shouldn't be an issue with my access point (Ubiquiti UniFi running 802.11n for 2.4GHz)
• Currently the webserver is the only task, though I want to run others

So with the default WiFi buffers (16 static, 16 dynamic) the download speed is great at around 400kB/s but my upload is a rather tiny 50kB/s. Increasing the RX WiFi buffers (like the iperf sdkconfig) to 64 dynamic will increase the speed to around 400kB/s but the downside is that there's no RAM for anything else and, quite often, doing an upload from Chrome will completely exhaust the free memory and you get a wifi disconnect/crash. Setting TCP_NODELAY has basically no effect on transfer speed.

The code that reads from the socket is super simple, basically just read from the socket using recv() and then fwrite() to the file. I've tried many chunk sizes (1k to 8k) but there's not a huge difference in speed. I've also tried using multiple threads with a RingBuffer, one thread reads from the socket and puts it in the ringbuffer, the second writes to the file. Unfortunately it runs slower

Can anyone suggest a better way?

My only thoughts at the moment would be to use LWIP directly instead of a the socket interface or add external RAM so I can have larger buffers without crashing. Hoping someone could offer some insight into squeezing a bit more out of the code.

Thanks!