low speed when reading constantly from Pendrive and performing http post to XAMP server

[suryasid09]

Hello Community,

I am using IDF example to read 2048 bytes from USB Pen drive and then perform HTTP post every time from esp32 to XAMP server until the complete file is read. When I test the read speed. I got 0.5 MB/s from USB for 6MB file. However, when I do multiple post with 2048 bytes read to the XAMP server, I get a very low speed of 12.5 KB/sec. Firstly, I don't know what ideal value I should expect but I am certain this is very poor upload speed. I also went through the iperf and tried changing the tx buffer size. But, it didn't help. Here is the main code content of the prototyping code.

I would greatly appreciate some inputs here!

//**--reading and performing post operation**--//

static void file_operations(void)
{
while (1)
{
memset(buffer, 0, sizeof(buffer));
read_bytes = fread(buffer, 1, sizeof(buffer), file);

if (read_bytes <= 0)
{
break; // End of file reached
}
else
{
err = post_rest_function();
}
}
}

//**--post is called after every read of 2048 bytes**--//

static esp_err_t post_rest_function(void)
{
static char *post_data = buffer;
esp_http_client_set_post_field(client, post_data, strlen(post_data);
esp_err_t err = esp_http_client_perform(client);
return err;

}

//** ---http intialisation happens in main**--//

void app_main()
{
esp_http_client_config_t config_post = {
.url = "http://192.168.10.21/uploads/upload_file.php",
.method = HTTP_METHOD_POST,
.cert_pem = NULL,
};
client = esp_http_client_init(&config_post);
}

[MicroController]

Code: Select all

esp_http_client_set_post_field(client, post_data, strlen(post_data);

strlen(post_data) is not what you want. It will 1) truncate every block of posted data at the first byte which happens to be 0 and, worse, 2) append a random amount of garbage data if there happens to be no 0-byte in the current data block.

Just pass read_bytes to post_rest_function().

Then, sending a new HTTP request for every 2kb worth of data, potentially even closing and opening a new connection each time, will definitely kill performance.

[suryasid09]

Code: Select all

esp_http_client_set_post_field(client, post_data, strlen(post_data);

strlen(post_data) is not what you want. It will 1) truncate every block of posted data at the first byte which happens to be 0 and, worse, 2) append a random amount of garbage data if there happens to be no 0-byte in the current data block.

Just pass read_bytes to post_rest_function().

Then, sending a new HTTP request for every 2kb worth of data, potentially even closing and opening a new connection each time, will definitely kill performance.

Thank you for the reply. I have noted it.
Additionally, After few experiments, I found out that reading followed by post is really slow.
If I do only read for 50MB file, I get speed of 1 MB/sec
If I do only post for 50MB dummy file (locally created with char buffer[4096] filled with some random value and have iteration over it), I get speed of 459KB/sec

But, if I do read and then post till I have completely send the file, I get speed of 12KB/sec.
I don't understand why though! I I should achieve good speed in this case as well because, my reading speed is twice as fast as uploading speed. And the CPU is configured to run at 240MHZ. so, where exactly is the problem. I don't understand why this is happening?