I need to be able to POST large amounts of data to a server. That is, the size of the data exceeds the ESP32 memory size (to big to malloc)
However, I cannot find any esp_http_client documentation or examples to do this as a chunked transfer.
esp_http_client - How to POST Chunked Data?
Re: esp_http_client - How to POST Chunked Data?
Use the stream functions of esp_http_client:
https://docs.espressif.com/projects/esp ... ttp-stream
Needs this:
eg.
https://docs.espressif.com/projects/esp ... ttp-stream
Needs this:
Code: Select all
esp_http_client_set_header(client, "Transfer-Encoding", "chunked");
Code: Select all
esp_http_client_write("5", 1); // length
esp_http_client_write("\r\n", 2);
esp_http_client_write("Hello", 5); // data
esp_http_client_write("\r\n", 2);
esp_http_client_write("6", 1); // length
esp_http_client_write("\r\n", 2);
esp_http_client_write(" World!", 7); // data
esp_http_client_write("\r\n", 2);
esp_http_client_write("0", 1); // end
esp_http_client_write("\r\n", 2);
esp_http_client_write("\r\n", 2);
Re: esp_http_client - How to POST Chunked Data?
Thanks boarchuz, this is the way to do it. I have this working and there are some details/corrections I can add:
You do NOT need to do "esp_http_client_set_header(client, "Transfer-Encoding", "chunked");"
Instead, you do this:
When you open with a write_len of -1, it will set the "Transfer-Encoding" header to "chunked" automatically. It will also force the http method to POST. (see http_client_prepare_first_line() in the source code https://github.com/espressif/esp-idf/bl ... p_client.c)
The resulting code:
Note to Espressif: the documentation for esp_http_client_open() should explain that a negative value of write_len is how chunked POST is to be performed.
You do NOT need to do "esp_http_client_set_header(client, "Transfer-Encoding", "chunked");"
Instead, you do this:
Code: Select all
esp_http_client_open(client, -1);
The resulting code:
Code: Select all
esp_http_client_config_t config = {
.url = "http://httpbin.org/post",
.method = HTTP_METHOD_POST, // This is NOT required. write_len < 0 will force POST anyway
};
esp_http_client_handle_t client = esp_http_client_init(&config);
esp_err_t err = esp_http_client_open(client, -1); // write_len=-1 sets header "Transfer-Encoding: chunked" and method to POST
if (err != ESP_OK) {
ESP_LOGE(TAG, "Failed to open HTTP connection: %s", esp_err_to_name(err));
return;
}
// Post some data
esp_http_client_write("5", 1); // length
esp_http_client_write("\r\n", 2);
esp_http_client_write("Hello", 5); // data
esp_http_client_write("\r\n", 2);
esp_http_client_write("6", 1); // length
esp_http_client_write("\r\n", 2);
esp_http_client_write(" World!", 7); // data
esp_http_client_write("\r\n", 2);
esp_http_client_write("0", 1); // end
esp_http_client_write("\r\n", 2);
esp_http_client_write("\r\n", 2);
// After the POST is complete, you can examine the response as required using:
// esp_http_client_fetch_headers() for content length
// esp_http_client_get_status_code()
// esp_http_client_read()
esp_http_client_close(client);
esp_http_client_cleanup(client);
boarchuz wrote: ↑Sun Sep 13, 2020 9:28 amUse the stream functions of esp_http_client:
https://docs.espressif.com/projects/esp ... ttp-stream
Needs this:eg.Code: Select all
esp_http_client_set_header(client, "Transfer-Encoding", "chunked");
<snip>
Re: esp_http_client - How to POST Chunked Data?
I wanted to post a reply for anyone else doiung this as I had a lot of trouble with sending chunked encoding, so hopefully it will help someone else out:)
The issue that was tripping me up was that my transfers all seemed to fail, and it turns out I was sending the length as decimal numbers - they need to be hexadecimal.
Also of note, if doing mixed transfers with the same handle, remember to delete any used headers before exiting the function, the transfer-encoding header seems to need to be deleted manually before using the handle for something else.
Here is some example code that I'm using - to send the end sequence, use transmit_chunk(""):
The issue that was tripping me up was that my transfers all seemed to fail, and it turns out I was sending the length as decimal numbers - they need to be hexadecimal.
Also of note, if doing mixed transfers with the same handle, remember to delete any used headers before exiting the function, the transfer-encoding header seems to need to be deleted manually before using the handle for something else.
Here is some example code that I'm using - to send the end sequence, use transmit_chunk(""):
Code: Select all
int transmit_chunk(char *string) {
int written_length = 0;
int total_length = 0;
char length_string[11] = "";
if (strlen(string) > 0) {
bzero(length_string, sizeof(length_string));
snprintf(length_string, sizeof(length_string), "%X", strlen(string));
written_length = esp_http_client_write(http_client, length_string, strlen(length_string));
if (written_length > 0) {
total_length += written_length;
} else {
return -1;
}
written_length = esp_http_client_write(http_client, "\r\n", 2);
if (written_length > 0) {
total_length += written_length;
} else {
return -1;
}
ESP_LOGD(UART_TAG, "Sent: %s", length_string);
written_length = esp_http_client_write(http_client, string, strlen(string));
if (written_length > 0) {
total_length += written_length;
} else {
return -1;
}
written_length = esp_http_client_write(http_client, "\r\n", 2);
if (written_length > 0) {
total_length += written_length;
} else {
return -1;
}
ESP_LOGD(UART_TAG, "Sent: %s", string);
} else {
written_length = esp_http_client_write(http_client, "0", 1); // end
if (written_length > 0) {
total_length += written_length;
} else {
return -1;
}
esp_http_client_write(http_client, "\r\n", 2);
if (written_length > 0) {
total_length += written_length;
} else {
return -1;
}
esp_http_client_write(http_client, "\r\n", 2);
if (written_length > 0) {
total_length += written_length;
} else {
return -1;
}
ESP_LOGD(UART_TAG, "Sent http end chunk");
}
return total_length;
}
Who is online
Users browsing this forum: No registered users and 225 guests