HTTP client: auto redirection not working in some cases

[mschultz]

I have found what I believe to be a bug in the ESP-IDF HTTP client.
I am using the master branch, ESP-IDF 5.0 which I last pulled at the start of 04/2022.

Automatic redirection, in which a site returning a 3xx status code requests a redirect to a different path or URL, is not working - even when client_config.disable_auto_redirection = false. It appears as if the determining factor as to whether it works or not is if the original site (the one returning the redirect status) does not provide any body data in its response - that is, it responds with header data only

To check this, I added a log output to the ESP-IDF internal routine esp_http_check_response() in esp_http_client.c:

Code: Select all

static esp_err_t esp_http_check_response(esp_http_client_handle_t client)
{
    ESP_LOGI(TAG, "esp_http_check_response() invoked with status code %d", client->response->status_code);
    if (client->response->status_code >= HttpStatus_Ok && client->response->status_code < HttpStatus_MultipleChoices) {
        return ESP_OK;
    }
    // ... more code dealing with redirect cases not shown here
}

When testing this with a site that returns a content-length > 0 (i.e. has body data), the above function gets called after the HTTP_EVENT_ON_DATA event is posted. On a site that does not provide body data, the esp_http_check_response function is never called. For a site with no body data, numerous HTTP_EVENT_ON_HEADER events are posted (including one for the "Location:" header) followed by a HTTP_EVENT_ON_FINISHED and finally a HTTP_EVENT_DISCONNECTED. The HTTP_EVENT_REDIRECT event is never posted in either case, which is expected as this event is only generated when manual redirection is requested.

All of my testing was done using normal/default blocking mode calls (client_config.is_async = false). None of the calls being made at the application level are returning errors. esp_http_client_get_status_code() returns HTTP status 302 Found after processing a no-body site that requests redirect. HTTP_EVENT_ERROR is never posted.

In the no-body redirect case, if I invoke the public function esp_http_client_set_redirection(handle) after checking for a 3xx HTTP status code and restart the process (set_method, client_open, fetch_headers, get_status_code) I can get redirection to work. The ESP-IDF documentation is unclear as to what should be done after calling esp_http_client_set_redirection() - Can I do what I'm doing above without risk of memory leaks?

I did try to find and fix the problem myself, but I gave up after failing to find out when and where (the private routine) http_on_body() is invoked.