HTTP Authentication fails when using config with username, password entry

[Shelton]

Hello,

I'm using ESP32 DevKitC and try to implement http basic authentication.
The example code of ESP HTTP Client Example works fine.
It accesses a HTTP basic authentication by prepending username:password@ to the hostname in the URL .

Code: Select all

    esp_http_client_config_t config = {
        .url = "http://user:passwd@httpbin.org/basic-auth/user/passwd",
        .event_handler = _http_event_handler,
        .auth_type = HTTP_AUTH_TYPE_BASIC,
    };

However, use of the format "user:password" in the userinfo field is deprecated by RFC 3986.
So I try to use config authentication example with username, password entry instead, like following:

Code: Select all

esp_http_client_config_t config = {
    .url = "http://httpbin.org/basic-auth/user/passwd",
    .username = "user",
    .password = "passwd",
    .auth_type = HTTP_AUTH_TYPE_BASIC,
};

It is the same as the API document of esp-idf.
https://esp-idf.readthedocs.io/zh_CN/la ... word-entry

But no luck with this config, the module would send the http request without "Authorization" filed in the header.

I'm using the lasted esp-idf.
Could you help to check this?
Thanks!

[thethinker]

Hi There!
Were you ever able to figure this out? I'm having the same problem.

[lesher]

I'm having trouble with this as well. Just adding username and password to the config does not add header "Authorization: Basic <base64_enc(user:pass)>"

I tried adding the header manually, as shown in the code below. The request header looks okay, but I'm still getting the same 401 error. Interestingly, the arduino libs work (as shown here), so I'm stepping through esp_http_client.c to see what's happening.

Code: Select all

  httpclient.setTimeout(20000);
  httpclient.begin(url);
  httpclient.setAuthorization(user, pass);

Code: Select all

  esp_http_client_handle_t client = esp_http_client_init(&config);
  if (client == NULL) {
    ESP_LOGE(TAG, "Failed to initialise HTTP connection");
    task_fatal_error();
  }

  // Add our own base64 encoded basic authorization.

  // base64 encoded string.
  size_t bdstlen = (strlen(config.username) + strlen(config.password) + 2) * 4;
  char * bdst = malloc(bdstlen);
  printf(" sizeof bdst %d\n", bdstlen);

  // value for Auth key/value pair.
  const char * basicstr = "Basic ";
  char * authvaluestr = malloc(bdstlen + strlen(basicstr));
  strcpy (authvaluestr, basicstr);

  // Assemble user:pass.
  char * bsrc = malloc(strlen(config.username) + strlen(config.password) + 2);  // ':' and null
  strcpy(bsrc, config.username);
  strcat(bsrc, ":");
  strcat(bsrc, config.password);
  printf(" strlen(bsrc)  %d\n", strlen(bsrc));

  // Encode user:pass.
  size_t olen = 0;
  int encret = mbedtls_base64_encode((unsigned char*)bdst, bdstlen, &olen,
    (unsigned char*)bsrc, strlen(bsrc) );
  printf(" encret, olen, bsrc, bdst   %d  %d  %s  %s\n", encret, olen, bsrc, bdst);

  // Assemble the value for key/value pair.
  strcat(authvaluestr, bdst);

  // Add the header.
  esp_http_client_set_header(client, "Authentication", authvaluestr);

  err = esp_http_client_open(client, 0);
  if (err != ESP_OK) {
    ESP_LOGE(TAG, "Failed to open HTTP connection: %s", esp_err_to_name(err));
    esp_http_client_cleanup(client);
    task_fatal_error();
  }
  esp_http_client_fetch_headers(client);

  code = esp_http_client_get_status_code(client);
  len = esp_http_client_get_content_length(client);
  ESP_LOGI(TAG, "HTTP GET Status = %d, content_length = %d", code, len);

  if (code == 401) {
    ESP_LOGE(TAG, "Authorization Required");
    http_cleanup(client);
    task_fatal_error();
    goto DONE;
  }

[thethinker]

HTTP Version was my issue I believe. I changed from the python server to Apache and it solved my issue.

[gr1483]

HI All,

Did anyone work this out? I have the basic code, as the example, and i have used WireShark, i can see it asking for Authentication, but the HTTP Client does not add it to the header?

Is there something I am missing? Did anyone get this to work?

Thanks

Simon

[gr1483]

I have managed to figure out the issue.

The username and Password set with the .username and .password in the config is being cleared by the function esp_http_client_set_url. If you remove the last 4 lines, it then works as expected. Posting so anyone else can benefit from this issue.

Thanks

Code: Select all

esp_err_t esp_http_client_set_url(esp_http_client_handle_t client, const char *url)
{
    char *old_host = NULL;
    struct http_parser_url purl;
    int old_port;

    if (client == NULL || url == NULL) {
        ESP_LOGE(TAG, "client or url must not NULL");
        return ESP_ERR_INVALID_ARG;
    }

    http_parser_url_init(&purl);

    int parser_status = http_parser_parse_url(url, strlen(url), 0, &purl);

    if (parser_status != 0) {
        ESP_LOGE(TAG, "Error parse url %s", url);
        return ESP_ERR_INVALID_ARG;
    }
    old_host = client->connection_info.host;
    old_port = client->connection_info.port;

    if (purl.field_data[UF_HOST].len) {
        http_utils_assign_string(&client->connection_info.host, url + purl.field_data[UF_HOST].off, purl.field_data[UF_HOST].len);
        HTTP_MEM_CHECK(TAG, client->connection_info.host, return ESP_ERR_NO_MEM);
    }
    // Close the connection if host was changed
    if (old_host && client->connection_info.host
            && strcasecmp(old_host, (const void *)client->connection_info.host) != 0) {
        ESP_LOGD(TAG, "New host assign = %s", client->connection_info.host);
        if (esp_http_client_set_header(client, "Host", client->connection_info.host) != ESP_OK) {
            return ESP_ERR_NO_MEM;
        }
        esp_http_client_close(client);
    }

    if (purl.field_data[UF_SCHEMA].len) {
        http_utils_assign_string(&client->connection_info.scheme, url + purl.field_data[UF_SCHEMA].off, purl.field_data[UF_SCHEMA].len);
        HTTP_MEM_CHECK(TAG, client->connection_info.scheme, return ESP_ERR_NO_MEM);

        if (strcasecmp(client->connection_info.scheme, "http") == 0) {
            client->connection_info.port = DEFAULT_HTTP_PORT;
        } else if (strcasecmp(client->connection_info.scheme, "https") == 0) {
            client->connection_info.port = DEFAULT_HTTPS_PORT;
        }
    }

    if (purl.field_data[UF_PORT].len) {
        client->connection_info.port = strtol((const char*)(url + purl.field_data[UF_PORT].off), NULL, 10);
    }

    if (old_port != client->connection_info.port) {
        esp_http_client_close(client);
    }

    if (purl.field_data[UF_USERINFO].len) {
        char *user_info = NULL;
        http_utils_assign_string(&user_info, url + purl.field_data[UF_USERINFO].off, purl.field_data[UF_USERINFO].len);
        if (user_info) {
            char *username = user_info;
            char *password = strchr(user_info, ':');
            if (password) {
                *password = 0;
                password ++;
                http_utils_assign_string(&client->connection_info.password, password, 0);
                HTTP_MEM_CHECK(TAG, client->connection_info.password, return ESP_ERR_NO_MEM);
            }
            http_utils_assign_string(&client->connection_info.username, username, 0);
            HTTP_MEM_CHECK(TAG, client->connection_info.username, return ESP_ERR_NO_MEM);
            free(user_info);
        } else {
            return ESP_ERR_NO_MEM;
        }
    } else {
       /* free(client->connection_info.username);
        free(client->connection_info.password);
        client->connection_info.username = NULL;
        client->connection_info.password = NULL;
        */
    }

[ESP_Sprite]

Note that you probably leak memory by commenting out these free() calls...

[OBDave]

I came here because I was having the same problem, and looked at the library code. It seems that setting .url is treated differently then setting .host and .path. And yeah, commenting out calls to free() probably will end in tears at some point.

I really don't understand why this should work, but this seems to have fixed the problem for me.

1) You have to edit kconfig to allow HTTP_AUTH_TYPE_BASIC

2) use .host and .path instead of .url. So instead of doing this:

Code: Select all

    esp_http_client_config_t config = 
	{
		.url = "https://www.domain.com/directory/",
 	       .username = "YourUsername",
 	       .password = "YourPassword",
 	       .auth_type = HTTP_AUTH_TYPE_BASIC,          /* need to edit sdkconfig to allow this! */
 	       .transport_type = HTTP_TRANSPORT_OVER_SSL,
 	       .event_handler = _http_event_handler,
 	       .cert_pem = domain_com_root_cert_pem_start,
    };

do this:

Code: Select all

    esp_http_client_config_t config = 
	{
		.host = "www.domain.com",
		.path = "/directory/",
 	       .username = "YourUsername",
 	       .password = "YourPassword",
 	       .auth_type = HTTP_AUTH_TYPE_BASIC,          /* need to edit sdkconfig to allow this! */
 	       .transport_type = HTTP_TRANSPORT_OVER_SSL,
 	       .event_handler = _http_event_handler,
 	       .cert_pem = domain_com_root_cert_pem_start,
    };

[autodog]

Folks:

I had the exact same issue as the original post.

Good news: your can ignore all the workarounds in this thread - it's been fixed in esp-idf release v3.3.1:
https://github.com/espressif/esp-idf/re ... tag/v3.3.1

From the HTTP Client release notes:

Fixed issue where calling esp_http_client_set_url() discarded username and password

You also have the option to enable HTTP basic authentication via menuconfig. Enable this and you should be good to go.

-AD