DNS not resolvable if hosting provider is using QUIC

[gmdriscoll]

We have an application using a WROOM-32E module. The IT department is moving our DNS hosting from Register.com to CloudFlare specifically. PlatformIO is the development environment and the version if ESP-IDF is 4.3.2.
PACKAGES:
- framework-espidf @ 3.40302.0 (4.3.2)
- tool-cmake @ 3.16.4
- tool-esptoolpy @ 1.30300.0 (3.3.0)
- tool-idf @ 1.0.1
- tool-mconf @ 1.4060000.20190628 (406.0.0)
- tool-ninja @ 1.9.0
- toolchain-esp32ulp @ 1.22851.191205 (2.28.51)
- toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3

DNS is resolved no problem at the current register.com hosting. It is running using UDP as the protocol. Below is a response from the google server using wireshark and browsing to site with dns hosted by register.com. This is what I expect is happening in the ESP32 as well.
41 3.831853 8.8.4.4 192.168.50.121 UDP 603 443 → 51450 Len=561

This is the response from the CloudFlare hosted DNS to the same website ip-address. The protocol is QUIC and the payload is not open text. The browser can handle this, but the ESP32 returns E (1657) TRANS_TCP: DNS lookup failed err=202 res=0x0
24 2.674502 8.8.4.4 192.168.50.121 QUIC 1292 Protected Payload (KP0)

The espressif provideed example of esp_http_client works with the original UDP DNS from register.com. If I change only the .host in the esp_http_client_config to the new domain name, it returns the E (1657) TRANS_TCP: DNS lookup failed err=202 res=0x0 error.

Here is the code setup for the http client config.
static void https_with_hostname_path(void)
{
esp_http_client_config_t config = {
.host = DEFAULT_WIFI_URL,
.path = DEFAULT_WIFI_PATH,
.cert_pem = server_root_cert_pem_start,
.event_handler = _http_event_handler,
//.transport_type = HTTP_TRANSPORT_OVER_SSL,
//.skip_cert_common_name_check = false,

.keep_alive_enable = true,

};
esp_http_client_handle_t client = esp_http_client_init(&config);
esp_err_t err = esp_http_client_perform(client);

if (err == ESP_OK) {
ESP_LOGI(TAG, "HTTPS Status = %d, content_length = %d",
esp_http_client_get_status_code(client),
esp_http_client_get_content_length(client));
} else {
ESP_LOGE(TAG, "Error perform http request %s", esp_err_to_name(err));
}
esp_http_client_cleanup(client);
}

The DNS servers are available and configured immediately after WiFi establishes an IP address and still there just before the esp_http_client_perform in the above code.

Does anyone know how to get the ESP32 using the standard example of esp_http_client to work properly with the protected version of QUIC DNS that is now being used at different service providers?

Thanks!

[ESP_YJM]

We not support QUIC. So DNS over Quic will fail.

[WiFive]

I doubt this has anything to do with QUIC. You should check the response using a command line dns lookup not a browser. And turn on lwip debugging to see why the error happens on esp32.

[gmdriscoll]

Here is the output with the LWIP debugging on. Again, if I change the nvsURL in the code below from the one at register.com to the one at CloudFlare the client can't connect. Definitely need help with the interpretation.

This is the code that is failing:
if (client == NULL){
ESP_LOGI(TAG, " INIT HTTP CLIENT!");
esp_http_client_config_t config = {};

//config.url = combURL.c_str();
config.host = nvsURL.c_str();
//config.port = 443;
config.path = nvsPATH.c_str();
config.cert_pem = server_root_cert_pem_start;
config.event_handler = _http_event_handler;
//config.transport_type = HTTP_TRANSPORT_OVER_SSL;
config.user_data = buffer;
//config.use_global_ca_store = false;
config.is_async = false; // false is non-blocking
config.skip_cert_common_name_check = false;

config.keep_alive_enable = true;


//esp_http_client_handle_t client = esp_http_client_init(&config);
client = esp_http_client_init(&config);

err = esp_http_client_perform(client);
if (err != ESP_OK){
ESP_LOGE(TAG, "Unable to establish https client");
if (client != NULL){
esp_http_client_close(client);
esp_http_client_cleanup(client);
client = NULL;
}
return 404;
}

}


I (5424) wifi station: INIT HTTP CLIENT!
pbuf_alloc(length=45)
pbuf_alloc(length=45) == 0x3ffc9390
pbuf_add_header: old 0x3ffc93f4 new 0x3ffc93ec (8)
pbuf_add_header: old 0x3ffc93ec new 0x3ffc93d8 (20)
ip4_output_if: st1
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 73 | (v, hl, tos, len)
+-------------------------------+
| 5 |000| 0 | (id, flags, offset)
+-------------------------------+
| 255 | 17 | 0xbcd9 | (ttl, proto, chksum)
+-------------------------------+
| 192 | 168 | 50 | 17 | (src)
+-------------------------------+
| 8 | 8 | 4 | 4 | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
pbuf_add_header: old 0x3ffc93d8 new 0x3ffc93ca (14)
ethernet_output: sending packet 0x3ffc9390
pbuf_free(0x3ffc9390)
pbuf_free: deallocating 0x3ffc9390
pbuf_alloc(length=87)
pbuf_alloc(length=87) == 0x3ffc9224
ethernet_input: dest:e8:9f:6d:b7:49:b8, src:3c:7c:3f:30:98:80, type:800
pbuf_remove_header: old 0x3ffc9424 new 0x3ffc9432 (14)
ip_input: iphdr->dest 0x1132a8c0 netif->ip_addr 0x1132a8c0 (0x32a8c0, 0x32a8c0, 0x11000000)
ip4_input: packet accepted on interface st
ip4_input:
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 73 | (v, hl, tos, len)
+-------------------------------+
| 37435 |000| 0 | (id, flags, offset)
+-------------------------------+
| 118 | 17 | 0xb3a3 | (ttl, proto, chksum)
+-------------------------------+
| 8 | 8 | 4 | 4 | (src)
+-------------------------------+
| 192 | 168 | 50 | 17 | (dest)
+-------------------------------+
ip4_input: p->len 73 p->tot_len 73
pbuf_remove_header: old 0x3ffc9432 new 0x3ffc9446 (20)
pbuf_remove_header: old 0x3ffc9446 new 0x3ffc944e (8)
pbuf_alloc(length=45)
pbuf_alloc(length=45) == 0x3ffc9490
pbuf_add_header: old 0x3ffc94f4 new 0x3ffc94ec (8)
pbuf_add_header: old 0x3ffc94ec new 0x3ffc94d8 (20)
ip4_output_if: st1
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 73 | (v, hl, tos, len)
+-------------------------------+
| 6 |000| 0 | (id, flags, offset)
+-------------------------------+
| 255 | 17 | 0xd63a | (ttl, proto, chksum)
+-------------------------------+
| 192 | 168 | 50 | 17 | (src)
+-------------------------------+
| 192 | 168 | 50 | 1 | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
pbuf_add_header: old 0x3ffc94d8 new 0x3ffc94ca (14)
ethernet_output: sending packet 0x3ffc9490
pbuf_free(0x3ffc9490)
pbuf_free: deallocating 0x3ffc9490
pbuf_free(0x3ffc9224)
pbuf_free: deallocating 0x3ffc9224
etharp_timer
etharp_timer: expired pending entry 0.
dhcp_coarse_tmr()
pbuf_alloc(length=45)
pbuf_alloc(length=45) == 0x3ffc9390
pbuf_add_header: old 0x3ffc93f4 new 0x3ffc93ec (8)
pbuf_add_header: old 0x3ffc93ec new 0x3ffc93d8 (20)
pbuf_alloc(length=87)
ip4_output_if: st1
pbuf_alloc(length=87) == 0x3ffc9224
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 73 | (v, hl, tos, len)
+-------------------------------+
| 7 |000| 0 | (id, flags, offset)
+-------------------------------+
| 255 | 17 | 0xd639 | (ttl, proto, chksum)
+-------------------------------+
| 192 | 168 | 50 | 17 | (src)
+-------------------------------+
| 192 | 168 | 50 | 1 | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
pbuf_add_header: old 0x3ffc93d8 new 0x3ffc93ca (14)
ethernet_output: sending packet 0x3ffc9390
pbuf_free(0x3ffc9390)
pbuf_free: deallocating 0x3ffc9390
pbuf_alloc(length=16)
pbuf_alloc(length=16) == 0x3ffc9390
pbuf_add_header: old 0x3ffc93e0 new 0x3ffc93b8 (40)
ip6_output_if: lo0
IPv6 header:
+-------------------------------+
| 6 | 0 | 0 | (ver, class, flow)
pbuf_alloc(length=87)
+-------------------------------+
pbuf_alloc(length=87) == 0x3ffc93f4
| 16 | 58 | 255 | (plen, nexth, hopl)
+-------------------------------+
| 0 | 0 | 0 | 0 | (src)
| 0 | 0 | 0 | 1 |
+-------------------------------+
| ff02 | 0 | 0 | 0 | (dest)
| 0 | 0 | 0 | 2 |
+-------------------------------+
netif->output_ip6()
pbuf_alloc(length=56)
pbuf_alloc(length=56) == 0x3ffc9628
pbuf_copy(0x3ffc9628, 0x3ffc9390)
pbuf_copy: end of chain reached.
pbuf_free(0x3ffc9390)
pbuf_free: deallocating 0x3ffc9390
ethernet_input: dest:e8:9f:6d:b7:49:b8, src:3c:7c:3f:30:98:80, type:800
pbuf_remove_header: old 0x3ffc94bc new 0x3ffc94ca (14)
ip_input: iphdr->dest 0x1132a8c0 netif->ip_addr 0x1132a8c0 (0x32a8c0, 0x32a8c0, 0x11000000)
ip4_input: packet accepted on interface st
ip4_input:
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 73 | (v, hl, tos, len)
+-------------------------------+
| 46816 |010| 0 | (id, flags, offset)
+-------------------------------+
| 64 | 17 | 0x9e60 | (ttl, proto, chksum)
+-------------------------------+
| 192 | 168 | 50 | 1 | (src)
+-------------------------------+
| 192 | 168 | 50 | 17 | (dest)
+-------------------------------+
ip4_input: p->len 73 p->tot_len 73
pbuf_remove_header: old 0x3ffc94ca new 0x3ffc94de (20)
pbuf_remove_header: old 0x3ffc94de new 0x3ffc94e6 (8)
pbuf_free(0x3ffc9224)
pbuf_free: deallocating 0x3ffc9224
ethernet_input: dest:e8:9f:6d:b7:49:b8, src:3c:7c:3f:30:98:80, type:800
pbuf_remove_header: old 0x3ffc95bc new 0x3ffc95ca (14)

E (5924) TRANS_TCP: DNS lookup failed err=202 res=0x0

ip_input: iphdr->dest 0x1132a8c0 netif->ip_addr 0x1132a8c0 (0x32a8c0, 0x32a8c0, 0x11000000)
ip4_input: packet accepted on interface st
ip4_input:
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 73 | (v, hl, tos, len)
+-------------------------------+
| 46820 |010| 0 | (id, flags, offset)
+-------------------------------+
| 64 | 17 | 0x9e5c | (ttl, proto, chksum)
+-------------------------------+
| 192 | 168 | 50 | 1 | (src)
+-------------------------------+
| 192 | 168 | 50 | 17 | (dest)
+-------------------------------+
ip4_input: p->len 73 p->tot_len 73
pbuf_remove_header: old 0x3ffc95ca new 0x3ffc95de (20)
pbuf_remove_header: old 0x3ffc95de new 0x3ffc95e6 (8)
pbuf_add_header: old 0x3ffc95e6 new 0x3ffc95ca (28)
pbuf_alloc(length=36)
pbuf_alloc(length=36) == 0x3ffc933c
icmp_time_exceeded from 192.168.50.1 to 192.168.50.17
pbuf_add_header: old 0x3ffc938c new 0x3ffc9378 (20)
ip4_output_if: st1
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 56 | (v, hl, tos, len)
+-------------------------------+
| 8 |000| 0 | (id, flags, offset)
+-------------------------------+
| 255 | 1 | 0xd659 | (ttl, proto, chksum)
+-------------------------------+
| 192 | 168 | 50 | 17 | (src)
+-------------------------------+
| 192 | 168 | 50 | 1 | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
pbuf_add_header: old 0x3ffc9378 new 0x3ffc936a (14)
ethernet_output: sending packet 0x3ffc933c
pbuf_free(0x3ffc933c)
pbuf_free: deallocating 0x3ffc933c
pbuf_free(0x3ffc93f4)
pbuf_free: deallocating 0x3ffc93f4
ip6_input: packet not for us.
pbuf_free(0x3ffc9628)
pbuf_free: deallocating 0x3ffc9628
E (6094) HTTP_CLIENT: Connection failed, sock < 0
pbuf_alloc(length=52)
pbuf_alloc(length=52) == 0x3ffc9224
ethernet_input: dest:01:80:c2:00:00:00, src:3c:7c:3f:30:98:82, type:0

pbuf_free(0x3ffc9224)
E (6104) wifi station: Unable to establish https client

[WiFive]

You don't really want raw packet debugging did you turn on LWIP_DNS_DEBUG

[gmdriscoll]

@WiFive - LWIP_DNS_DEBUG does not exist in the LWIP debugging section of menuconfig or sdkconfig. I have given up on this method although we have been using it for a few years now. The https_connect_example with esp_tls_write and esp_tls_read is working and I will move over to that as it has no trouble with either domain registrar.

Thanks for the help.