Page 1 of 1

Unreliable secure websocket connection

Posted: Tue Jul 25, 2023 6:10 pm
by tobobo
I'm having major issues with WebSocket reliability on an ESP32-S3 with esp-idf 4.4.5.

When sitting idle, the WebSocket mostly just receives ping messages from the server and responds. However, in this state the WebSocket disconnects frequently.

I've noticed a few things
- On my main home Wi-FI network (ASUS router), reliability is terrible. I also get tons of DHCP UDP packets in the lwip logs, which sometimes seem to coincide with disconnections occurring. Disconnections can happen several times per hour and can last a few minutes at a time. The product I'm building needs to work on a wide variety of Wi-FI networks, and a previous ESP32 WROOM version of the product has much better reliability.
- Using insecure WebSockets does not appear to improve the reliability.
- Using the guest network on my ASUS router does not improve reliability.
- Changing Wi-Fi Tx buffer size does not improve reliability.
- Moving Wi-Fi/mbedTLS to internal ram does not improve reliability.
- When I connect to a Wi-Fi network (Linksys router) that only has the ESP32-S3 connected, reliability is improved. Of note is that this router's WAN port is connected to my laptop, which is connected to the ASUS Wi-Fi network, so traffic is still going through the ASUS router.

I've attached a couple of logs. In the "error read data" log, the WebSocket disconnects when mbedtls claims to receive a "close notify message." I am not closing the connection on my side on the server, but I acknowledge that it's possible that Heroku or Cloudflare is closing the connection somewhere outside my application.

In the "ping timeout" log, a more severe issue is demonstrated. The WebSocket client doesn't receive any ping frames for 30s, so it disconnects. After disconnecting, it's unable to reconnect for a few minutes, with the error "Failed to open new connection in specified timeout" each time it attempts to connect.

Some other ideas—
- Maybe I should be changing the socket QoS? Not sure how to get that from the WebSocket client—the socket ID appears to be 54, but is it guaranteed to always be 54?

Any help is appreciated!

Re: Unreliable secure websocket connection

Posted: Wed Jul 26, 2023 2:12 am
by tobobo
Update—I've turned off IPV6, which I believe was off by default in older versions of esp-idf but is on by default in 4.4.5. I still periodically get "error read data" errors, but this seems to have fixed the issue of ping/select() timeouts.

My working theory is that my main home Wi-Fi router supports some IPV6 features that my old test network router does not support, and overwhelm the ESP32 when a WebSocket is open.

Re: Unreliable secure websocket connection

Posted: Wed Jul 26, 2023 4:20 am
by tobobo
I spoke too soon—turning off IPV6 resulted in a stable connection for a few hours, but now the connection is failing again.