Why would streaming UART traffic break wifi?

[seancross]

Hi,

I'm working with an ESP32 using ESP-IDF-4.4. I want to enable interactive remote serial sessions, so I'm implementing my own interrupt handler. The problem I'm running into is that wifi seems to die whenever I send "a lot" of data via the UART and the UART speed is above around 38400 baud.

My interrupt handler looks something like this:

Code: Select all

#if CONFIG_TARGET_UART_IDX == 0
#define TARGET_UART_DEV UART0
#define PERIPH_UART_MODULE PERIPH_UART0_MODULE
#define PERIPH_UART_IRQ ETS_UART0_INTR_SOURCE
#elif CONFIG_TARGET_UART_IDX == 1
#define TARGET_UART_DEV UART1
#define PERIPH_UART_MODULE PERIPH_UART1_MODULE
#define PERIPH_UART_IRQ ETS_UART1_INTR_SOURCE
#elif CONFIG_TARGET_UART_IDX == 2
#define TARGET_UART_DEV UART2
#define PERIPH_UART_MODULE PERIPH_UART2_MODULE
#define PERIPH_UART_IRQ ETS_UART2_INTR_SOURCE
#else
#error "No target UART defined"
#endif

static uart_isr_handle_t console_isr_handle;

static void IRAM_ATTR console_isr(void *param)
{
	uint32_t uart_intr_status = uart_ll_get_intsts_mask(&TARGET_UART_DEV);

	int bytes_read = 0;
	uart_irq_count += 1;

	if ((uart_intr_status & UART_INTR_RXFIFO_TOUT) || (uart_intr_status & UART_INTR_RXFIFO_FULL) ||
	    (uart_intr_status & UART_INTR_CMD_CHAR_DET)) {
		while (TARGET_UART_DEV.mem_rx_status.wr_addr != TARGET_UART_DEV.mem_rx_status.rd_addr) {
			bytes_read += 1;
			char c = (*((volatile uint32_t *)UART_FIFO_REG(CONFIG_TARGET_UART_IDX)));
			(void)c;
		}
		uart_rx_count += bytes_read;
	}
	uart_ll_clr_intsts_mask(&TARGET_UART_DEV, uart_intr_status);
}

static void setup_uart(void) {
	uart_config_t uart_config;
	memset(&uart_config, 0, sizeof(uart_config));
	uart_config.baud_rate = 460800;
	uart_config.data_bits = UART_DATA_8_BITS;
	uart_config.parity = UART_PARITY_DISABLE;
	uart_config.stop_bits = UART_STOP_BITS_1;
	uart_config.flow_ctrl = UART_HW_FLOWCTRL_DISABLE;

	ESP_ERROR_CHECK(uart_param_config(CONFIG_TARGET_UART_IDX, &uart_config));

	const uart_intr_config_t uart_intr = {
		.intr_enable_mask = UART_RXFIFO_FULL_INT_ENA_M | UART_FRM_ERR_INT_ENA_M | UART_RXFIFO_OVF_INT_ENA_M
		/*|UART_TXFIFO_EMPTY_INT_ENA_M*/,
		.rx_timeout_thresh = 9999,
		.txfifo_empty_intr_thresh = 1,
		.rxfifo_full_thresh = 126,
	};
	ESP_ERROR_CHECK(uart_intr_config(CONFIG_TARGET_UART_IDX, &uart_intr));

	ESP_ERROR_CHECK(esp_intr_alloc(PERIPH_UART_IRQ, ESP_INTR_FLAG_IRAM | ESP_INTR_FLAG_LEVEL1, console_isr,
				       &uart_context[CONFIG_TARGET_UART_IDX], &console_isr_handle));
}

I can see that `uart_irq_count` goes up, as does `bytes_read`. Note that I'm not actually doing anything with the data in this test, I'm just wanting to make sure that I could theoretically do something. Also note that the handler is in RAM.

I'm testing this connection by sending a copy of Pride & Prejudice, which is about 800 kB.

Almost immediately, the ESP32 stops responding to pings. I can tell from the serial console that it hasn't crashed. When I stop the serual transfer, it recovers after a few seconds.

If I set the serial port speed to around 38400 baud, it behaves much better. 57600 is a speed at which things start to really break down, and at 115200 it drops off wifi as soon as the transfer starts.

How can I track down what's going on here? Is this something odd like a clock-domain crossing issue? Or is there a resource starvation issue? What's causing the ESP32 to fall over after receiving such a small amount of data?

[seancross]

In order to simplify the issue and demonstrate it more fully, I've trimmed my code down and created a brand-new project. This project is posted at https://github.com/xobs/esp32-uart-issue

This smaller version simply runs a TFTP server that makes updating easier. I've verified that stopping that TFTP server does not change things, so I've left it in because it makes it easier to iterate.

[mikemoy]

Having a while loop in a IRQ is very bad practice. With IRQ's you want to do the absolute minimum in there because while you are in there doing something nothing else gets done.

[WiFive]

Using a while loop to empty the fifo is not a problem, the idf driver does the same thing. The fact that your interrupt runs every 2 bytes is not great though since bit times get smaller with higher baud rates.

[seancross]

As WiFive noted, the `while (1)` idiom came from the ESP-IDF uart.c driver.

Additionally, I've updated my test program to read 126 bytes at a time. So I know that's not it.

Things have gotten incredibly bizarre, however. I have the UART connected to GPIO22, and I've discovered that wifi breaks even if I don't configure the UART at all.

I have it configured as a simple GPIO with a PU:

Code: Select all

I (747) gpio-test: GPIO_FUNC22_IN_SEL: 00000038 (connect: 0  inv: 0  peripheral: 56)
I (2937) gpio-test: GPIO_FUNC22_OUT_SEL: 00000100 (oen_inv: 0  en: 0  inv: 0  func: 256)
I (3237) gpio-test: IO_MUX_GPIO22_REG: 00002900 (mcu_sel: 2)

Additionally, I have no IRQ configured, and UART1 isn't configured at all.

And wifi still breaks when UART traffic is received.

I don't believe it's a problem with the power supply, since the V3V3 line looks stable on my scope. I also don't think it's a problem with an unstable clock line since I configured the MCPWM to generate a 200 kHz pulse out one of the pins and that looks very stable.

GPIO22 is relatively far from the antenna, so I don't think that's a problem.

[seancross]

Thanks to everyone for their help!

I've done a more thorough analysis, and I've discovered it's a design flaw in the PCB.

Due to space constraints, the ground plane is a bit thin around the level shifter. Unfortunately, the ground return for the UART RX pin is shared with the ground return for the 40 MHz XTAL. Worse, the UART RX actually wraps around the XTAL, effectively turning the whole thing into a very nice differential signal.

At higher speeds, the difference is enough to detune the XTAL and cause it to drift ever so slightly from 40 MHz, which knocks the device off wifi. When the UART finishes transmitting, the XTAL goes back to its normal state and the ESP32 can reassociate with wifi.

In this case, the solution is a PCB redesign to get rid of this issue. RF is spooky.

[ESP_Sprite]

Oooh, that is super-tricky; kudos for finding that!