Uart RX losing data when recieving data at high baud rates

[DanCurram]

So set up a serial port to work at 115200bps. Once I have the rest of my code stable I want to go up to 460800 bps to reduce data transfer times.

The CPU frequency is set to 240Mhz
I process the data in the event handler set up for that RX.
I look for a terminating character eg \r of \n. when I find terminating character I copy it to a message que and fire it at task that deals with that uart.

Currently I am transferring a Server Certificate via serial to be stored in a file in spiffs.
What I am seeing is basically every 128 chars I lose 8 to 10.

Currently I am using 2 uarts eg one as a debugging console ( eg the default console uart used for programming and the output of
ESP_LOGI() the other is the uart interface to an embedded device.

So:
is this a conflict of buffers etc I have set up ring buffers of 256 I use to pass data between FreeRtos tasks but I see the uarts have

#define UART_FIFO_LEN (128) /*!< Length of the hardware FIFO buffers */
#define UART_FULL_THRESH_DEFAULT (120)

https://www.espressif.com/sites/default ... f#page=343

Or

Priorities and processing speed of the message ques and FreeRtos
I am going to start investigating but basically am I spending to long in the event handler looking for a terminating character, then bundling up the data to sent via a message que to hold the event handler so the RX isr handler misses characters.

Those who use the uarts to transfer data what is the fastest stable speed you have it working with when you transfer more than 120 consecutive bytes ?

Thanks in Advance for any contribution
Danny

[DanCurram]

viewtopic.php?f=19&t=13312&p=52666&hilit=UART+RX#p52666

viewtopic.php?f=2&t=6130&p=51693&hilit=UART+RX#p51693

So swagging it looks like I have to change how deal with the data from the event handler or the priorites around the uart task

[DanCurram]

I have based the uart RX side of the code, on this example.

https://github.com/espressif/esp-idf/bl ... ple_main.c

The test file I am sending currently sending 632 bytes. ( cut and paste of abcdefghijklmnopqrstuvwxyz) I drag the file to teraterm and let it send it.
a test sum value has it receiving 624 char.
I started at 130 chars and increased the size of the file, during testing.

I modified the code

//Create a task to handler UART event from ISR
xTaskCreate(uart_event_task, "uart_event_task", 2048, NULL, 12, NULL);

to

xTaskCreate(uart_event_task, "uart_event_task", 2048, NULL, 21, NULL); // made it a higher priority task

So the first test simply stored event.size each time it had the event UART_DATA in an array after this call

uart_read_bytes(EX_UART_NUM, dtmp, event.size, portMAX_DELAY);

so the array would have 0x78,0 when sending 130 bytes, this ties in with the UART_FULL_THRESH_DEFAULT (120) but its missing characters.

Sending more data eg 150 was 0x78, 1A when it should have been 0x78,0x1E.
I modified the test file to 632 bytes and I see 0x78,0x78,0x78,0x78,0x78,0x18 when it should have ended 20

It doesn't seem to matter what size you set rx buffer size it always return 120 chars

#define C_USART_BUF_SIZE (512)
uart_driver_install(UART_NUM_1, C_USART_BUF_SIZE, C_USART_BUF_SIZE, 10, &msWlanUart1_queue, 0);

Looking at the buffered data the first 120 chars appears ok it, then looses a few in sequence, then is okay for the rest of the data to the next 120 byte boundary . A quick inspection of the data at the 120 boundaries sometimes is ok, sometimes is out of sequence

So initially it looks like the ISR always struggles to populate the message que and receive characters in the driver for the first queued data and then randomly looses it for others.

Only significant difference from the example code is how I initialise the uart driver

Code: Select all

configMAX_PRIORITIES = 25

#define C_TASK_PRIORITY_DEFAULT_HIGHEST	3
#define C_TASK_PRIORITY_UARTS    configMAX_PRIORITIES - (C_TASK_PRIORITY_DEFAULT_HIGHEST + 1) 
#define C_USART_BUF_SIZE		   (512)

void hw_Wlan_StartUartTsk(void)
{

	uart_driver_install(UART_NUM_1, C_USART_BUF_SIZE, C_USART_BUF_SIZE, 10, &msWlanUart1_queue, 0);
	uart_set_rx_timeout(UART_NUM_1, 100);
	/**********************************/
	uart_enable_pattern_det_intr(UART_NUM_1, '+', C_USART_PATTERN_CHR_NUM, 1000, 50, 50);
	uart_pattern_queue_reset(UART_NUM_1, 10);
	/**********************************/
	xTaskCreate(WLANUart_event_task, "WLAN_UEVTSK", 4096, NULL, C_TASK_PRIORITY_UARTS, NULL);
}

IDF version used: v4.0-beta2-174-g99fb9a3f7-dirty
code is edited and debugged using visualGDB
the dev board is a Esp WROVER V4.1

[WiFive]

You have to lower the threshold so the interrupt is triggered sooner with high data rates.

[DanCurram]

@WiFive I will drop the FIFO buffer threshold and see how I get on.


I have looked at the uart.c file and the function static void uart_rx_intr_handler_default(void *param)
If this is an ISR routine at 300 odd lines covering all the uart isr events (uart_event_type_t) is this why it misses chars being sent to the uart ?

I currently set the pattern detect so perhaps I need to drop this functionality and carry it out later in my code.

Has anyone else come across this issue ?
How did you overcome it ?

[DanCurram]

Changing the FIFO full threshold did not cure the problem, it still loses data on the FIFO full threshold.

Tried setting it to 96 , 64 and it just moved the issue.

This is perhaps in the wrong topic area and should be in the IDF area as it is a driver in the IDF

There is apart of the isr code that has a loop for a max of 127 counts while looping in the TX functionality, so is there a conflict between the cores and the logging and the uarts receiving data ?

[dldtechnology]

I see this problem even on small packets when at 460800 baud.

Using the ROM based uart_rx_one_char() seems reliable at that speed (although misses bytes when sending larger packets due to RTOS etc).

345600 works for smaller packets, but not reliable on YMODEM

230400 seems about the highest I can go without having issues.

[WiFive]

At 460800 a byte comes every 2us so if your Rx thresh is 120 you have 8x2=16us to start copying bytes out of the fifo before it overflows. If thresh is 64 then you have 64x2=128us but your interrupt runs twice as often. If you are not able to get good performance from tuning thresh and buffer size you may have to use your own custom isr/driver.