How to configure UART RX Timeout?

[Tropaion]

Hello,
I have a smartmeter which sends two MBUS frames every five seconds.
When the two frames are sent via UART, I want to process them.
Since I don't have much experience with UART and ESP I wanted to ask here.
As far as I understand, the best way is to set UART_INTR_RXFIFO_TOUT and process the data after the interrupt.
I managed to configure the UART and can receive data but I don't understand how the program the interrupt.
https://github.com/Tropaion/ZigBee_SmartMeter_Reader

Code: Select all

esp_err_t smartmeter_init()
{
    /* === CONFIGURE UART ===*/
    /* Create basic configuration */
    uart_config_t uart_config = {
        .baud_rate = UART_BAUD_RATE,
        .data_bits = UART_DATA_8_BITS,
        .parity = UART_PARITY_EVEN,
        .stop_bits = UART_STOP_BITS_1,
        .flow_ctrl = UART_HW_FLOWCTRL_DISABLE,
        .source_clk = UART_SCLK_DEFAULT,
    };

    /* Set configuration */
    esp_err_t err = uart_param_config(UART_PORT_NUMBER, &uart_config);
    if(err != ESP_OK){ return err; }

    /* Create interrupt configuration */
    uart_intr_config_t uart_intr = {
        .intr_enable_mask = UART_INTR_RXFIFO_TOUT,
        .rx_timeout_thresh = RX_TIMEOUT_TRESHOLD,
    };

    /* Set interrupt configuration */
    err = uart_intr_config(UART_PORT_NUMBER, &uart_intr);
    if(err != ESP_OK){ return err; }

    /* Enable interrupts */
    err = uart_enable_rx_intr(UART_PORT_NUMBER);
    if(err != ESP_OK){ return err; }

    /* Set communication pins */
    err = uart_set_pin(UART_PORT_NUMBER, UART_TX_GPIO, UART_RX_GPIO, UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE);
    if(err != ESP_OK){ return err; }

    /* Install driver */
    err = uart_driver_install(UART_PORT_NUMBER, DATA_BUFFER_SIZE, 0, 20, &uart1_queue, 0);
    if(err != ESP_OK){ return err; }

    /* Create a task to handle events */
    xTaskCreate(uart_event_task, "uart_event_task", DATA_BUFFER_SIZE + 2048, NULL, 12, NULL);

    return err;
}

I thought it is handled via a uart event, but I think I'm wrong, but I didn't find any example in the internet.
Can someone guide me? Or is there a better way to process the two frames?

Thanks for your help

[MicroController]

I thought it is handled via a uart event, but I think I'm wrong, but I didn't find any example in the internet.

https://github.com/espressif/esp-idf/tr ... art_events

[Tropaion]

I have read this example a hundred times. Sadly it doesn't explain anything about how to catch and handle the RX Timeout Interrupt.

[MicroController]

Notice that uart_driver_install(...) sets up the UART interrupts for the driver's requirements, so any interrupt configuration you do beforehand is overwritten.

What are you trying to do?

I recommend using the IDF UART driver and just processing the events it sends you. It does all the interrupt handling for you.
You can use uart_set_rx_timeout(...) with it.

Note that, for receiving, there are two relevant interrupts: UART_INTR_RXFIFO_FULL and UART_INTR_RXFIFO_TOUT. These two work together:
As long as data is continuously flowing, UART_INTR_RXFIFO_FULL is raised whenever the RX FIFO is about to be full. This implies that data is available in the FIFO to be read.
At the time the inbound data pauses, the FIFO may be only partially filled, so no UART_INTR_RXFIFO_FULL is raised. This is when the TOUT setting/interrupt come into play. The TOUT is raised when there was no new data received for the specified time, indicating that there is data in the RX FIFO to read, the FIFO is not full, but the other side may have paused sending.

That said, RXFIFO_FULL and RXFIFO_TOUT interrupts can be handled identically, i.e. by consuming available data from the RX FIFO. Reacting to RXFIFO_TOUT only is not sufficient because the FIFO can overflow without ever raising a TOUT.


I don't know the structure of the messages you're receiving, but if possible, I would advise to use the actual data received to determine when/where a message ends instead of relying on a timeout.

[Tropaion]

Hello @MicroController,
thanks for your help.

I don't know the structure of the messages you're receiving, but if possible, I would advise to use the actual data received to determine when/where a message ends instead of relying on a timeout.

I would also prefer it that way, but sadly in my opinion the smartmeter interface really sucks, and every smartmeter is different.
As far as I found out, most smartmeters like mine, which use M-BUS for communication send either alway one frame or two after a time intervall of a few seconds. Mine sends two frames every 10 seconds. These two frames consists of 282 Bytes.
Sadly I don't think there is a way to know if a second frame follows after the first one, or I simply didn't find it.
The structure of my data (as far as I found out) looks like this:

My idea was to process the received data if I don't receive anything for, maybe a second or so.
But as far as I understood from your explanation, UART_INTR_RXFIFO_FULL and UART_INTR_RXFIFO_TOUT is not usefull for this, since I think the FIFO is full before these 282 Bytes are send.
Can you recommend/propose a better solution?

[MicroController]

Using the UART driver's events, a very simple solution could look like this:

Code: Select all

    const uint32_t timeout_ms = 1000;
    
    const TickType_t timeout_ticks = timeout_ms / portTICK_PERIOD_MS;
    
    uart_event_t event;
    
    for(;;) {
    
        // Wait indefinitely for the first byte(s), but not more than timeout_ms for all other bytes.
        
        bool data_received = false;
        
        while( xQueueReceive(uart0_queue, (void * )&event, data_received ? timeout_ticks : portMAX_DELAY ) ) {
            switch(event.type) {

                case UART_DATA:
                    data_received = true; // We have at least some data. Limit further waiting time.
                    break;
                ...
            }
        }
        
        // No UART event received within the given timeframe.
        // See what we got:
        
        size_t bytes_available = 0;
        
        uart_get_buffered_data_len(EX_UART_NUM, &bytes_available);

        if(bytes_available > 0) {
            
            uart_read_bytes(EX_UART_NUM, my_frame_buffer, bytes_available, portMAX_DELAY);
            
            // RX'd data is now in "my_frame_buffer"
            
            // Handle data, or pass on to another task...

        }
    }

We just let any received data sit in the UART driver's RX buffer until we receive no event from the driver for 1 second.
Then we check how much data is available and process it.

This example is of course very minimalistic. One would want to add some checks/error handling, e.g. in case the RX buffer becomes full before a timeout happens.

[Tropaion]

Looks pretty good, thank you, I will try that.

[Tropaion]

I now implemented you example and tried it out, but it doesn't work.
Not sure why, but the while-loop is alway instantly left after exactly 120 bytes where received.
I tried increasing the timeout but it feels like this doesn't change anything at all.
This is my code:

Code: Select all

#define UART_RX_TIMEOUT		7000
#define DATA_BUFFER_SIZE		1024
static void uart_event_task(void *pvParameters)
{
    /* Store current event */
    uart_event_t event;

    /* First buffer for processing data */
    uint8_t buff0[DATA_BUFFER_SIZE];

    /* Second buffer for processing data */
    uint8_t buff1[DATA_BUFFER_SIZE];

    /* Ticks to wait before received bytes are processed */
    const TickType_t timeout_ticks = UART_RX_TIMEOUT / portTICK_PERIOD_MS;

    for(;;)
    {
        /* Remember if first data after timeout was received */
        bool data_received = false;

        /* Loop infinitely and check for event */
        /* Wait indefinitely for the first byte(s), but not more than timeout_ms for all other bytes */
        if(xQueueReceive(uart1_queue, (void *)&event, data_received ? timeout_ticks : portMAX_DELAY))
        {
            /* Switch according to event type */
            switch(event.type)
            {
                /* Received data */
                case UART_DATA:
                    /* Set data received to true */
                    data_received = true;
                    break;

                /* FIFO Overflow */
                case UART_FIFO_OVF:
                    ESP_LOGI(TAG, "UART: FIFO overflow");

                    /* Flush ringbuffer */
                    uart_flush_input(UART_PORT_NUMBER);

                    /* Reset event queue */
                    xQueueReset(uart1_queue);
                    break;

                /* Ringbuffer full */
                case UART_BUFFER_FULL:
                    ESP_LOGI(TAG, "UART: Ringbuffer full");
                    
                    /* Flush ringbuffer */
                    uart_flush_input(UART_PORT_NUMBER);

                    /* Reset event queue */
                    xQueueReset(uart1_queue);
                    break;

                /* RX break detected */
                case UART_BREAK:
                    ESP_LOGI(TAG, "UART: RX Break");
                    break;

                /* Parity check error */
                case UART_PARITY_ERR:
                    ESP_LOGI(TAG, "UART: Parity Error");
                    break;

                /* Frame error */
                case UART_FRAME_ERR:
                    ESP_LOGI(TAG, "UART: Frame Error");
                    break;

                /* Other events */
                default:
                    /* Write event to log */
                    ESP_LOGI(TAG, "UART: Event Type: %d", event.type);
                    break;
            }
        }

        /* Set buffer sizes */
        size_t buff0_size = 0;
        size_t buff1_size = 0;

        /* No data received within the given time UART_RX_TIMEOUT */
        /* Check how many bytes were received */
        uart_get_buffered_data_len(UART_PORT_NUMBER, &buff0_size);

        /* If received bytes are more than minimum of MBUS frame */
        if(buff0_size >= (MBUS_HEADER_LENGTH + MBUS_FOOTER_LENGTH))
        {
            /* Write bytes to buffer0 */
            uart_read_bytes(UART_PORT_NUMBER, &buff0, buff0_size, portMAX_DELAY);

            //TEST: PRINT DATA
            printf("Read bytes: %d\n", buff0_size);
        	for (int i = 0; i < buff0_size; i++)
        	{
        	    printf("%02X", buff0[i]);
        	}
            printf("\n");

            /* Process received data from buffer0 and write to buffer1 */
            esp_err_t err = parse_mbus_long_frame_layer(&buff0[0], buff0_size, &buff1[0], &buff1_size);
            
            /* Check if mbus parsing was successfull */
            if(err == ESP_OK)
            {
                /* Process mbus data from buffer1 and write to buffer0 */
                err = parse_dlms_layer(&buff1[0], buff1_size, &buff0[0], &buff0_size, &gue_key[0]);
            }

            /* Check if dlms parsing was successfull */
            if(err == ESP_OK)
            {
                /* Process dlms data from buffer0 */
                err = parse_obis(&buff0[0], buff0_size);
            }
            
            /* Check if parsing failed */
            if(err != ESP_OK)
            {
                ESP_LOGE(TAG, "UART: Parsing failed.");
                
                /* Flush ringbuffer */
                uart_flush_input(UART_PORT_NUMBER);

                /* Reset event queue */
                xQueueReset(uart1_queue);
            }
        }
    }
    /* Delete this task */
    vTaskDelete(NULL);
}

esp_err_t smartmeter_init()
{
    /* === CONFIGURE UART ===*/
    /* Create basic configuration */
    uart_config_t uart_config = {
        .baud_rate = UART_BAUD_RATE,
        .data_bits = UART_DATA_8_BITS,
        .parity = UART_PARITY_EVEN,
        .stop_bits = UART_STOP_BITS_1,
        .flow_ctrl = UART_HW_FLOWCTRL_DISABLE,
        .source_clk = UART_SCLK_DEFAULT,
    };

    /* Set configuration */
    esp_err_t err = uart_param_config(UART_PORT_NUMBER, &uart_config);
    if(err != ESP_OK){ return err; }

    /* Set communication pins */
    err = uart_set_pin(UART_PORT_NUMBER, UART_TX_GPIO, UART_RX_GPIO, UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE);
    if(err != ESP_OK){ return err; }

    /* Install driver */
    err = uart_driver_install(UART_PORT_NUMBER, DATA_BUFFER_SIZE, 0, 20, &uart1_queue, 0);
    if(err != ESP_OK){ return err; }

    /* Create a task to handle events */
    xTaskCreate(uart_event_task, "uart_event_task", ((3 * DATA_BUFFER_SIZE) + 2048), NULL, 12, NULL);

    return err;
}

This is my output after a while:

I (23) boot: ESP-IDF v5.2-dev-1736-ga2d76ad38a 2nd stage bootloader
I (23) boot: compile time Aug 15 2023 18:02:34
I (24) boot: chip revision: v0.0
I (27) boot.esp32c6: SPI Speed : 80MHz
I (32) boot.esp32c6: SPI Mode : DIO
I (37) boot.esp32c6: SPI Flash Size : 2MB
I (42) boot: Enabling RNG early entropy source...
I (47) boot: Partition Table:
I (51) boot: ## Label Usage Type ST Offset Length
I (58) boot: 0 nvs WiFi data 01 02 00009000 00006000
I (65) boot: 1 phy_init RF data 01 01 0000f000 00001000
I (73) boot: 2 factory factory app 00 00 00010000 000a2800
I (80) boot: 3 zb_storage Unknown data 01 81 000b3000 00004000
I (88) boot: 4 zb_fct Unknown data 01 81 000b7000 00000400
I (95) boot: End of partition table
I (99) esp_image: segment 0: paddr=00010020 vaddr=42018020 size=0bb90h ( 48016) map
I (118) esp_image: segment 1: paddr=0001bbb8 vaddr=40800000 size=04460h ( 17504) load
I (123) esp_image: segment 2: paddr=00020020 vaddr=42000020 size=14da4h ( 85412) map
I (142) esp_image: segment 3: paddr=00034dcc vaddr=40804460 size=06474h ( 25716) load
I (179) esp_image: segment 4: paddr=0003b248 vaddr=4080a8e0 size=010f8h ( 4344) load
I (184) boot: Loaded app from partition at offset 0x10000
I (184) boot: Disabling RNG early entropy source...
I (399) cpu_start: Unicore app
I (400) cpu_start: Pro cpu up.
W (409) clk: esp_perip_clk_init() has not been implemented yet
I (415) cpu_start: Pro cpu start user code
I (416) cpu_start: cpu freq: 160000000 Hz
I (416) cpu_start: Application information:
I (418) cpu_start: Project name: smartmeter
I (424) cpu_start: App version: PCB_v0.2-27-g7db03cb-dirty
I (430) cpu_start: Compile time: Aug 15 2023 18:12:10
I (436) cpu_start: ELF file SHA256: 87d2aad35...
I (442) cpu_start: ESP-IDF: v5.2-dev-1736-ga2d76ad38a
I (448) cpu_start: Min chip rev: v0.0
I (453) cpu_start: Max chip rev: v0.99
I (458) cpu_start: Chip rev: v0.0
I (463) heap_init: Initializing. RAM available for dynamic allocation:
I (470) heap_init: At 4080C8B0 len 0006FD60 (447 KiB): D/IRAM
I (476) heap_init: At 4087C610 len 00002F54 (11 KiB): STACK/DIRAM
I (483) heap_init: At 50000000 len 00003FE8 (15 KiB): RTCRAM
I (490) spi_flash: detected chip: generic
I (494) spi_flash: flash io: dio
W (498) spi_flash: Detected size(4096k) larger than the size in the binary image header(2048k). Using the size in the binary image header.
I (511) sleep: Configure to isolate all GPIO pins in sleep state
I (518) sleep: Enable automatic switching of GPIO sleep configuration
I (525) coexist: coex firmware version: 80b0d89
I (530) coexist: coexist rom version 5b8dcfa
I (535) app_start: Starting scheduler on CPU0
I (540) main_task: Started on CPU0
I (540) main_task: Calling app_main()
I (540) uart: queue free spaces: 20
I (550) main_task: Returned from app_main()
Read bytes: 120
68FAFA6853FF000167DB085341475905EAB33D81F820007B3E18A20403A12F98B82AD3C4C325F9150AD46B91F1A063BB6ADDE519990B6FC707D22ABC3A6E3522BE62FE011463EFA3BB329F05DF6B8A92C7E1F412F1D1A5A40CA949FDA262F2FE80A256091448C00DC1B4315855F4628DA9439592CDBDA925
E (1640) smartmeter: MBUS: Invalid stop byte!
E (1640) smartmeter: UART: Parsing failed.
Read bytes: 120
BB5C695AC0EFB3262D8F6C3CD602EFE5E7A3B49FEEF10A17003F301F521C2B3EB94464B1F4E73AEC273DB73B149675ECDAC32478647DAD3C732C260F75D8EEAB817A55E9C1F7A23E1BB5BF7C3AB31564F09423D327467F4A7D00303EAE7F9DA580D5C9717D3931A5C94A948275AB1978A2B188687316CD43
E (2210) smartmeter: MBUS: Invalid start bytes!
E (2210) smartmeter: UART: Parsing failed.
Read bytes: 34
2402B31FAC79E0166814146853FF11016707EF424C046D557FE14B7F05DD475EC616
E (2420) smartmeter: MBUS: Invalid start bytes!
E (2420) smartmeter: UART: Parsing failed.
Read bytes: 120
68FAFA6853FF000167DB085341475905EAB33D81F820007B3E19A936FD6D1887339F222A598B479154BDB7EF2075CEA82BF7A4A6B4B3849ACD603900F2BD6EBFF830A5F125A4F4CA7EDA61E97BB3E95A067CEE623745D034854F89EA05ABB8F8B25C502D24B8FA9B8CEA0A160BCE8B39A7C48306D4346FFF
E (6640) smartmeter: MBUS: Invalid stop byte!
E (6640) smartmeter: UART: Parsing failed.
Read bytes: 120
3A712359E7A1BAD533DAF19723BB596E7E14F241FABD162B2DF5613981EC9C47B1F081D7D1C0A3FA45A694A72FAB1781599043ADBA80A96F013529875986BF4872211F5F868E270D372A67589D524434F34D94701F483075197B12CC52E7FEAF48939571418BE0E3ECD3CC1549B855FF0D222C5ADB56A298
E (7210) smartmeter: MBUS: Invalid start bytes!
E (7210) smartmeter: UART: Parsing failed.
Read bytes: 34
2207F715749342166814146853FF110167997AD23DB7A23AE7FDD792879F8ED45516
E (7420) smartmeter: MBUS: Invalid start bytes!
E (7420) smartmeter: UART: Parsing failed.

So basically the 274 bytes I want at once are separated into three parts. After the 34 bytes are sent, the next data is delayed for a while. This could only happen when xQueueReceive returns false, but I don't know why it should to that.

[MicroController]

Not sure why, but the while-loop is alway instantly left after exactly 120 bytes where received.

That's because

Code: Select all

if(xQueueReceive(uart1_queue, (void *)&event, data_received ? timeout_ticks : portMAX_DELAY))

is not a (while) loop

[MicroController]

Btw,

Code: Select all

    uint8_t buff0[DATA_BUFFER_SIZE];
    uint8_t buff1[DATA_BUFFER_SIZE];    

is not a good idea to have as local variables. Make them (static) globals or malloc memory from the heap.