ESP32 Forum

Posted: **Fri May 24, 2024 7:18 am**

Hello guys!

Please help me understand IDF FreeRTOS task usages.

I have wrote a function which monitors all of the running tasks and saves their stats to an Arduino Json Document.
(I'm using Arduino as a component of IDF with PlatformIO)

Here is the method which tracks and retrives task information

Code: Select all

void Sys::getTaskInfo(JsonObject& taskInfo) {
    JsonArray taskList = taskInfo["taskList"].to<JsonArray>();
    UBaseType_t uxArraySize;
    TaskStatus_t* taskStatusArray;
    uint32_t ulTotalRunTime, ulTaskRuntimePercentage;
    uxArraySize = uxTaskGetNumberOfTasks();
    taskStatusArray = (TaskStatus_t*)pvPortMalloc(uxArraySize * sizeof(TaskStatus_t));

    if (taskStatusArray == NULL) {
        return;
    }
    // Since CONFIG_FREERTOS_RUN_TIME_STATS_USING_ESP_TIMER is set to true
    // The clock uses the esp timer which is 1Mhz
    // No convresion needed in this case, the time base is already in milliseconds
    uxArraySize = uxTaskGetSystemState(taskStatusArray, uxArraySize, &ulTotalRunTime);

    if (ulTotalRunTime > 0) {
        taskInfo["totalRunTime"] = ulTotalRunTime;

        for (UBaseType_t i = 0; i < uxArraySize; i++) {
            JsonObject task = taskList.add<JsonObject>();
            TaskStatus_t* taskStatus = &taskStatusArray[i];

            // Convert task runtime to the same unit
            uint32_t taskRunTime = taskStatus->ulRunTimeCounter;
            uint64_t tempPercentage = (static_cast<uint64_t>(taskRunTime) * 100UL) / ulTotalRunTime;
            ulTaskRuntimePercentage = static_cast<uint32_t>(tempPercentage);

            // Store the task information
            task["name"] = taskStatus->pcTaskName;
            task["state"] = taskStatus->eCurrentState;
            task["priority"] = taskStatus->uxCurrentPriority;
            task["basePriority"] = taskStatus->uxBasePriority;
            task["stackHighWaterMark"] = taskStatus->usStackHighWaterMark;
            task["coreId"] = taskStatus->xCoreID;
            task["number"] = taskStatus->xTaskNumber;
            task["runTimeCounter"] = taskRunTime;
            task["runTimePercent"] = ulTaskRuntimePercentage;
            // Stack size can not be retrived from taskStatus. There is a separate logic for that.
            task["stackSize"] = taskHandler.getStackSize(taskStatus->xHandle);
        }
    }

    vPortFree(taskStatusArray);
}

I'm using it like this ( with ArduinoJson V7 )
This function is called every minute

Code: Select all

void Sys::getSysInfo(JsonDocument& info) {
    JsonObject taskInfos = info["tasks"].to<JsonObject>();
    getTaskInfo(taskInfos);
}

It is working great so far.

Here is some example results

Code: Select all

[
      {
        "name": "systemTask",
        "state": 1,
        "priority": 1,
        "basePriority": 1,
        "stackHighWaterMark": 13296,
        "coreId": 0,
        "number": 10,
        "runTimeCounter": 41724638,
        "runTimePercent": 2,
        "stackSize": 16384
      },
      {
        "name": "IDLE",
        "state": 1,
        "priority": 0,
        "basePriority": 0,
        "stackHighWaterMark": 912,
        "coreId": 1,
        "number": 6,
        "runTimeCounter": 1624700697,
        "runTimePercent": 87,
        "stackSize": 0
      },
      {
        "name": "IDLE",
        "state": 1,
        "priority": 0,
        "basePriority": 0,
        "stackHighWaterMark": 940,
        "coreId": 0,
        "number": 5,
        "runTimeCounter": 1579542035,
        "runTimePercent": 84,
        "stackSize": 0
      },
      {
        "name": "loopTask",
        "state": 2,
        "priority": 1,
        "basePriority": 1,
        "stackHighWaterMark": 12504,
        "coreId": 1,
        "number": 8,
        "runTimeCounter": 109263130,
        "runTimePercent": 5,
        "stackSize": 0
      }
]

As you can see, the loopTask and the systemTask is below 10% usage.
For some reason I have TWO IDLE tasks and they both at 84%.

Couple of questions.

Why is there two IDLE tasks? ( one for each core? )

Why is the IDLE task get more time than my other tasks?

Please explain the following tasks, what they do and why are they there.

Tasks that aren't documneted or I couldn't found an explanation for

tiT

Tmr Svc

ipc0

ipc1

emac_rx

sys_evt

Your help is greatly appreciated and thank you for taking the time to read it!

Posted: **Fri May 24, 2024 8:20 am**

HitecSmartHome wrote: ↑
Fri May 24, 2024 7:18 am

Why is there two IDLE tasks? ( one for each core? )

Yes, each core has its own IDLE task.

Why is the IDLE task get more time than my other tasks?

Because in FreeRTOS there must be one task active at any point in time. When all other tasks are inactive (waiting/blocked), the IDLE task becomes the active task. For the most part, the IDLE tasks do nothing but make the CPU wait for the next interrupt, but this waiting time counts towards their CPU time in FreeRTOS. The IDLE tasks have the lowest priority, so whenever any other task becomes runnable again, FreeRTOS will immediately switch from the IDLE task to the other task.
Hence, the time spent in an IDLE task is basically the idle time of the CPU/core; i.o.w. 100%-(IDLE task %) is the actual CPU load of one core.

Please explain the following tasks, what they do and why are they there.

tiT

Dunno.

Tmr Svc

Should be the task for handling of timer events (esp_timer).

ipc0

ipc1

"Inter-processor call" tasks, one on each core. These would handle/execute inter-processor calls when code on one core needs the other core to perform some action.

emac_rx

Dunno.

sys_evt

Sounds like an event processing task. Default event loop, maybe.

Posted: **Fri May 24, 2024 9:08 am**

Thank you very much for the explanations.
Almost all of my tasks are higher priority than the IDLE task.
They do a

Code: Select all

vTaskDelay(1);

in every iteration to feed the system but I expected them to run much more than the IDLE task.
This is a little bit strange to me.
I did not experience any performance bottleneck but how would I make my tasks run more?

Would it be possible for them to run more, thus increasing the overall performance of my system?
I have several other tasks to run.

The tiT task looks like this from the taskList

Code: Select all

     {
        "name": "tiT",
        "state": 2,
        "priority": 18,
        "basePriority": 18,
        "stackHighWaterMark": 1868,
        "coreId": 2147483647,
        "number": 11,
        "runTimeCounter": 11584659,
        "runTimePercent": 0,
        "stackSize": "Cannot be extracted"
      }

Posted: **Fri May 24, 2024 10:47 am**

HitecSmartHome wrote: ↑
Fri May 24, 2024 9:08 am
Almost all of my tasks are higher priority than the IDLE task.
They do a
Code: Select all
vTaskDelay(1);
in every iteration to feed the system but I expected them to run much more than the IDLE task.
This is a little bit strange to me.
I did not experience any performance bottleneck but how would I make my tasks run more?

Would it be possible for them to run more, thus increasing the overall performance of my system?

Yes. (polling and) vTaskDelay()-ing just to yield the CPU to other tasks should be avoided. Instead, tasks should use blocking operations/APIs which improves reaction times and can reduce CPU load (e.g. context-switching overhead) at the same time.

vTaskDelay(1) would be 'sleeping' for ~10ms by default; usually, the actual computations you do in one task loop's iteration run much faster than that. If one iteration would take, say, 1ms to run, then sleep for 10ms, the maximum CPU utilization that one task can achieve would be 1ms/(1ms+10ms)~9%. And you'll always have a 'laggy' response of that task, because it will only react to any event once every 10ms.

Posted: **Fri May 24, 2024 11:22 am**

MicroController wrote: ↑
Fri May 24, 2024 10:47 am
HitecSmartHome wrote: ↑
Fri May 24, 2024 9:08 am
Almost all of my tasks are higher priority than the IDLE task.
They do a
Code: Select all
vTaskDelay(1);
in every iteration to feed the system but I expected them to run much more than the IDLE task.
This is a little bit strange to me.
I did not experience any performance bottleneck but how would I make my tasks run more?

Would it be possible for them to run more, thus increasing the overall performance of my system?
Yes. (polling and) vTaskDelay()-ing just to yield the CPU to other tasks should be avoided. Instead, tasks should use blocking operations/APIs which improves reaction times and can reduce CPU load (e.g. context-switching overhead) at the same time.

vTaskDelay(1) would be 'sleeping' for ~10ms by default; usually, the actual computations you do in one task loop's iteration run much faster than that. If one iteration would take, say, 1ms to run, then sleep for 10ms, the maximum CPU utilization that one task can achieve would be 1ms/(1ms+10ms)~9%. And you'll always have a 'laggy' response of that task, because it will only react to any event once every 10ms.

Oh, thats nice to hear, thank you.
But my tasks are independent from each other. How would you use blocking operations in this case?

task1 -> doing it's own things -> yielding so the OS can task switch
task2 -> doing it's own things -> yielding so the OS can task switch

Posted: **Fri May 24, 2024 1:50 pm**

Do i need a main task which loops forever and controlls all the other tasks?

Posted: **Fri May 24, 2024 3:07 pm**

tiT is LWIP's "TCP IP Task".
emac_rx is the receiving task for an Ethernet MAC layer.

Posted: **Fri May 24, 2024 7:05 pm**

HitecSmartHome wrote: ↑
Fri May 24, 2024 11:22 am
But my tasks are independent from each other. How would you use blocking operations in this case?

task1 -> doing it's own things -> yielding so the OS can task switch
task2 -> doing it's own things -> yielding so the OS can task switch

In almost all cases, tasks don't do calculations all the time; they usually have to wait for some event to happen before they can continue processing. This "event" may come from another task or from hardware via an interrupt/ISR.

Can't tell you more since I don't know any of your code/tasks, but as I said, actively polling for some event should and can be avoided most of the time. The key here is to use FreeRTOS's facilities like queues or semaphores, which provide a blocking API. Many IDF components/drivers already do this for you under the hood; basically every (IDF) function which accepts a TickType_t maxwait argument can and should be used with a non-zero timeout tick count, possibly even portMAX_DELAY for an infinite timeout, to block until some event happens.

Btw, the "Arduino pattern" of having a single loop which does 'everything' in your application is not how things should be done in a multi-tasking environment like FreeRTOS.

Posted: **Mon May 27, 2024 7:02 am**

MicroController wrote: ↑
Fri May 24, 2024 7:05 pm

HitecSmartHome wrote: ↑
Fri May 24, 2024 11:22 am
But my tasks are independent from each other. How would you use blocking operations in this case?

task1 -> doing it's own things -> yielding so the OS can task switch
task2 -> doing it's own things -> yielding so the OS can task switch
In almost all cases, tasks don't do calculations all the time; they usually have to wait for some event to happen before they can continue processing. This "event" may come from another task or from hardware via an interrupt/ISR.

Can't tell you more since I don't know any of your code/tasks, but as I said, actively polling for some event should and can be avoided most of the time. The key here is to use FreeRTOS's facilities like queues or semaphores, which provide a blocking API. Many IDF components/drivers already do this for you under the hood; basically every (IDF) function which accepts a TickType_t maxwait argument can and should be used with a non-zero timeout tick count, possibly even portMAX_DELAY for an infinite timeout, to block until some event happens.

Btw, the "Arduino pattern" of having a single loop which does 'everything' in your application is not how things should be done in a multi-tasking environment like FreeRTOS.

I see.
I have several classes, each class have it's own task.

Here is some example of my architecture

Time class -> loops forever, tracks the time, refreshes from ntp or from rtc ( it has to run since this is the class that gives the time to other classes and it has to be up to date )

Hardware class -> loops forever, communicates with other hardwares which are connected to it via modbus ( it has to run since this is the class that continously need to push and pull data to/from the hardwares )

Component class -> loops forever, handles user defined things like lights,thermostats etc.. ( it has to run since this task needs live data from the Hardware class and update the states of the components accordingly )

These classes are independent from each other in a sense that they make their own states and manages their own data.

For example the Time class can't wait for any event, it has to track the time no matter what. It has to update it's internal time structure so it is up to date when others ask it. It also handles an uptime counter which also has no external event factor. It just loops and checks and stores the uptime of the chip.

Hardware class also has no need for any external event. It manages the communication between the connected hardwares. It has to be continous and has to have live data. It must be as fast as it can. The flow is that it pushes data to uart, waits for the response and it pushes the next packet as soon as it can.

Anyway, I will think trought what you just said, which I thank you very much. I hope I can get a good grap on the concept and will rethink my approach.

Posted: **Mon May 27, 2024 8:43 am**

HitecSmartHome wrote: ↑
Mon May 27, 2024 7:02 am
For example the Time class ... just loops and checks and stores the uptime of the chip.

Why does it "store" the uptime? Can't it just "calculate"/provide the uptime when requested?

Hardware class ... pushes data to uart, waits for the response and it pushes the next packet as soon as it can.

This task should use the blocking UART API. I.e.,
do not do something like

Code: Select all

// BAD:
// Busy waiting:
while(!UartDataAvailable()) {
// Nothing here, just burning CPU cycles.
}

but instead:

Code: Select all

size_t rxLen;
while((rxLen = uart_read_bytes(..., portMAX_DELAY)) >= 0) {
// Process rxLen bytes of received data...
}

Component class ... needs live data from the Hardware class and update the states of the components accordingly )

The Component task could wait for the Hardware task to provide new data, e.g. via a queue (xQueueSend() in Hardware task, xQueueReceive() in Component).

ESP32 Forum

Task usage understanding

Task usage understanding

Re: Task usage understanding

Re: Task usage understanding

Re: Task usage understanding

Re: Task usage understanding

Re: Task usage understanding

Re: Task usage understanding

Re: Task usage understanding

Re: Task usage understanding

Re: Task usage understanding