Page 1 of 2

What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Sun Sep 10, 2017 11:01 pm
by tuskiomi
I have a simple sketch for the arduino DE that has 3 tasks running simultaneously. On seemingly random timings, I get the following error:
Task watchdog got triggered. The following tasks did not feed the watchdog in time:
Tasks currently running:
CPU 0: ALTIMITER
CPU 1: IDLE
With no rhyme or reason as to WHEN this starts happening, it always eventually will, and when it starts, it repeats, and does not let my code's debug output print to serial.

I'm at a loss for what is even happening.

What is a watchdog?
Why does it get triggered?
What is feeding it?
what is it being fed?
What are common causes of watchdog starvation?

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Mon Sep 11, 2017 1:42 am
by ESP_Sprite
You may want to read up on the esp-idf watchdogs: http://esp-idf.readthedocs.io/en/latest ... /wdts.html .

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Mon Sep 11, 2017 2:04 am
by tuskiomi
Yes. I've read that. I can't make heads or tails of it. For example:
By default, the task watchdog watches the idle tasks.
Does this mean it's also fed by idle tasks? what's an 'Idle task'? one that's called vTaskDelay?
The usual cause of idle tasks not feeding the watchdog is a higher-priority process looping without yielding to the lower-priority processes, and can be an indicator of badly-written code that spinloops on a peripheral or a task that is stuck in an infinite loop.
How does a task Yield to another? why would a HIGHER priority task yield to a LOWER priority task? This happened when I only had 2 tasks as well. With 2 cores, this should never happen, no? what are the idle tasks actually feeding the watchdogs? The two loops are using I2c. is this indicative of a non-responsive device?
Other task can elect to be watched by the task watchdog by calling esp_task_wdt_feed(). Calling this routine for the first time will register the task to the task watchdog; calling it subsequent times will feed the watchdog. If a task does not want to be watched anymore (e.g. because it is finished and will call vTaskDelete() on itself), it needs to call esp_task_wdt_delete().
So a watched task must feed the dog, but an unwatched task doesn't? what happens when the dog would normally throw a fit?
The task watchdog is built around the hardware watchdog in timer group 0.

This is the processor clock, right?
If this watchdog for some reason cannot execute the interrupt handler that prints the task data (e.g. because IRAM is overwritten by garbage or interrupts are disabled entirely) it will hard-reset the SOC.
so a dead watchdog makes the software reset?

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Mon Sep 11, 2017 2:23 am
by kolban
What APIs or libraries or logic are you using to create tasks in an Arduino environment?

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Mon Sep 11, 2017 2:40 am
by ESP_Sprite
The watchdog timer issue you're running into is that by default, the idle task calls the watchdog as well. It's needed for the idle task to run every now and then, because it's responsible for household tasks like cleaning up TCBs of deleted tasks, so by default it feeds a task watchdog. If this watchdog triggers, it essentially means a high-priority task is using up all the CPU time, without calling (directly or indirectly) a FreeRTOS function that yields execution from that high-prio task to lower-prio tasks.

Also, the task watchdog warning you see is just that: a warning. If your code doesn't run properly, it's because of something else.

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Mon Sep 11, 2017 7:43 pm
by tuskiomi
ESP_Sprite wrote:The watchdog timer issue you're running into is that by default, the idle task calls the watchdog as well. It's needed for the idle task to run every now and then, because it's responsible for household tasks like cleaning up TCBs of deleted tasks, so by default it feeds a task watchdog. If this watchdog triggers, it essentially means a high-priority task is using up all the CPU time, without calling (directly or indirectly) a FreeRTOS function that yields execution from that high-prio task to lower-prio tasks.

Also, the task watchdog warning you see is just that: a warning. If your code doesn't run properly, it's because of something else.
Yes, but remember that the other CPU is IDLE. Why won't the task spool up on that core?

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Tue Sep 12, 2017 1:04 am
by ESP_Sprite
Because of the way things work, we have two idle tasks. This means that both CPUs need to be able to run the idle task every now and then.

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Thu Jan 10, 2019 6:10 pm
by burkulesomesh43
ESP_Sprite wrote:
Mon Sep 11, 2017 1:42 am
You may want to read up on the esp-idf watchdogs: http://esp-idf.readthedocs.io/en/latest ... /wdts.html .
Is there any effect if we disable the watchdogs ??
I am using upto 10 tasks in my code but giving me an TG1WDT RESET ERROR. So I want to remove this error to prevent esp32 from reset. I want my esp32 live for long time.
Could you please explain if any issue by disabling watchdogs?

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Thu Jan 10, 2019 7:20 pm
by mikemoy
burkulesomesh43 wrote:
Thu Jan 10, 2019 6:10 pm
ESP_Sprite wrote:
Mon Sep 11, 2017 1:42 am
You may want to read up on the esp-idf watchdogs: http://esp-idf.readthedocs.io/en/latest ... /wdts.html .
Is there any effect if we disable the watchdogs ??
I am using upto 10 tasks in my code but giving me an TG1WDT RESET ERROR. So I want to remove this error to prevent esp32 from reset. I want my esp32 live for long time.
Could you please explain if any issue by disabling watchdogs?
You might want to re-visit how your coding the application. you should not be starving the watchdog. In any even you can change its settings in menuconfig

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Posted: Thu Jan 10, 2019 9:47 pm
by fly135
burkulesomesh43 wrote:
Thu Jan 10, 2019 6:10 pm
Is there any effect if we disable the watchdogs ??
I am using upto 10 tasks in my code but giving me an TG1WDT RESET ERROR. So I want to remove this error to prevent esp32 from reset. I want my esp32 live for long time.
Could you please explain if any issue by disabling watchdogs?
Is the WD resetting your ESP32? If you preform a make menuconfig, there is a setting under component config/esp32-specific called "Invoke panic handler on Task watchdog timeout". If you un-select that, then the WD timeouts will print bu not reset the chip or affect anything except for printing the statement. You can also change the time from 60 secs to a higher number.

The reason you are getting the WD statement is because the task ALTIMITER is executing with no statements to allow other lower priority tasks to execute. The IDLE task is a low priority task that tells the WD that everything is OK because the application is not hung. If you don't care about the device rebooting itself if it's hung then you don't need a WD.

The reason that ALTIMITER is not allowing the IDLE task to run in a 60 sec period is the issue here. If you know that you have 100 secs of processing in ALTIMITER before you use some mechanism to allow the task to go idle then you can just change the WD timeout to a number higher than 100. Or maybe the design of your app doesn't let tasks other than ALTIMITER to run. Only you know what your intentions are.

It's always a good idea to have a task supervise the application and make sure it is functioning. The IDLE task is the default "supervisor". When it doesn't run it lets you know with a WD. You can code up an alternative way of supervising. But if the supervisor task never gets invoked because your application is hung up, then the only way the supervisor can do the job of resetting the ESP32 is with a WD of some kind.

John A