Error trying to download large file using WiFiClient.connect() "Task watchdog got triggered"

[Munque]

I'm testing code downloading large files from a server (working toward an OTA function)
Error I'm getting -- midway through the download...

-> E (15787) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
-> E (15787) task_wdt: - async_tcp (CPU 0/1)
-> E (15787) task_wdt: Tasks currently running:
-> E (15787) task_wdt: CPU 0: IDLE0
-> E (15787) task_wdt: CPU 1: IDLE1
-> E (15787) task_wdt: Aborting.
-> abort() was called at PC 0x400e16af on core 0

...which, per this link, might be the result of the download process hogging resources, keeping the ESP from performing necessary functions?

I'm guessing the client.read() within the while() loop (in the code below) was the culprit.

Tried testing delay() / vTaskDelay() at various durations without any improvement.



Also, still new at C++ coming from JS, so lots of questions:

• Do delay() and/or vTaskDelay() actually free up the ESP32 to handle other tasks, or do they block the processing even further?

• Are delay() and vTaskDelay() different functions or (per this link) is delay() just a wrapper of vTaskDelay()

• Trying to work with char arrays instead of Strings, but having difficulty coercing them to do what I want in various contexts. Any advice in doing so greatly appreciated. (char_str()? String()? itoa()? wtf()?)

• Re IPAddress class: Are those just int arrays? How to turn them into a char or String value of that same IP address?

• I'm under the impression that variables declared in a C++ function are automatically freed up on the function's completion. Correct?

The code below is adapted from an Espressif OTA Example.

Code: Select all

String GetBigFile(){
	WiFiClient thisClient;
	IPAddress thisHost(192, 168, 1, 10);
	char thisPath[]="/testBigFile.ino.esp32.bin"; //Big = about 800k OTA bin file
	String thisPart, theseHeaders, thisBody;
	if(!thisClient.connect(thisHost, 8465)){
		Serial.println("Connection Failed");
		return "{\"Error\":\"Connection Failed\"}";
	}
	Serial.println(thisPath);
	Serial.println(String(thisPath));
	Serial.println("Connection succeeded");
	Serial.print("thisClient.available(): ");
	Serial.println(thisClient.available());
	String thisReq=String("GET ") + String(thisPath) + " HTTP/1.1\r\n" +
		"Host: 192.168.1.10:8465\r\n" +
		"Cache-Control: no-cache\r\n" +
		"Connection: close\r\n\r\n";
	Serial.println("thisReq: " + thisReq);
	thisClient.print(thisReq);
	unsigned long timeout = millis();
	while(thisClient.available()==0) {
		if(millis()-timeout > 5000){
			Serial.println("Client timed out");
			thisClient.stop();
			Serial.println("Client timed out");
			return "{\"Error\":\"Client timed out\"}";
			
		}
	}
	Serial.println("Headers Begin");
	thisPart="Header";
	while(thisClient.available()){
		Serial.println(thisClient.available());
		if(thisPart=="Header"){
			String thisLine=thisClient.readStringUntil('\n');
			theseHeaders.concat(thisLine);
			thisLine.trim();
			if(!thisLine.length()){
				Serial.println("Headers Complete:\n" + theseHeaders + "\n------\n");
				thisPart="Body";
			}
		}else{ //Error happens in this block, after about 50 successful character reads with delay
			char thisChar=thisClient.read();
			thisBody.concat(thisChar);
			//delay(10); //Tested at various durations to see if it adds to or frees up blocking.  It seems to add further blocking?
			//vTaskDelay(15); //Also tested, at various duration.
		}
	}
	Serial.println("Body Complete");
	thisClient.stop();
	return "{\"Headers\":\"" + theseHeaders + "\",\"Body\":\"" + thisBody + "\"}";
}

[fasani]

I would also like to understand exactly how delay(), yield() and so on help to free resources. I understand it more like that the ESP32 needs to have some time to do it's internal WiFi stuff so that's why in large loops, you need to add yield() or a minimal delay so it has time to do that tasks.

What I would suggest in this case is to check how other programs already built do the OTA update. There are many out there including WiFi managers that have this functionality built in.
As pointer, for example this one has an OTA branch included: https://github.com/tzapu/WiFiManager/tree/ota

Please also note that in my experience:

while(thisClient.available())

This can get out the loop before the server received the whole file, or at least it did for me. I do it differently in my own way, I send the Content-Length header in the backend response and fetch all the bytes till the last one. Here a different example just fetching a JPEG (is 10 times smaller than your OTA)

https://github.com/martinberlin/eink-ca ... n.cpp#L318

Just as a reference on how to do the download differently. But again, just want to point out that I had issues looping on client.available() since it sometimes returns false for a short time, and it get's out of the loop. Check if in the way you are doing it is really fetching all the bytes you need for the new program.

[Munque]

fasani -- thanks for the followup and the links. Will look into them.

Here's what I've worked out since the OP:

1 - The problem was calling not calling GetBigFile() per se. The problem was calling the GetBigFile() function within an "AsyncWebServer.on()" listener (assuming the term is "listener" in c++). Running GetBigFile blocks the AsyncWebServer from doing its routine causing the watchdog error.

2 - delay() in the way I was testing it does not free up resources, exactly the opposite: it ties up 100% of the processor. Indeed, testing with a delay() resulted in less remote file getting retrieved before the watchdog error triggered than without. (Side note: There are two cores on the ESP32, so certain things can indeed process simultaneously, but I'm not entirely sure what context that plays out in, and this is not one of them)

One solution that solves the issue completely: Do not run the GetBigFile from within the listener. Use the listener only to set a flag (using a global variable), then use the main loop of the code to check that flag, and run GetBigFile from there.

So like this...

Code: Select all

#include <WiFi.h>
#include <Update.h>
#include "AsyncJson.h"
AsyncWebServer EspServer(80);
bool GetBigFileFlag=false; //This is the flag

void setup(){
	//[Code to set up server etc. here]
	EspServer.on("*", HTTP_POST, [](AsyncWebServerRequest * Req){
		AsyncWebParameter* keyVal=Req->getParam(0);
		String key=keyVal->name();
		String val=keyVal->value();
		if(key=="req" && val=="GetBigFile"){
			Serial.println("Updating...");
			Req->send(200, "text/plain", "Updating...");
			//GetBigFile(); //***** This is the mistake.  Blocks EspServer
			GetBigFileFlag=true; //Global flag instead, see main loop below.
		}else{
			Serial.println("Request ignored");
			Req->send(200, "text/plain", "Request ignored");
		}
	});
	EspServer.begin();
}

void GetBigFile(){
	//Etc. code to load the big file from remote server
}

void loop(){
	if(GetBigFileFlag) GetBigFile(); //Run GetBigFile from here instead of within EspServer.on() 
}

Other possible solutions:
- Breaking the loading of the big file into pieces, though the above fix renders it unnecessary.
- Making uses of Tasks. Not sure about that yet.

Unrelated: looks like FZ in your user thumb?

[fasani]

off-topic of mine: The thumb was created long-time ago when I was starting with Arduino.
Then I built my first Led Matrix that it was called Peggy2 https://www.youtube.com/watch?v=rzfG5oioG-k
No ESP32 back then We just sent data via Serial using USB and Processing.org