WiFi task on core 0 halts task on core 1

[benskiskull]

Hello all,
I'm trying to use my ESP32 (WROOM-DA) to control WS2812 and SK6812 led strips with the Neopixel library. I want to be able to control the animations etc via a web server, similar to projects like WLED. I program the ESP with the Arduino IDE.
Right now I essentially create two tasks, one for the webserver, and one for the color calculation and led programming loop. I pin the webserver task to core 0, since if I understand correctly, wifi tasks only run on core 0, or at least need core 0? The led loop I pin to core 1, which, unless the GPIO stuff is also managed by core 0, should work.
The whole thing works fine with both tasks running concurrently, but it fails if I press buttons on the website, therefore sending data to the web server. The led tasks completely halts for a good second before resuming. I thought if i ran both tasks on two different cores, I could avoid that and both tasks run concurrently? Why would it even halt for that long, it's not like there is a lot of data being sent?

Here is my cleaned up code. Note that the web server doesn't actually influence the led loop yet, this is just for testing, but that also means that IMO no multithreading issues like race conditions or whatever should be the problem.
Setup and the two task loops are at the bottom of the code, those are the most relevant, you can ignore all the led helper functions before that.

Code: [Select all] [Expand/Collapse]

1. #include <math.h>
2. #include <stdlib.h>
3. #include <WiFi.h>
4. #include <string.h>
5. #include <Adafruit_NeoPixel.h>
6. #include "driver/gpio.h"
7. #ifdef __AVR__
8. #include <avr/power.h>
9. #endif
10. #define NUMPIXELS_RIGHT 72
11. #define NUMPIXELS_LEFT 73
12. #define NUMPIXELS_TOP 95
13. #define NUMPIXELS 240
14.  
15. #define RIGHT_PIN 4
16. #define TOP_PIN 26
17. #define LEFT_PIN 33
18. Adafruit_NeoPixel rightStrip = Adafruit_NeoPixel(NUMPIXELS_RIGHT, RIGHT_PIN, NEO_RGBW + NEO_KHZ800);
19. Adafruit_NeoPixel topStrip = Adafruit_NeoPixel(NUMPIXELS_TOP, TOP_PIN, NEO_RGB + NEO_KHZ800);
20. Adafruit_NeoPixel leftStrip = Adafruit_NeoPixel(NUMPIXELS_LEFT, LEFT_PIN, NEO_RGBW + NEO_KHZ800);
21. int running;
22. int test;
23.  
24.  
25. const char* ssid = "ObfuscatedRouterName";
26. const char* password = "ObfuscatedPassword";
27.  
28. WiFiServer server(80);
29.  
30. String header;
31.  
32. unsigned long currentTime = millis();
33. unsigned long previousTime = 0;
34. const long timeoutTime = 2000;
35.  
36. TaskHandle_t  webserver ;  
37. TaskHandle_t  ledcontroller ;
38.  
39. std::uint32_t randomColor(){
40.   return leftStrip.Color(rand() % 255, rand() % 255, rand() % 255, 0);
41. }
42.  
43. std::uint32_t col1 = randomColor();
44. std::uint32_t col2 = randomColor();
45. std::uint32_t col3 = randomColor();
46.  
47. int getRed(int x){
48.   int y = constrain((abs((x % 150) - 75) -25 ) / 50.0f * 255, 0, 255);
49.   return y;
50. }
51.  
52. int getGreen(int x){
53.   return constrain((50 - abs((x % 150) - 50)) / 50.0f * 255, 0, 255);
54. }
55.  
56. int getBlue(int x){
57.   return constrain((50 - abs((x % 150) - 100)) / 50.0f * 255, 0, 255);
58. }
59.  
60. void sendToRespectiveStripesW(int n, int r, int g, int b, int w){
61.   if(n < NUMPIXELS_RIGHT){
62.     rightStrip.setPixelColor(n, rightStrip.Color(g, r, b, w));
63.   } else if (n < NUMPIXELS_RIGHT + NUMPIXELS_TOP){
64.     topStrip.setPixelColor(n - NUMPIXELS_RIGHT, topStrip.Color(g, r, b,0));
65.   } else if (n < NUMPIXELS_RIGHT + NUMPIXELS_TOP + NUMPIXELS_LEFT){
66.     leftStrip.setPixelColor(n - NUMPIXELS_RIGHT - NUMPIXELS_TOP, leftStrip.Color(g, r, b, w));
67.   }
68. }
69.  
70. void setRangeSingleColor(int first, int second, int r, int g, int b, int w){
71.   for(int counter = first; counter <= second; counter++){
72.     sendToRespectiveStripesW(counter, r, g, b, w);
73.   }
74. }
75.  
76. void setSingleColor(int r, int g, int b, int w){
77.   for(int n = 0; n < NUMPIXELS; n++){
78.     sendToRespectiveStripesW(n, r, g, b, w);
79.   }
80. }
81.  
82. void setAllBlack(){
83.   for(int n = 0; n < NUMPIXELS; n++){
84.     sendToRespectiveStripesW(n, 0, 0, 0, 0);
85.   }
86. }
87.  
88. void gaussDistribution(int center, int variance, int r, int g, int b, int w){
89.   int cutoff = sqrt(log(255) * 2 * variance);
90.   for(int n = center - cutoff; n <= center + cutoff; n++){
91.     double coeff = exp(-1 * pow(n - center, 2) / (2 * variance));
92.     sendToRespectiveStripesW(n, (int) ( r * coeff), (int) (g * coeff), (int) (b * coeff), (int)(w * coeff));
93.   }
94. }
95.  
96. void startupAnimation(int time){
97.   if(time < 100){
98.     setAllBlack();
99.     setRangeSingleColor(time, time + 20, 255, 255, 255, 255);
100.     setRangeSingleColor(240 - 20 - time, 240 - time, 255, 255, 255, 255);
101.   } else if(time <= 240){
102.     setRangeSingleColor(100, 140, 255, 255, 255, 255);
103.     setRangeSingleColor(200 - time, 100, 255, 255, 255, 255);
104.     setRangeSingleColor(140, time + 40, 255, 255, 255, 255);
105.   }
106. }
107.  
108. void setup() {
109.   Serial.begin(115200);
110.   Serial.println("Starting");
111.  
112.   srand(234231);
113.   running = 0;
114.   test = 0;
115.  
116.   pinMode(RIGHT_PIN, OUTPUT);
117.   pinMode(TOP_PIN, OUTPUT);
118.   pinMode(LEFT_PIN, OUTPUT);
119.   pinMode(21, OUTPUT);
120.  
121.   rightStrip.begin();
122.   topStrip.begin();
123.   leftStrip.begin();
124.   setAllBlack();
125.  
126.   Serial.print("Connecting to ");
127.   Serial.println(ssid);
128.   WiFi.begin(ssid, password);
129.   while (WiFi.status() != WL_CONNECTED) {
130.     delay(500);
131.     Serial.print(".");
132.   }
133.  
134.   Serial.println("");
135.   Serial.println("WiFi connected.");
136.   Serial.println("IP address: ");
137.   Serial.println(WiFi.localIP());
138.   server.begin();
139.  
140.   Serial.println("Task 1 starting");
141.   xTaskCreatePinnedToCore (
142.     loop1,     // Function to implement the task
143.     "loop1",   // Name of the task
144.     30000,      // Stack size in bytes
145.     NULL,      // Task input parameter
146.     4,         // Priority of the task
147.     &ledcontroller,      // Task handle.
148.     1          // Core where the task should run
149.   );
150.   Serial.println("Task 1 started");
151.   xTaskCreatePinnedToCore (
152.     loop2,     // Function to implement the task
153.     "loop2",   // Name of the task
154.     10000,      // Stack size in bytes
155.     NULL,      // Task input parameter
156.     2,         // Priority of the task
157.     &webserver,      // Task handle.
158.     0          // Core where the task should run
159.   );
160.   Serial.println("Task 2 started");
161. }
162.  
163. void loop1(void* pvParameters) {
164.   for(;;){
165.     long time1 = millis();
166.     if(running % 240 == 0){
167.         col1 = randomColor();
168.     }
169.     if((running - 80) % 240 == 0){
170.         col2 = randomColor();
171.     }
172.     if((running - 160) % 240 == 0){
173.         col3 = randomColor();
174.     }
175.     setAllBlack();
176.     gaussDistribution(running % 240, 30, (col1 >> 0) & 0xff, (col1 >> 8) & 0xff, (col1 >> 16) & 0xff, 0);
177.     gaussDistribution((running + 80) % 240 , 30, (col2 >> 0) & 0xff, (col2 >> 8) & 0xff, (col2 >> 16) & 0xff, 0);
178.     gaussDistribution((running + 160) % 240, 30, (col3 >> 0) & 0xff, (col3 >> 8) & 0xff, (col3 >> 16) & 0xff, 0);
179.     running += 1;
180.     test += 1;
181.     if(running == NUMPIXELS){
182.           running = 0;
183.     }
184.     rightStrip.show();
185.     topStrip.show();
186.     leftStrip.show();
187.  
188.     delay(3);
189.   }
190. }
191.  
192. void loop2(void* pvParameters){
193.   for(;;){
194.  
195.   WiFiClient client = server.available();
196.  
197.   if (client) {                            
198.     currentTime = millis();
199.     previousTime = currentTime;
200.     Serial.println("New Client.");          
201.     String currentLine = "";                
202.     while (client.connected() && currentTime - previousTime <= timeoutTime) {  
203.       currentTime = millis();
204.       if (client.available()) {            
205.         char c = client.read();            
206.         Serial.write(c);                    
207.         header += c;
208.         if (c == '\n') {                    
209.           if (currentLine.length() == 0) {
210.             client.println("HTTP/1.1 200 OK");
211.             client.println("Content-type:text/html");
212.             client.println("Connection: close");
213.             client.println();
214.            
215.  
216.             if (header.indexOf("GET /26/on") >= 0) {
217.               Serial.println("GPIO 26 on");
218.             } else if (header.indexOf("GET /26/off") >= 0) {
219.               Serial.println("GPIO 26 off");
220.             } else if (header.indexOf("GET /27/on") >= 0) {
221.               Serial.println("GPIO 27 on");
222.             } else if (header.indexOf("GET /27/off") >= 0) {
223.               Serial.println("GPIO 27 off");
224.             }
225.            
226.  
227.             client.println("<!DOCTYPE html><html>");
228.             client.println("<head><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">");
229.             client.println("<link rel=\"icon\" href=\"data:,\">");
230.            
231.             client.println("<style>html { font-family: Helvetica; display: inline-block; margin: 0px auto; text-align: center;}");
232.             client.println(".button { background-color: #4CAF50; border: none; color: white; padding: 16px 40px;");
233.             client.println("text-decoration: none; font-size: 30px; margin: 2px; cursor: pointer;}");
234.             client.println(".button2 {background-color: #555555;}</style></head>");
235.            
236.             client.println("<body><h1>ESP32 Web Server</h1>");
237.            
238.             if (output26State=="off") {
239.               client.println("<p><a href=\"/26/on\"><button class=\"button\">ON</button></a></p>");
240.             } else {
241.               client.println("<p><a href=\"/26/off\"><button class=\"button button2\">OFF</button></a></p>");
242.             }
243.                  
244.             if (output27State=="off") {
245.               client.println("<p><a href=\"/27/on\"><button class=\"button\">ON</button></a></p>");
246.             } else {
247.               client.println("<p><a href=\"/27/off\"><button class=\"button button2\">OFF</button></a></p>");
248.             }
249.             client.println("</body></html>");
250.            
251.  
252.             client.println();
253.  
254.             break;
255.           } else {
256.             currentLine = "";
257.           }
258.         } else if (c != '\r') {  // if you got anything else but a carriage return character,
259.           currentLine += c;      // add it to the end of the currentLine
260.         }
261.       }
262.    
263.     header = "";
264.    
265.     client.stop();
266.     Serial.println("Client disconnected.");
267.     Serial.println("");
268.   }
269.   delay(30);
270.   }
271. }
272.  
273. void loop(){ }
274.  

GeSHi © Codebox Plus Extension

[benskiskull]

Would shortening the code help? How is it possible to not get any answer in one of the biggest categories on the forum literally centering on the esp32? I would have guessed that I'm not the only one trying multithreading on the esp32, and to me, my application seems incredibly common...how would you otherwise handle having two tasks run concurrently and at least one of them being time critical? Let alone, having one main task and another one for user interface seems like basically anyone runs into this.

[ESP_Sprite]

That, or no one knows. Let's approach it from a different side: how do you conclude the LED task does not run at all for that second or so? Could there be other things that could lead to the symptoms you're seeing?

[boarchuz]

Put a vTaskDelete(NULL); as the last line in your setup(). It won't completely solve your problem but the loopTask on core 1 busily doing nothing in loop() isn't helping your situation.

[benskiskull]

Put a vTaskDelete(NULL); as the last line in your setup().

Your suggestion unfortunately didn't change anything visibly, but still thank you for the command, I wasn't aware of that. I'm not sure, but I think my set priorities for the tasks i started myself should make it so that the loop task gets interrupted immediately by the other tasks if they're ready again.

Let's approach it from a different side: how do you conclude the LED task does not run at all for that second or so? Could there be other things that could lead to the symptoms you're seeing?

I concluded it by seeing the led animation pause for around two seconds and then resume. Any iteration of the for loop in the led task, even just one, would advance the animation visibly, meaning that every time I pressed a button, interacting with the webserver, the led loop didn't get executed for a long time.
I have some news though: I verified that by timing the webserver codeblock in the while loop. As it turns out, the webserver tasks blocks for pretty much exactly 2000 milliseconds, which is the set timeout time and therefore the duration of the while loop that reads HTTP requests and sends back data. It's this block:

Code: [Select all] [Expand/Collapse]

1. while (client.connected() && currentTime - previousTime <= timeoutTime) {  // loop while the client's connected
2.       currentTime = millis();
3.       if (client.available()) {             // if there's bytes to read from the client,
4.         char c = client.read();             // read a byte, then
5.         Serial.write(c);                    // print it out the serial monitor
6.         header += c;
7.         if (c == '\n') {                    // if the byte is a newline character
8.           // if the current line is blank, you got two newline characters in a row.
9.           // that's the end of the client HTTP request, so send a response:
10.           if (currentLine.length() == 0) {
11.             // HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
12.             // and a content-type so the client knows what's coming, then a blank line:
13.             client.println("HTTP/1.1 200 OK");
14.             client.println("Content-type:text/html");
15.             client.println("Connection: close");
16.             client.println();
17.  
18.             // ---------------------SENDING BACK WEBSITE DATA --------------------
19.  
20.             // Break out of the while loop
21.             break;
22.           } else { // if you got a newline, then clear currentLine
23.             currentLine = "";
24.           }
25.         } else if (c != '\r') {  // if you got anything else but a carriage return character,
26.           currentLine += c;      // add it to the end of the currentLine
27.         }
28.       }
29.     }
30.     // Clear the header variable
31.     header = "";
32.     // Close the connection
33.     client.stop();
34. }

GeSHi © Codebox Plus Extension

This means that forever reason, the while loop containing web client reads and prints completely blocks the entire esp32, even though the task is pinned to core 0. Digging a bit further, it seems like I'm not the only one having somewhat similar issues (https://www.reddit.com/r/FastLED/commen ... on_core_0/, https://www.reddit.com/r/esp32/comments ... _blocking/, especially the latest comment in the second link).
One question: Do you guys know if I can verify if the code is multithreading correctly? Would the code fail if running code on both cores for whatever reason is disabled or whatever, or would it just silently assign both tasks to a core (propably core 1)?
As for me, I think I'm going to try using WebServer.h, or even ESPAsyncWebServer, it seems like with their callback-based architecture, the blocking could maybe be avoided, I welcome comments on that. Thank you all for your replies so far already.

[ESP_Sprite]

It's odd... not sure if you're using the correct term, but Arduino for ESP32 is built on ESP-IDF which uses FreeRTOS under the hood, and 'blocking' in that context means that when a task is waiting for ('blocking on') something, FreeRTOS de-schedules it and allows other tasks to run until whatever the task is waiting for actually happens. So even if the WiFi task is waiting for a response or something, I'd still expect your LED task to run.

[boarchuz]

You aren't sending a content length header, so I suspect what's happening is that the client is keeping the socket open and the while loop is being entered immediately again until it times out and reaches the client.stop. One of the big downsides with Arduino is that there's so much abstraction it's a complete mystery what any of these "client" calls are doing without digging through the source code, so without doing so I'd guess it's repeatedly unblocking a high priority task on core 1 (quick search shows this possibly? https://github.com/espressif/arduino-es ... ts.cpp#L81).

Can you include your serial output here? Do you see multiple "New client" messages when you click the button?

When exactly does the request end on the client side? If you open developer tools in your browser and click the button, does the request appear to complete almost immediately or does it wait for about 2 seconds?

Can you try another 'client.stop()' immediately before 'break' to close it on the ESP32 side?

Does changing the LED task priority or unpinning have any effect?

[benskiskull]

Alright, I have an update. I managed to minimize the problem to some extent. The code now breaks out of the while loop if the client doesn't send any new characters (of a HTTP request). This leads to the code not blocking for the whole timeout period, which is not expected behaviour, but just for the time spent on reading the HTTP request and sending a HTTP response, so far so good. This now leads to the LEDs only halting for a fraction of second, which is better, but the problem still remains.

'blocking' in that context means that when a task is waiting for ('blocking on') something, FreeRTOS de-schedules it and allows other tasks to run until whatever the task is waiting for actually happens. So even if the WiFi task is waiting for a response or something, I'd still expect your LED task to run.

Here is the thing. So far I thought that descheduling only occurs when a task executes something like vTaskDelay() (or delay(), which to my understanding is mapped to vTaskDelay on the esp32), which means the webserver loop would be able to be descheduled once in every for loop iteration (on the delay after the while loop). But that shouldn't matter right? The whole reason I put the led loop on core 1 and the webserver loop on core 0 is so that the led task doesn't have to wait for the webserver task to be descheduled, but that it, well, runs in parallel. The problem of the blocking between the cores remains for me.

You aren't sending a content length header, so I suspect what's happening is that the client is keeping the socket open and the while loop is being entered immediately again until it times out and reaches the client.stop. One of the big downsides with Arduino is that there's so much abstraction it's a complete mystery what any of these "client" calls are doing without digging through the source code, so without doing so I'd guess it's repeatedly unblocking a high priority task on core 1 (quick search shows this possibly? https://github.com/espressif/arduino-es ... ts.cpp#L81).

Can you include your serial output here? Do you see multiple "New client" messages when you click the button?

When exactly does the request end on the client side? If you open developer tools in your browser and click the button, does the request appear to complete almost immediately or does it wait for about 2 seconds?

Can you try another 'client.stop()' immediately before 'break' to close it on the ESP32 side?

See the beginning of this answer, I now solved the unnecessary long blocking, and the whole loop behaves as it should now (client connects, sends HTTP request, gets response, the while loop breaks, the connection is closed).
As for your github link, I'm wondering the same. I don't think it's that code line since the created task seems to run on the event processing core, which is set to core 0 in the Arduino IDE (if I can trust it). But the only possible explanation for this blocking seems to me too that the webserver creates internal background tasks on core 1 that block it while web communication is happening, the only problem is that I can't find anything on that, I found nothing online as to which background tasks are created by the webserver.

[lbernstone]

Your task is running your code, not the whole tcpip stack. WiFi is running on the events core, which defaults to core1.

Screenshot from 2024-09-21 09-47-50.png (39.52 KiB) Viewed 2560 times

[benskiskull]

Your task is running your code, not the whole tcpip stack. WiFi is running on the events core, which defaults to core1.

That's what I thought, therefore I set it to core 0, as I said in the end of my last reply. Maybe that doesn't work, but I don't see any reason why.

arduinoScreenshot.png (19.57 KiB) Viewed 2544 times