single core freertos and tasks

[jakehuang]

Hi guys,

I am in the middle of a project that does real time network communicate and at the same time the project also runs a timer interrupt at 30kHz. As you can see it is a very noisy ISR environment and latency is very important to me.

I am trying to understand the behavior of all the tasks and freertos scheduler here. The questions I have are.

1. It seems by default freertos only runs in the first core. Does that mean that the second core (APP_CPU) is sitting idle doing nothing?
2. How many tasks are actually running in a simple app where there is only the app_main task (which does only ESP_LOGD() call and no other components are enabled) and no other tasks are created?
3. What if, wifi , bluetooth and ethernet components are enabled?
4. I noticed that the maximum length an UDP packet can reach is around 800 bytes? I've used much better packet size than that in desktop developments. Is that a inherent limitation of lwip?

Thanks a bunch. This will help me better use esp32 to achieve minimal latency and maximum throughput.

-jake

[jakehuang]

Nobody have answers for these questions or the answers are already scattered in the forum...?

[ESP_igrr]

1. By default, dual core mode is enabled. However if you choose"run FreeRTOS on single core only", then APP_CPU will be clock gated and only PRO_CPU will be used by FreeRTOS.

2. Empty app: IDLE task (one per CPU), FreeRTOS timer task, IPC task (one per CPU), esp_timer task, main task.

3. With WiFi: + wifi tasks (2), tcp-ip task, event loop task
With BT: + BT controller task, BT profile task

You can get the task list using FreeRTOS functions, btw.

4. I don't think there is a limit of 800 bytes in LwIP. If you see issues sending larger packets, please report the issue on GitHub along with the code to reproduce. Thanks.

[jakehuang]

Thanks ESP_igrr.

I found that in the default template app. "make menuconfig" will show up with unicore freertos already selected. So by default freertos will only run in single core mode.

In my application, I have a timer ISR which will run at 30kHz. This means the core is interrupted 30,000 times per second just for that one ISR. That's a huge ISR overhead and will be good to assign one spare core to just do that and have freertos to run on the other core, taking care of wifi, ethernet and other freertos tasks.

I looked at the source code cpu_start.c. Looks like if you just enable two cores, both cores will be running freertos and there is no way to just leave the spare core to run an ISR and not run freertos.


I've updated my toolchain and idf to the lateset versions and will be doing some testing to get more sense of what's going under the hook.

I think this understanding is paramount to helping esp32 achieve widespread adaption in commercial products. In reality some sort of FreeRTOS+TraceTM should be supported in the near future otherwise it's difficult to understand system behavior and this has crucial consequence for real time applications, which a device with so much connectivity options will most likely be used for.

I will report back what I found here to share with others.
Thanks a lot!

Best
-jake

[jakehuang]

BTW, I found that example apps have both cores enabled by default.
https://github.com/espressif/esp-idf-template only uses one core by default.

[ESP_igrr]

Well, from IDF perspective the default is to have dual core mode enabled. But the template app has single core mode config option set, which is why it runs in single core mode. If you start with a project which doesn't have an sdkconfig or sdkconfig.defaults, you will get dual core mode by default.

With regards to startup procedure, you may find this part of documentation useful:
http://esp-idf.readthedocs.io/en/latest ... on-startup

Since you have mentioned FreeRTOS+Trace, I'm also going to point you to our docs about SystemView support: http://esp-idf.readthedocs.io/en/latest ... systemview

Regarding handling of interrupts on one core: this is possible, using esp_intr_alloc API. You can install the interrupt handler on the core of your choice (e.g. on APP CPU) and do the rest of the tasks on the PRO CPU.
http://esp-idf.readthedocs.io/en/latest ... alloc.html

[jakehuang]

Thanks for the fast reply.

I checked the source code for start_cpu0 and start_cpu1, while it seems it's viable to just roll my own start_cpu1 but the fact that esp32's version of freertos has been heavily modded and the OS behaves differently depending on whether or not both cores are enabled , makes it a very risky strategy.

I should just use esp_intr_alloc to put the timer interrupt on APP_CPU. The idea is to keep the noisy ISR environment to APP_CPU while PRO_CPU does all other tasks (mostly getting data from network).

Is there a way to check if how much of the system tasks are run on each core? For example, I would like all ethernet, wifi and system event tasks to be running on PRO_CPU only.

The only I can think of is using xPortGetCoreID() in related calls in idf such as wifi or system event handlers.

BTW, Using the example blink app, even if I have both cores enabled, xPortGetCoreID() keeps giving me "0", meaning its always run in the first core. Only after using xTaskCreatePinnedToCore to pin it to second core did it start running on core 1. It makes me think that the current freertos is kind of "sticky" in terms of core affinity for tasks.

Thanks for all the links. I've read the system startup procedure and it is very help indeed in understanding the boot process. However it falls short on how to actually custom roll our own start_cpu1 and use it in non-freertos way.

SystemView needs JTAG, and getting esp32 to work with JTAG is painful at this point to say the least. I used my JLink but the documentation is so lacking I have no success getting it running. My esp32-wrover devkit v3's micro USB connector broke off from the board after a couple of plugging/unplugging. It is something that expressif should really do something about...

Still, esp32 is a wonderful platform and thanks for all the hard work you guys have done. This tiny devices aims to do so mush with so little it's amazing how far you guys have come in the span of a year.

[ESP_igrr]

Is there a way to check if how much of the system tasks are run on each core? For example, I would like all ethernet, wifi and system event tasks to be running on PRO_CPU only.

You can also use FreeRTOS uxTaskGetSystemState API to get information about tasks.

BTW, Using the example blink app, even if I have both cores enabled, xPortGetCoreID() keeps giving me "0", meaning its always run in the first core.

I think that given the fact that there is only one running task, this is reasonable behavior.

However it falls short on how to actually custom roll our own start_cpu1 and use it in non-freertos way.

Rolling cusom code for CPU1 is, in fact, fairly hard to do for any non-trivial application. If your application is going to enable/disable cache (and that happens e.g. when SPI flash operations are performed, or Flash MMU is reconfigured), then there has to be some mechanism which would allow code running on one CPU to tell the code running on the other CPU to avoid executing cached instructions while cache is disabled. In IDF, this mechanism uses FreeRTOS primitives for cross-core communication.
The other reason is that the code running on CPU1 will need to communicate with the code running on CPU0 somehow. That usually means using queues, semaphores, and so on. Without these primitives only the simplest interactions are possible.
So we think that running FreeRTOS on CPU1 is usually a more useful option.

Re JTAG: have you seen JTAG debugging tutorial, http://esp-idf.readthedocs.io/en/latest ... index.html? If yes, would you mind elaborating a bit which parts you found lacking, so that we can improve the instructions?

Re USB socket on WROVER: next version of WROVER (v4) comes with a micro-USB connector with through-hole side pins (we have popped enough of these connectors from dev boards ourselves as well...)