Isolating Application Code to APP Core

[berlinetta]

Hello All,

I am new to the ESP32 and attempting to learn as much as I can as fast as I can. I am attempting to architect a design which will utilize the PRO core for supporting the WiFi and BLE stacks with the FreeRTOS while utilizing the APP core to execute my application-specific code (bare metal). The idea behind this is the desire to allow the PRO core exclusive control of the communications stacks without interruption by the application-specific code. A secondary goal was to reduce the interrupt latency when handling application-specific communications between the ESP32 and the host controller.

Documentation informs me that the developer should create an "app_main" function which will be called by the startup routine provided with the ESP-IDF. In essence, the ESP-IDF provides all the code necessary to startup the device and initialize things before passing control to the developers code (app_main). I found the cpu_start.c module within the ESP-IDF source code which is responsible for this initialization process. From what I can tell, the startup process executes as follows:
1) I presume the call_start_cpu0() function is executed first, while the cpu1 core is held in reset.
2) After initializing memory, it appears the APP CPU (cpu1) is brought out of reset and configured to execute the call_start_cpu1() function. The PRO CPU (cpu0) then waits for the APP_CPU (cpu1) to report it has started.
3a) The PRO_CPU (cpu0) will execute the start_cpu0() function - aliased as start_cpu0_default() - where initialization of cpu0 continues before creating the "main_task" (pinned to core 0) and initiating the task scheduler.
3b) The APP_CPU (cpu1) will execute the start_cpu1() function - aliased as start_cpu1_default() - where the code waits until the task scheduler on cpu0 is running before starting the task scheduler on cpu1.
4) The PRO_CPU will execute the created main_task, which waits until the task scheduler is running on cpu1 before calling app_main().

Since the app_main() function is called from the main_task() routine operating on cpu0, the developer's code will start on cpu0 by default. Documentation and other conversations in the forum mention that the developer's code can create its own tasks and pin them to cpu1 if desired.

I can see how it is necessary for the developer code to provide an app_main() routine which will configure the desired features/characteristics of the ESP-IDF that will be executed on cpu0. Based on the construction of the ESP-IDF code, it is clear the intention is to have an author create tasks which FreeRTOS is capable of executing on either core - as it sees fit.

In order to implement my intended architecture where the application-specific code is executed solely on cpu1, I believe I would have to modify the start_cpu1_default() routine to call an equivalent "app_main" routine which would be responsible for setting up the cpu1 code. Am I interpreting this correctly?

Would I also need to create FreeRTOS tasks pinned to cpu1 in order to interact with the communications stacks running on cpu0?

Thanks in advance!
Mark

[mikemoy]

If i didn't read your post to fast your looking for something like this.
If you are not aware yet, there is a great source here.
https://github.com/espressif/esp-idf/tr ... r/examples

Code: Select all

#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/gpio.h"
#include "sdkconfig.h"

#define PRO_CPU	0
#define APP_CPU	1

xTaskHandle TaskHandle_Task1;
xTaskHandle TaskHandle_Task2;


void task1(void *pvParameter)
{
	for(;;)
	{
		printf("Task1 running...\r\n");
		vTaskDelay(500 / portTICK_PERIOD_MS);
	}
}
void task2(void *pvParameter)
{
	for (;;)
	{
		printf("Task2 running...\r\n");
		vTaskDelay(500 / portTICK_PERIOD_MS);
	}
}

void app_main()
{
	xTaskCreatePinnedToCore(task1, "Task1", 2048, NULL, 10, &TaskHandle_Task1, APP_CPU);
	xTaskCreatePinnedToCore(task2, "Task2", 2048, NULL, 10, &TaskHandle_Task1, PRO_CPU);
}

[berlinetta]

If i didn't read your post to fast your looking for something like this.

I believe you read my post too quickly. Your reference is an example of how to pin tasks to cores. I am looking to restrict the FreeRTOS and stack operation to the PRO core (cpu0) while giving the application-specific code free reign over the APP core (cpu1). As of now, the ESP-IDF code structure controls all interrupts (including cpu1 interrupts) from the cpu0 core. Latency is an issue, and I would prefer to keep my application code running on a separate core so as not to impede any operations of the communications stacks.

Thanks,
Mark

[WiFive]

viewtopic.php?f=2&t=10006
viewtopic.php?f=2&t=10901

[berlinetta]

Hello WiFive,

Thanks for the links, but I have already perused those posts. Both of those authors are referencing a desire to implement designs which seek to disable interrupts on the APP core. I am seeking to take control of the interrupts on the APP core within bare metal application code and segregate the FreeRTOS and the WiFi and BLE stacks to the PRO core.

After reviewing all posts and documentation that I can find regarding my original topic, I realize that the solutions derived with the provided ESP-IDF suite are highly reliant upon FreeRTOS controlling everything which occurs on each of the cores. I was informed that it would be possible to segregate bare metal application code to the APP processor, but the more I dig into exactly what is required to do so, I fear this is a monumental task - and may be impossible without a complete knowledge of the underpinnings of this architecture which are not adequately documented.

At this point in my journey, my biggest concern is in relation to the caching of instructions from the SPI flash device. Since the ESP32 does not contain any of its own flash memory, all instructions must be fetched from the external flash memory device - regardless of what core they are running on. So if I were to manage to pull off the segregation of my code to the APP core, I fear I could still run into issues with the instruction cache operation. I am not certain how that mechanism works and if I could cause problems with bare metal code execution on the APP core.

My second concern is the interrupt management. The research I have done thus far indicates that each core contains its own interrupt controller, and the interrupts from peripherals are mapped into that controller through the interrupt matrix. The default ESP32 code configuration creates a "master" interrupt handler mechanism which services any interrupt occurring on either core. The user's code may register an interrupt handler, but it is not tied directly to the vector table without the user creating their own vector table for the core. I am not certain what the pitfalls are for creating this custom vector table, because I do not understand which APP core interrupts (if any) must be accessible to the PRO core in order to properly handle things such as memory management and core-to-core communications.

The third challenge for me is properly managing the communication between my bare metal application code and the WiFi and BLE stacks. If I architect a scheme which precludes the use of tasks on the APP core, I presume the bare metal code would not be capable of using the provided API calls which were developed for communicating with the WiFi and BLE stacks through tasks. It would be a daunting task if I am required to architect a custom communications interface between the cores for such purposes.

So perhaps the best approach for me would be the implementation of a FreeRTOS task-based design. However, I would still like to address the large latency encountered when using the SPI peripheral to communicate with our host controller. Is there any way to overcome the latency involved with the default interrupt handler mechanism when it comes to a specific peripheral interrupt - such as when implementing a SPI slave interface? Is it possible to create a high-priority task which waits on the reception of data from the SPI peripheral and can immediately queue up the appropriate responses for data requests without incurring the delays through queues?

Thanks in advance!
Mark

[WiFive]

It's all under the same umbrella of wanting to use the app core independently and has the same basic implementation requirements. Some of the things you said are not correct. You shouldn't need to use the flash cache for bare metal code, it should fit in the 32k iram normally used for the cache and be static. The things to worry about are internal bus mediation requirements, memory separation, and cross core communication. For your particular case optimizing the driver and using a high priority task is probably better.

[berlinetta]

Hi WiFive,

Thanks for the quick response!

I am not surprised some of my presumptions were incorrect, as I am still learning this architecture.

Good point regarding a bare metal code implementation... provided the code size falls within the 32KB limit, the cache should not need to be updated.

For your particular case optimizing the driver and using a high priority task is probably better.

Yes, I have been leaning toward this form of an implementation. For obvious reasons, I would prefer to keep any modifications of ESP-IDF code modules to an absolute minimum - or avoid modifying them entirely. I received a suggestion from ESP_Sprite in a previous post to consider utilizing the SDIO Slave peripheral rather than SPI. Do you think that would be a better solution?

Best Regards,
Mark

[berlinetta]

I have discovered yet another caveat...

After taking a day or so to investigate the plausibility of using the SDIO Slave peripheral for communications with our host processor, I have discovered that the pin assignments are not conducive for development. The first module is not usable because the pin assignments conflict with the main SPI port which is utilized for accessing external memory. The second module is not feasible because its pin assignments conflict with the JTAG port pins, which would prevent the ability to operate the debugger during code development.

So I am back to investigating potential modifications to the SPI slave driver to see if I can create a useful implementation.

Best Regards,
Mark

[AkhileshBelore]

Hello,
Were you able to achieve what you desired.