Increasing RTOS Tick Rate, >1000Hz

[romba89]

HI all,

I need to get a tick rate above 1000Hz to be able to get sub-millisecond delays in vTaskDelay, vTaskDelayUntil API methods.

Ideally my goal is to establish 2.5ms delays without using timers.

Right now SDK seems to limit it to 1000Hz, even if i change the sdkconfig parameters it defaults to something =<1000Hz

Cheers.

[WiFive]

http://stackoverflow.com/questions/2750 ... -in-a-rtos

[kolban]

Romba89,
Purely for my curiosity, can you describe the design/background for the need for sub millisecond delays?

My thought process goes like this ... within ESP-IDF, we have many tasks (threads) running. The granularity of a context switch through preemption is 1ms. If there are 5 tasks running plus your own ... then if your task is preempted and all the other tasks are ready to run and all have equal priority, then it could be as long as 5ms before you get control again. Even if you delay in your own task for a sub millisecond period, it is absolutely possible for your task to have control taken away from it (in this example) for up to 5ms at a time. As such, ultra low-level timing is likley not to be possible AND take advantage of FreeRTOS processing. If we could understand your design needs then maybe we can comprehend better the need. One can always "disable" FreeRTOS preemption and then delay for a low level period and then re-enable context switching.

[romba89]

Hi kolban,

Thanks for the explanation mate. I've to agree with you on what said above, my goals was to poll an mpu6050 at 400hz. I know i always have the options like MPU interrupts and timer interrupts to try out. I just wanted to see the capability to do in this manner given I don't have any other tasks running at the moment, i'm in the process of developing a separate module for the MPU6050.

In the long run i'm gonna be needing to connect at least 3 MPU9250 + 3 high range IMU together into one ESP32 and sample data at 400hz. Do you think this is a realistic approach with single ESP32 given i'll be using i2c mux to solve the addressing issues.

Cheers.

[ESP_Angus]

There are some problems pushing the RTOS tick rate higher than the default 1000Hz. 1000Hz is already quite high for an RTOS tick rate! As well as increasing the context switch overhead for worker tasks, a lot of FreeRTOS code uses semantics like vTaskDelay(1000 / portTICK_PERIOD_MS) and there are problems if portTICK_PERIOD_MS becomes less than 1!

Timer interrupts are probably the best way to solve this problem. Here's a rough overview of how you might do this:

- Create a semaphore for signalling between the timer interrupt and a task where you'd do that actual I2C heavy lifting.
- Pin the worker task to a core at system maximum priority:

Code: Select all

xTaskCreatePinnedToCore(sample_timer_task, "sample_timer", 4096, NULL, configMAX_PRIORITIES - 1, NULL, 1);

- In the worker task, configure the timer for 400Hz and register a timer interrupt (this ensures it sits on the same CPU as the task, which isn't essential but it increases chance of shorter preemption time.)

(Please treat the following as pseudo-code rather than something you can use as-is.)

Code: Select all

static SemaphoreHandle_t timer_sem;

void sample_timer_task(void *param)
{
   timer_sem = xSemaphoreCreateBinary();
   
   // timer group init and config goes here (timer_group example gives code for doing this)
   
   timer_isr_register(timer_group, timer_idx, timer_isr_handler, NULL, ESP_INTR_FLAG_IRAM, NULL);

  while (1) {
     xSemaphoreTake(timer_sem, portMAX_DELAY);
     
     // sample sensors via i2c here      
     
     // push sensor data to another queue, or send to a socket...
  }
}

void IRAM_ATTR timer_isr_handler(void *param)
{
    static BaseType_t xHigherPriorityTaskWoken = pdFALSE;
    
     TIMERG0.hw_timer[timer_idx].update = 1;
     // any other code required to reset the timer for next timeout event goes here
    
    xSemaphoreGiveFromISR(xSemaphore, &xHigherPriorityTaskWoken);
    if( xHigherPriorityTaskWoken) {
        portYIELD_FROM_ISR(); // this wakes up sample_timer_task immediately
    }
}

The above pattern works like this:

• The sensor sampling task blocks to take the semaphore. While it's blocked, other tasks can run on this core.
• When the timer interrupt happens (configured for 400Hz), it gives the semaphore. This will wake up the highest priority task which is blocking on the semaphore (the sampling task). By calling portYIELD_FROM_ISR this task will be run immediately, no need to wait for a tick to expire.
• Sampling task runs, as it has the highest priority in the system nothing(*) can preempt it from the CPU. It may need to also block waiting for I2C interrupts, but this code will also wake the task immediately via portYIELD_FROM_ISR.
• When done, the sampling task will block on the semaphore again and allow other tasks to run on this CPU (or, if sampling was slow, it will run immediately as the semaphore has already been Given - but you obviously want to avoid that!)

This might seem like a lot of work compared to a non-RTOS approach (which you could possibly also do, if you ran the RTOS on one core and sampled sensors on the other core. Although I don't advise doing that.) The thing which is really nice is that the above pattern for real-time behaviour is all you need to guarantee those events run in real-time. Everywhere else you can do whatever you want without needing to worry about tasks which mess up the timing of your timing-critical section(*).

(*) This isn't entirely true, as interrupts and anything which disables the scheduler will still take this task off-CPU. But these are generally very quick, the only exception I can think of is anything which writes to the SPI flash.

In the long run i'm gonna be needing to connect at least 3 MPU9250 + 3 high range IMU together into one ESP32 and sample data at 400hz. Do you think this is a realistic approach with single ESP32 given i'll be using i2c mux to solve the addressing issues.

I think this is feasible, provided the total sensor sampling + i2c transfer times don't add up close to the 2.5ms timeslice you have for each sampling period (which seems unlikely).

[romba89]

Hi Angus,

Thanks a lot for the detailed explanation. I'm going to try the timer interrupt method and it seems the best way to do it yet.

On a side note, what do you meant by Running RTOS on one core and something else on the other core?

Thanks again.
Cheers.

[ESP_Angus]

On a side note, what do you meant by Running RTOS on one core and something else on the other core?

In the default configuration, the FreeRTOS scheduler runs in SMP mode and schedules tasks on both cores.

It is possible to enable "unicore" mode in the config, so the FreeRTOS scheduler only runs on one core. The other core, by default, does nothing. However if you provide your own implementation of start_cpu1() then you can write code which runs simultaneously on this core. You don't have access to any part of IDF though, you'd have to "pull yourself up by your bootstraps" entirely. Think of it as a technical possibility rather than a supported configuration!

[romba89]

Ah I see. It's quite interesting fact though, to know that possibility exists. Thanks Angus.

[spanky]

I'm trying to do something very similar to the original post. I successfully implemented the method posted by ESP_Angus for reading sensor data at high frequency. The issue is that I'm also trying to update an OLED screen concurrently (although at a slower rate), and am getting frequent timeouts on the I2C read/writes.

I split the OLED (I2C_NUM_0) and sensors (I2C_NUM_1) to different I2C ports, which I though would inherently isolate the read/write operations to their given port. I've also tried to pin the display and sensor tasks to the same, and to different cores, but still get random timeouts on i2c_master_cmd_begin() from each port. Increasing the timeout on i2c_master_cmd_begin() appears to make things worse.

I found this post that suggests changing the I2C timeout, which I did through i2c_set_timeout(), But that didn't seem to help (I'm not sure I understand the timeout values with that function). I also turned off ACK's, which did help, but I dont think that's a proper solution since acks work at high speed when only one port is active.
https://esp32.com/viewtopic.php?f=2&t=1226

Slowing the update rate way down helps, but they work individually at the higher speeds. Does the I2C implementation share some resource between physical ports?

Any suggestions would be appreciated.

[ESP_Angus]

I'm trying to do something very similar to the original post. I successfully implemented the method posted by ESP_Angus for reading sensor data at high frequency. The issue is that I'm also trying to update an OLED screen concurrently (although at a slower rate), and am getting frequent timeouts on the I2C read/writes.

Great! For what it's worth, the posts above pre-date the esp_timer API but I would now recommend using that:
http://esp-idf.readthedocs.io/en/latest ... timer.html

(esp_timer lets you run a callback at a precise time, in a high priority task.)

Regarding the I2C timeouts, it may be better to start a new topic as this seems like it's more about the I2C driver than precise timing. But, to help establish that:

• If you run either one of the two I2C buses only, do you still get timeouts?
• If you run the I2C operations from a normal priority task (ie slower and with less accurate timing), do you still get timeouts?
• What IDF versions are you using?