The exact frequency of the timer

[ghost07]

I see you are facing very similar problem as I do with frequent SPI reads triggered by interrupt.
In my case, I found out that 4 ms delay was caused by xTaskList() or uxTaskGetSystemState() that calls vTaskSuspendAll() in its implementation.
Check if you don't call one of these functions every 5 sec.
Or maybe ESP_LOGx() could have similar effect, try to disable or prolong prints to console and check if the 40 ms delay occurs less frequently.

Alternatively, if you can modify HW, you can try to use some Paralel-in Serial-out shift register and convert it to some form of SPI communication.

Fortunately, running an ISR per se doesn't require a context switch between tasks.

How do interrupt execution work then? Do interrupts have its own set of CPU registers, so the current task's registers can stay untouched without any context switch?

[MicroController]

Fortunately, running an ISR per se doesn't require a context switch between tasks.

How do interrupt execution work then? Do interrupts have its own set of CPU registers, so the current task's registers can stay untouched without any context switch?

You are right in that interrupts and task switches have much in common; the difference is just that, in a "pure" (non-yielding) ISR, FreeRTOS's vTaskSwitchContext isn't invoked to find another runnable/"ready" task before returning.
So, yes, there is a context switch between a running task and an ISR, but an ISR alone doesn't imply switching from one task to another. Fine hair to split

[mikl604]

The question remains open: how can I temporarily disable all interrupts except for the selected timer? Without FreeRTOS.

[tasha6]

Any luck with this?