CPU Clock source and frequency

[Aircraft]

Hello!!!
I'm trying to reduce esp32 cpu frequency in order to lower power consumption. If PLL Clock Source from internal oscillator is used I can reduce CPU frequency till 80 MHz, with freeRTOS support and project configuration found in menuconfig options. However, I want to use the 40 MHz On-Chip Crystal as my clock source (I'm using ESP32-WROOM32 chip) to reduce CPU frequency even more than I've already have. I tried soc/rtc.h API functions (ex: rtc_clk_cpu_freq_set_config), but the API itself recommends not to use these functions due to other clocks and timers that should be reconfigured according to the new CPU frequency and this issue is not implemented by the API, so the correct working of freeRTOS programs using these functions is not guaranteed. I ignored the recommendation and used the API, and suddenly I started getting incorrect data from serial communication which I see in my serial monitor. Can someone from Espressifs or someone who has already dealt with this comment on this issue as this is not deeply treated in esp-idf documentation? Is there any other way to reduce CPU frequency?
Thanks in advance
Regards