RTC clock accuracy / clock source

[RetroarchYoshi]

Hi guys,
I hope this is the right place to ask this. I have built a project around an e paper board which needs to display the current time, and due to size constrains, I want to add as little external components as possible. That's why I am using the internal RTC of the esp32. I am updating it via ntp wherever I have a wifi connection, but part of the project is that it might be without WiFi connection for a few days, in between which the esp32 has to go into deep sleep.

The problem: RTC is pretty inaccurate during deep sleep, as talked about a lot online. Basically after a day of periodic deep sleep the time is often off by 5-10 minutes.

Possible solution: Espressifs website (https://docs.espressif.com/projects/esp ... _time.html) states that one can switch the rtc clock source during deep sleep to a "Internal 8.5MHz oscillator, divided by 256" to get much better accuracy. However, I am not able to change this precompiled (?) parameter while using the esp32 core for the arduino ide.
1) I have heard that the official espressif-idf can change this easily. But I do not want to rewrite all my code (which is quite long) and I am not sure how to get the epaper working there
2) There's a tool to rebuild the precompiled binarys running inside the arduino ide: https://github.com/espressif/esp32-arduino-lib-builder However, I did not manage to find any documentation saying how to change parameters before compiling

Any ideas?