Activate internal 8M/256 clock in light sleep

[beninator]

I am working on a compact watch built with the TTGO ESP32 T-Display.

My problem is to keep accurate time while the ESP32 is in light sleep mode.
In active mode the time is accurate enough but as soon as it sleeps the time drift is bad.

On the internet I found that the esp is switching to the inaccurate internal 150khz RTC while in sleep mode.

So my question is:
Can I somehow tell the ESP to use the 8M/256 RTC in the Arduino IDE while it is in sleep mode? (or whatever RTC it uses in active mode)

I would accept the slightly higher current but I dont want to use a external RTC-module because the watch should be as compact as possible.

Thanks for your answers!