Nevermind, I found the solution.
It was to use before initializing INPUT_PULLUP pin.
But thanks for giving me hope that it's solvable and to push through.

Hello,

I'm having the same issue with my ESP32 C3.
I'm using deep sleep and after which I cannot set INPUT_PULLUP. IT doesn't do anything.

Could you please elaborate on how did you solve the issue?
I'm using Arduino IDE.

Thx in advance