BBPLL Calibration never finishes in second stage bootloader in RTC Configuration

[tchapin]

We are experiencing issues with our ESP32-S3-WROOM-1 modules, they are hanging in the bootloader and then resetting, then repeat. The ESP32-S3 modules are blocking in the second stage bootloader and being continuously reset by TG0WDT.

We have narrowed the issue down to the rtc_clk_bbpll_configure() function that is being called but never returns. The problem is the line while(!regi2c_ctrl_ll_bbpll_calibration_is_done()); which waits for the bbpll to be calibrated, but is blocking indefinitely.

At first we thought it may have been a faulty ESP32-S3 because when we heated the chip up to ~30C or higher, the ESP32-S3 chip would boot up properly but once we let it cool down, it would go into a bootloop again. This was making us think it was some sort of race condition that the heat was helping slow down just enough to help it work.

We found that the espressif team had made some changes 2 weeks ago (Jan 3, 2024) that pertained to this specific issue (we believe), so we tried pulling those changes, and still have the same error. We tried turning off the 'new' BBPLL Recalibrate setting (CONFIG_ESP_SYSTEM_BBPLL_RECALIB is not set) which was added in the last software version and still not working properly for us.

We manually made a counter inside of rtc_clk_bbpll_configure() function that will force it to pop out of the function. This seems to fix our issue but the question we have is, what are we losing when/if the bbpll is not calibrated or working properly?

Other questions:
1. Why does heat change the outcome of functioning properly?
2. Are we setting configuration settings incorrectly somehow? If so, how would we be able to tell?

[ESP_michael]

Hi tchapin,

Thanks for reporting this issue. Could you provide more details about this issue?
1. On which IDF version you are experiencing this issue?
2. Is this happening on all ESP32-S3-WROOM-1 modules you have? or only one or two of them?

btw, about that patch we have just provided, https://github.com/espressif/esp-idf/co ... 5c6a6a9349 The main part is in the bootloader. If you also rebuild your bootloader, the patch will take effect regardless of the Kconfig option.

The recalibration part is only for OTA case when the bootloader is compiled without the patch.

[tchapin]

Hi,

I can provide more details for you.
1. We are using IDF version 5.1.2 and also have tried master on commit (b3f7e2c8a). Both showed the same issue.
2. As of right now, it is only a few of the modules we have. The thing is though, that we can not confirm if the other modules have the same issue 100%. They don't seem to be susceptible to the temperature change but the ones we found only showed problems after they had been tested numerous and were functionally correct initially.

I have attached an image that shows the patch you are referring to. The regi2c_ctrl_ll_bbpll_calibration_is_done() function never comes back true, so we are just waiting indefinitely for calibration to complete.

[ESP_michael]

Hi tchapin,

We can't reproduce this issue from our side... I have two more questions,

1. What's the normal temperature (which can reproduce the issue) you are testing under? and after heated 30 degree, what's the working temperature.

2. Could you provide the MAC address of your module? Is it modified or not?

[tchapin]

Hi Michael,

1. The normal temperature we are testing under is room temperature which for us is around 20-22C. After heating the board in a curing oven, the board temperature reads no higher than 35C. The testing atmosphere doesn't change so it is tested again after being heated up in a 20-22C ambient temperature.

2. We have had three modules act up in total so far. None of them have been modified. The list of MAC Addresses are below.
Board 1: 7c:df:a1:f248 MAC address
Board 2: 7c:df:a1:f2a4 MAC address
Board 3: 7c:df:a1:f298 MAC address