ESP32-WROOM Occasional reset, and then locked in bootloader WDT
Posted: Tue Oct 22, 2019 3:16 pm
We have a device based on the ESP32-WROOM which works most of the time. The ESP32 is reading an I2C accelerometer constantly, once every 20 ms. Some of our products can last forever running code - but every so often we see a device get a reset in the ESP I2C driver code. This in of itself is interesting, but what is really bad is we see the device get into a continuous WDT reset in the bootloader when this happens. It takes a manual reset (or pulling of battery) to get it out of this state.
So, two questions. One - has anybody seen an I2C exception like below. And, more interesting - how and why could the device get into the continuous reset state shown below when this happens? NOTE!! We have also tried to "force" the continuous reset situation to debug by setting cmd to 1 in between the two lines below. When we do that, the device does indeed reset (panic) as expected - but boots normally in that case.
The I2C fault that causes the crash in the first place is:
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
I2C_ERR_CHECK(i2c_master_start(cmd));
Somehow cmd (the I2C handle) is 1 (obviously an invalid address) when the problem happens. i2c_cmd_link_create(), which is a core ESP library function, is just calling a calloc - so how could it ever return a 1?
Here's the log dump:
[10/10/19 12:09:28:631] Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception
was unhandled.
[10/10/19 12:09:28:940] Core 0 register dump:
[10/10/19 12:09:28:942] PC : 0x400eccff PS : 0x00060630 A0 : 0x800ed990 A
1 : 0x3ffd39d0
[10/10/19 12:09:28:957] A2 : 0x00000001 A3 : 0x3ffd3a0c A4 : 0x00060220 A
5 : 0x00000000
[10/10/19 12:09:28:957] A6 : 0x00060420 A7 : 0x00000000 A8 : 0x3ffd3bc0 A
9 : 0x000000ff
[10/10/19 12:09:28:957] A10 : 0x61523ff5 A11 : 0x00000001 A12 : 0x7e000000 A
13 : 0x00000001
[10/10/19 12:09:28:957] A14 : 0x00060223 A15 : 0x00000001 SAR : 0x00000007 E
XCCAUSE: 0x0000001c
[10/10/19 12:09:28:957] EXCVADDR: 0x00000001 LBEG : 0x400e9e14 LEND : 0x400e9e33 L
COUNT : 0x00000000
[10/10/19 12:09:28:957]
[10/10/19 12:09:28:957] Backtrace: 0x400eccff:0x3ffd39d0 0x400ed98d:0x3ffd39f0 0x400d4efb:0
x3ffd3a40 0x400d501f:0x3ffd3a70 0x400d71ef:0x3ffd3ae0 0x400d2092:0x3ffd3b30
[10/10/19 12:09:28:957]
[10/10/19 12:09:28:957] Rebooting...
[10/10/19 12:09:28:957] ets Jun 8 2016 00:22:57
[10/10/19 12:09:28:957]
[10/10/19 12:09:28:957] rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:28:957] configsip: 0, SPIWP:0xee
[10/10/19 12:09:28:957] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:28:957] mode:DIO, clock div:2
[10/10/19 12:09:28:957] load:0x3fff0018,len:4
[10/10/19 12:09:28:957] load:0x3fff001c,len:5728
[10/10/19 12:09:28:957] ho 0 tail 12 room 4
[10/10/19 12:09:28:957] load:0x40078000,len:7684
[10/10/19 12:09:28:957] load:0x40080000,len:7392
[10/10/19 12:09:28:957] entry 0x4008039c
[10/10/19 12:09:29:623] ets Jun 8 2016 00:22:57
[10/10/19 12:09:29:625]
[10/10/19 12:09:29:626] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:29:643] configsip: 0, SPIWP:0xee
[10/10/19 12:09:29:645] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:29:658] mode:DIO, clock div:2
[10/10/19 12:09:29:658] load:0x3fff0018,len:4
[10/10/19 12:09:29:659] load:0x3fff001c,len:5728
[10/10/19 12:09:29:660] ho 0 tail 12 room 4
[10/10/19 12:09:29:660] load:0x40078000,len:7684
[10/10/19 12:09:29:661] load:0x40080000,len:7392
[10/10/19 12:09:29:662] entry 0x4008039c
[10/10/19 12:09:29:981] ets Jun 8 2016 00:22:57
[10/10/19 12:09:29:983]
[10/10/19 12:09:29:984] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:29:993] configsip: 0, SPIWP:0xee
[10/10/19 12:09:29:994] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:30:003] mode:DIO, clock div:2
[10/10/19 12:09:30:004] load:0x3fff0018,len:4
[10/10/19 12:09:30:004] load:0x3fff001c,len:5728
[10/10/19 12:09:30:011] ho 0 tail 12 room 4
[10/10/19 12:09:30:012] load:0x40078000,len:7684
[10/10/19 12:09:30:012] load:0x40080000,len:7392
[10/10/19 12:09:30:013] entry 0x4008039c
[10/10/19 12:09:30:343] ets Jun 8 2016 00:22:57
[10/10/19 12:09:30:345]
[10/10/19 12:09:30:346] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:30:361] configsip: 0, SPIWP:0xee
[10/10/19 12:09:30:363] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:30:376] mode:DIO, clock div:2
[10/10/19 12:09:30:377] load:0x3fff0018,len:4
[10/10/19 12:09:30:377] load:0x3fff001c,len:5728
[10/10/19 12:09:30:378] ho 0 tail 12 room 4
[10/10/19 12:09:30:378] load:0x40078000,len:7684
[10/10/19 12:09:30:379] load:0x40080000,len:7392
[10/10/19 12:09:30:379] entry 0x4008039c
[10/10/19 12:09:30:699] ets Jun 8 2016 00:22:57
[10/10/19 12:09:30:700]
[10/10/19 12:09:30:700] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:30:707] configsip: 0, SPIWP:0xee
[10/10/19 12:09:30:707] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:30:716] mode:DIO, clock div:2
[10/10/19 12:09:30:722] load:0x3fff0018,len:4
[10/10/19 12:09:30:723] load:0x3fff001c,len:5728
[10/10/19 12:09:30:731] ho 0 tail 12 room 4
[10/10/19 12:09:30:732] load:0x40078000,len:7684
[10/10/19 12:09:30:732] load:0x40080000,len:7392
[10/10/19 12:09:30:733] entry 0x4008039c
[10/10/19 12:09:31:059] ets Jun 8 2016 00:22:57
[10/10/19 12:09:31:062]
[10/10/19 12:09:31:064] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:31:072] configsip: 0, SPIWP:0xee
[10/10/19 12:09:31:073] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:31:086] mode:DIO, clock div:2
[10/10/19 12:09:31:087] load:0x3fff0018,len:4
[10/10/19 12:09:31:087] load:0x3fff001c,len:5728
[10/10/19 12:09:31:094] ho 0 tail 12 room 4
[10/10/19 12:09:31:095] load:0x40078000,len:7684
[10/10/19 12:09:31:096] load:0x40080000,len:7392
[10/10/19 12:09:31:096] entry 0x4008039c
[10/10/19 12:09:31:418] ets Jun 8 2016 00:22:57
[10/10/19 12:09:31:420]
[10/10/19 12:09:31:421] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:31:436] configsip: 0, SPIWP:0xee
[10/10/19 12:09:31:438] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:31:453] mode:DIO, clock div:2
[10/10/19 12:09:31:454] load:0x3fff0018,len:4
[10/10/19 12:09:31:455] load:0x3fff001c,len:5728
[10/10/19 12:09:31:456] ho 0 tail 12 room 4
[10/10/19 12:09:31:457] load:0x40078000,len:7684
[10/10/19 12:09:31:458] load:0x40080000,len:7392
[10/10/19 12:09:31:468] entry 0x4008039c
[10/10/19 12:09:31:777] ets Jun 8 2016 00:22:57
So, two questions. One - has anybody seen an I2C exception like below. And, more interesting - how and why could the device get into the continuous reset state shown below when this happens? NOTE!! We have also tried to "force" the continuous reset situation to debug by setting cmd to 1 in between the two lines below. When we do that, the device does indeed reset (panic) as expected - but boots normally in that case.
The I2C fault that causes the crash in the first place is:
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
I2C_ERR_CHECK(i2c_master_start(cmd));
Somehow cmd (the I2C handle) is 1 (obviously an invalid address) when the problem happens. i2c_cmd_link_create(), which is a core ESP library function, is just calling a calloc - so how could it ever return a 1?
Here's the log dump:
[10/10/19 12:09:28:631] Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception
was unhandled.
[10/10/19 12:09:28:940] Core 0 register dump:
[10/10/19 12:09:28:942] PC : 0x400eccff PS : 0x00060630 A0 : 0x800ed990 A
1 : 0x3ffd39d0
[10/10/19 12:09:28:957] A2 : 0x00000001 A3 : 0x3ffd3a0c A4 : 0x00060220 A
5 : 0x00000000
[10/10/19 12:09:28:957] A6 : 0x00060420 A7 : 0x00000000 A8 : 0x3ffd3bc0 A
9 : 0x000000ff
[10/10/19 12:09:28:957] A10 : 0x61523ff5 A11 : 0x00000001 A12 : 0x7e000000 A
13 : 0x00000001
[10/10/19 12:09:28:957] A14 : 0x00060223 A15 : 0x00000001 SAR : 0x00000007 E
XCCAUSE: 0x0000001c
[10/10/19 12:09:28:957] EXCVADDR: 0x00000001 LBEG : 0x400e9e14 LEND : 0x400e9e33 L
COUNT : 0x00000000
[10/10/19 12:09:28:957]
[10/10/19 12:09:28:957] Backtrace: 0x400eccff:0x3ffd39d0 0x400ed98d:0x3ffd39f0 0x400d4efb:0
x3ffd3a40 0x400d501f:0x3ffd3a70 0x400d71ef:0x3ffd3ae0 0x400d2092:0x3ffd3b30
[10/10/19 12:09:28:957]
[10/10/19 12:09:28:957] Rebooting...
[10/10/19 12:09:28:957] ets Jun 8 2016 00:22:57
[10/10/19 12:09:28:957]
[10/10/19 12:09:28:957] rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:28:957] configsip: 0, SPIWP:0xee
[10/10/19 12:09:28:957] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:28:957] mode:DIO, clock div:2
[10/10/19 12:09:28:957] load:0x3fff0018,len:4
[10/10/19 12:09:28:957] load:0x3fff001c,len:5728
[10/10/19 12:09:28:957] ho 0 tail 12 room 4
[10/10/19 12:09:28:957] load:0x40078000,len:7684
[10/10/19 12:09:28:957] load:0x40080000,len:7392
[10/10/19 12:09:28:957] entry 0x4008039c
[10/10/19 12:09:29:623] ets Jun 8 2016 00:22:57
[10/10/19 12:09:29:625]
[10/10/19 12:09:29:626] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:29:643] configsip: 0, SPIWP:0xee
[10/10/19 12:09:29:645] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:29:658] mode:DIO, clock div:2
[10/10/19 12:09:29:658] load:0x3fff0018,len:4
[10/10/19 12:09:29:659] load:0x3fff001c,len:5728
[10/10/19 12:09:29:660] ho 0 tail 12 room 4
[10/10/19 12:09:29:660] load:0x40078000,len:7684
[10/10/19 12:09:29:661] load:0x40080000,len:7392
[10/10/19 12:09:29:662] entry 0x4008039c
[10/10/19 12:09:29:981] ets Jun 8 2016 00:22:57
[10/10/19 12:09:29:983]
[10/10/19 12:09:29:984] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:29:993] configsip: 0, SPIWP:0xee
[10/10/19 12:09:29:994] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:30:003] mode:DIO, clock div:2
[10/10/19 12:09:30:004] load:0x3fff0018,len:4
[10/10/19 12:09:30:004] load:0x3fff001c,len:5728
[10/10/19 12:09:30:011] ho 0 tail 12 room 4
[10/10/19 12:09:30:012] load:0x40078000,len:7684
[10/10/19 12:09:30:012] load:0x40080000,len:7392
[10/10/19 12:09:30:013] entry 0x4008039c
[10/10/19 12:09:30:343] ets Jun 8 2016 00:22:57
[10/10/19 12:09:30:345]
[10/10/19 12:09:30:346] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:30:361] configsip: 0, SPIWP:0xee
[10/10/19 12:09:30:363] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:30:376] mode:DIO, clock div:2
[10/10/19 12:09:30:377] load:0x3fff0018,len:4
[10/10/19 12:09:30:377] load:0x3fff001c,len:5728
[10/10/19 12:09:30:378] ho 0 tail 12 room 4
[10/10/19 12:09:30:378] load:0x40078000,len:7684
[10/10/19 12:09:30:379] load:0x40080000,len:7392
[10/10/19 12:09:30:379] entry 0x4008039c
[10/10/19 12:09:30:699] ets Jun 8 2016 00:22:57
[10/10/19 12:09:30:700]
[10/10/19 12:09:30:700] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:30:707] configsip: 0, SPIWP:0xee
[10/10/19 12:09:30:707] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:30:716] mode:DIO, clock div:2
[10/10/19 12:09:30:722] load:0x3fff0018,len:4
[10/10/19 12:09:30:723] load:0x3fff001c,len:5728
[10/10/19 12:09:30:731] ho 0 tail 12 room 4
[10/10/19 12:09:30:732] load:0x40078000,len:7684
[10/10/19 12:09:30:732] load:0x40080000,len:7392
[10/10/19 12:09:30:733] entry 0x4008039c
[10/10/19 12:09:31:059] ets Jun 8 2016 00:22:57
[10/10/19 12:09:31:062]
[10/10/19 12:09:31:064] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:31:072] configsip: 0, SPIWP:0xee
[10/10/19 12:09:31:073] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:31:086] mode:DIO, clock div:2
[10/10/19 12:09:31:087] load:0x3fff0018,len:4
[10/10/19 12:09:31:087] load:0x3fff001c,len:5728
[10/10/19 12:09:31:094] ho 0 tail 12 room 4
[10/10/19 12:09:31:095] load:0x40078000,len:7684
[10/10/19 12:09:31:096] load:0x40080000,len:7392
[10/10/19 12:09:31:096] entry 0x4008039c
[10/10/19 12:09:31:418] ets Jun 8 2016 00:22:57
[10/10/19 12:09:31:420]
[10/10/19 12:09:31:421] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10/10/19 12:09:31:436] configsip: 0, SPIWP:0xee
[10/10/19 12:09:31:438] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:
0x00
[10/10/19 12:09:31:453] mode:DIO, clock div:2
[10/10/19 12:09:31:454] load:0x3fff0018,len:4
[10/10/19 12:09:31:455] load:0x3fff001c,len:5728
[10/10/19 12:09:31:456] ho 0 tail 12 room 4
[10/10/19 12:09:31:457] load:0x40078000,len:7684
[10/10/19 12:09:31:458] load:0x40080000,len:7392
[10/10/19 12:09:31:468] entry 0x4008039c
[10/10/19 12:09:31:777] ets Jun 8 2016 00:22:57