Help with crash

[newhobby]

Can someone please help me understand what happened on this crash?

Code: Select all

Guru Meditation Error: Core  0 panic'ed (Interrupt wdt timeout on CPU0). 

Core  0 register dump:
PC      : 0x40099d3c  PS      : 0x00060135  A0      : 0x80097167  A1      : 0x3ffbe900
0x40099d3c: spinlock_acquire at C:/esp/esp-idf/components/esp_hw_support/include/soc/spinlock.h:124
 (inlined by) xPortEnterCriticalTimeout at C:/esp/esp-idf/components/freertos/port/xtensa/port.c:288

A2      : 0x3ffd5c70  A3      : 0x00000000  A4      : 0x00000000  A5      : 0x00060123
A6      : 0x00060123  A7      : 0x00000001  A8      : 0x00000001  A9      : 0x00000001
A10     : 0x00000003  A11     : 0x00060123  A12     : 0x00060123  A13     : 0x3ffd5c7c  
A14     : 0x007d5c70  A15     : 0x003fffff  SAR     : 0x0000001a  EXCCAUSE: 0x00000005
EXCVADDR: 0x00000000  LBEG    : 0x400933dc  LEND    : 0x400933f8  LCOUNT  : 0xffffffff  
0x400933dc: memcpy at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/machine/xtensa/memcpy.S:175

0x400933f8: memcpy at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/machine/xtensa/memcpy.S:197

Core  0 was running in ISR context:
EPC1    : 0x400d41bf  EPC2    : 0x4000bff0  EPC3    : 0x4008573f  EPC4    : 0x4008573f
0x400d41bf: uart_hal_write_txfifo at C:/esp/esp-idf/components/hal/uart_hal_iram.c:35

0x4008573f: _xt_lowint1 at C:/esp/esp-idf/components/freertos/port/xtensa/xtensa_vectors.S:1111

0x4008573f: _xt_lowint1 at C:/esp/esp-idf/components/freertos/port/xtensa/xtensa_vectors.S:1111



Backtrace:0x40099d39:0x3ffbe9000x40097164:0x3ffbe940 0x4008d7ed:0x3ffbe970 0x40085769:0x3ffbe9a0 0x4008d925:0x3ffbb200 0x400823bd:0x3ffbb220 0x4008247d:0x3ffbb240 0x4008fea9:0x3ffbb290 0x401b328f:0x3ffbb2b0 0x401bbb17:0x3ffbb2d0 0x401b9139:0x3ffbb2f0 0x400daef2:0x3ffbb310 0x400daf3d:0x3ffbb340 0x40099b09:0x3ffbb360
0x40099d39: spinlock_acquire at C:/esp/esp-idf/components/esp_hw_support/include/soc/spinlock.h:123
 (inlined by) xPortEnterCriticalTimeout at C:/esp/esp-idf/components/freertos/port/xtensa/port.c:288

0x40097164: vPortEnterCritical at C:/esp/esp-idf/components/freertos/port/xtensa/include/freertos/portmacro.h:578
 (inlined by) xQueueGenericSendFromISR at C:/esp/esp-idf/components/freertos/queue.c:1082

0x4008d7ed: i2c_isr_handler_default at C:/esp/esp-idf/components/driver/i2c.c:506

0x40085769: _xt_lowint1 at C:/esp/esp-idf/components/freertos/port/xtensa/xtensa_vectors.S:1111

0x4008d925: i2c_isr_handler_default at C:/esp/esp-idf/components/driver/i2c.c:531

0x400823bd: heap_caps_malloc at C:/esp/esp-idf/components/heap/heap_caps.c:167

0x4008247d: heap_caps_malloc_prefer at C:/esp/esp-idf/components/heap/heap_caps.c:261

0x4008fea9: wifi_malloc at C:/esp/esp-idf/components/esp_wifi/esp32/esp_adapter.c:78

0x401b328f: ieee80211_timer_process at ??:?

0x401bbb17: pp_timer_process at ??:?

0x401b9139: lmac_stop_hw_txq at ??:?

0x400daef2: timer_process_alarm at C:/esp/esp-idf/components/esp_timer/src/esp_timer.c:360

0x400daf3d: timer_task at C:/esp/esp-idf/components/esp_timer/src/esp_timer.c:386 (discriminator 1)

0x40099b09: vPortTaskWrapper at C:/esp/esp-idf/components/freertos/port/xtensa/port.c:131



Core  1 register dump:
PC      : 0x40097324  PS      : 0x00060d35  A0      : 0x8015455c  A1      : 0x3ffd55a0
0x40097324: xQueueReceive at C:/esp/esp-idf/components/freertos/queue.c:1386

A2      : 0x3ffd5c24  A3      : 0x3ffd55f0  A4      : 0x00000001  A5      : 0x00000000
A6      : 0x00000001  A7      : 0x00000000  A8      : 0x800973b3  A9      : 0x3ffd5580
A10     : 0x3ffc4e18  A11     : 0x3ffd55f0  A12     : 0x3ffc4e1c  A13     : 0x00060d23
A14     : 0x00060d20  A15     : 0x00000001  SAR     : 0x0000001e  EXCCAUSE: 0x00000005
EXCVADDR: 0x00000000  LBEG    : 0x400933dc  LEND    : 0x400933f8  LCOUNT  : 0xffffffff
0x400933dc: memcpy at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/machine/xtensa/memcpy.S:175

0x400933f8: memcpy at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/machine/xtensa/memcpy.S:197



Backtrace:0x40097321:0x3ffd55a00x40154559:0x3ffd55e0 0x400e4217:0x3ffd5620 0x400e4240:0x3ffd5650 0x400e6d54:0x3ffd5670 0x400e7dda:0x3ffd5740 0x400e9471:0x3ffd5770 0x40099b09:0x3ffd57a0
0x40097321: xQueueReceive at C:/esp/esp-idf/components/freertos/queue.c:1386

0x40154559: i2c_master_cmd_begin at C:/esp/esp-idf/components/driver/i2c.c:1466

0x400e4217: GetValue_I2C_Repeat_Start at c:\users\owner\dropbox\azena\glass2\esp32\azena_glass2_production\build/../main/i2c_control.c:878

0x400e4240: GetValueLP5036 at c:\users\owner\dropbox\azena\glass2\esp32\azena_glass2_production\build/../main/i2c_control.c:451

0x400e6d54: Refresh_Buttons at c:\users\owner\dropbox\azena\glass2\esp32\azena_glass2_production\build/../main/i2c_control.c:2580

0x400e7dda: ResetTouchInterrupt at c:\users\owner\dropbox\azena\glass2\esp32\azena_glass2_production\build/../main/i2c_control.c:2799 (discriminator 6)

0x400e9471: gpio_task at c:\users\owner\dropbox\azena\glass2\esp32\azena_glass2_production\build/../main/pins.c:74

0x40099b09: vPortTaskWrapper at C:/esp/esp-idf/components/freertos/port/xtensa/port.c:131





ELF file SHA256: 94e7e271fd5897b9

I (5984) esp_core_dump_flash: Save core dump to flash...
I (5991) esp_core_dump_flash: Erase flash 36864 bytes @ 0xfc2000
I (6483) esp_core_dump_flash: Write end offset 0x8c84, check sum length 4
I (6483) esp_core_dump_flash: Core dump has been saved to flash.

[ESP_Sprite]

- You're seeing an interrupt watchdog timeout. This means that *something* has kept interrupts disabled for too long. This can be either a critical section (which disables interrupts) or an ISR (as interrupts are disabled when an interrupt is handled.
- When the watchdog went off, core 0 was in an ISR servicing the UART.
- Before servicing the UART, that core was doing I2C-related things in a critical section.

Can you tell us a bit more about the program that generates this as well as the crash itself? What does the program do wrt input/output? How often do you see the crash, and is it reproducible or random?

[newhobby]

It may be the way I am handling the interrupt.
I will change it to see if I can resolve the issue.
The interrupt is triggered on GPIO, and I am reading I2C inside the interrupt handler.
I will move it out and hopefully this goes away.
It doesn't happen all the time. In fact, I would say 99% of the time, it works fine. but all of a sudden, it will crash with like the log above.

[ESP_Sprite]

Yeah, you're not supposed to do anything in an ISR that is not specifically marked as interrupt-safe. Better to set a semaphore in the ISR, then send the data in a task that blocks on that semaphore.

[newhobby]

Do you have any example for that?
Bonus if there are multiple tasks that need to read I2C values in addition to the GPIO interrupt.

[ESP_Sprite]

Not immediately, they're probably around... generally, people use the drivers which encapsulate these interrupts into a blocking API.

[MikeMyhre]

Yeah, you're not supposed to do anything in an ISR that is not specifically marked as interrupt-safe. Better to set a semaphore in the ISR, then send the data in a task that blocks on that semaphore.

Thank you so much for the help! You really made my day/week/month!
I ended up using this function instead and it worked:
uart_ll_write_txfifo(&UART_PRG, prgTxPkt,7);
Not HAL, but direct to the FIFO.