esp32 jtag debug issue

[jeromeh]

Hi,

I'm using easyOpenJTAG to debug a custom Esp32 board, but facing problem as below:

GDB log:

Code: Select all

$ xtensa-esp32-elf-gdb -ex 'target remote localhost:3333' ./build/helloworld.elf 
GNU gdb (crosstool-NG 8d95cad) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-build_pc-linux-gnu --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/helloworld.elf...done.
Remote debugging using localhost:3333
0x00000000 in ?? ()
(gdb) b app_main
Breakpoint 1 at 0x40102fcc: file ~/esp/helloworld/main/./main.c, line 15.
(gdb) c
Continuing.
Warning:
Cannot insert breakpoint 1.
Cannot access memory at address 0x40102fcc

(gdb) 

And openocd log as below:

Code: Select all

openocd -s ./tcl -f esp32.cfg 
Open On-Chip Debugger 0.10.0-dev-g90071eb (2017-01-01-22:30)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
adapter speed: 200 kHz
force hard breakpoints
Info : clock speed 200 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : accepting 'gdb' connection on tcp/3333
Info : Auto-detected RTOS: FreeRTOS
Warn : esp32.cpu0: target not halted
Error: can't add breakpoint: target running

Can someone help to check and find some clue here? Thanks,

[ESP_Sprite]

First of all: the first halt (with PC=00000000) actually is an invalid one. The way to get out of it is to continue ('c') and then break (ctrl-c) or reset the esp32.

Secondly, normal breakpoints only work with functions in IRAM. For functions in flash, please use hardware breakpoints (use 'hb' instead of 'b').

Hope that helps.

[jeromeh]

After I connect a SRST to CHIP_UP, actually I'm able to setup a working debug flow, which means 'b'/'c'/'p' commands all working as expected...but I need to type 'monitor reset' in gdb after the gdb session is established.

Now I'm facing a new problem, my test application run into a core dump error, and keep rebooting, so gdb side cannot working as expect to trap any exception.

Minicom output like below:

Code: Select all

ets Jun  8 2016 00:22:57

rst:0x3 (SW_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0x00
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffc0008,len:0
load:0x3ffc0008,len:1964
load:0x40078000,len:3668
load:0x40080000,len:260
entry 0x40080034
I (1370) heap_alloc_caps: Initializing. RAM available for dynamic allocation:
I (1371) heap_alloc_caps: At 3FFC141C len 0001EBE4 (122 KiB): DRAM
I (1402) heap_alloc_caps: At 3FFE8000 len 00018000 (96 KiB): D/IRAM
I (1466) heap_alloc_caps: At 4009B920 len 000046E0 (17 KiB): IRAM
I (1528) cpu_start: Pro cpu up.
I (1563) cpu_start: Single core mode
I (1603) cpu_start: Pro cpu start user code
I (2492) phy: phy_version: 258, Nov 29 2016, 15:51:07, 0, 0
I (3604) cpu_start: Starting scheduler on PRO CPU.
tcpip_task_hdlxxx : 3ffc4f38, prio:18,stack:2048
I (3617) wifi: frc2_timer_task_hdl:3ffc69cc, prio:22, stack:2048
I (3617) wifi: pp_task_hdl : 3ffc9218, prio:23, stack:8192
I (3617) mqtt: Setting WiFi configuration SSID cjzc-wifi...
I (3627) wifi: mode : sta (24:0a:c4:04:04:e4)
mg_connect(www.xxxx.com:1883) OK.
Guru Meditation Error of type IllegalInstruction occurred on core  0. Exception was unhandled.
Register dump:
PC      : 0x40104746  PS      : 0x00060430  A0      : 0x00000000  A1      : 0x3ffcaae0  
A2      : 0x00000000  A3      : 0x00000000  A4      : 0x00000000  A5      : 0x00000000  
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x80104746  A9      : 0x3ffcaa90  
A10     : 0x00000025  A11     : 0x3f406e60  A12     : 0x3f406e30  A13     : 0x00000000  
A14     : 0x00000000  A15     : 0x00000004  SAR     : 0x00000012  EXCCAUSE: 0x00000000  
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffa  

Backtrace: 0x40104746:0x3ffcaae0 0x00000000:0x3ffcab20

                                                      Rebooting...
ets Jun  8 2016 00:22:57

rst:0x3 (SW_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0x00
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffc0008,len:0
load:0x3ffc0008,len:1964
load:0x40078000,len:3668
load:0x40080000,len:260
entry 0x40080034
I (1370) heap_alloc_caps: Initializing. RAM available for dynamic allocation:
I (1371) heap_alloc_caps: At 3FFC141C len 0001EBE4 (122 KiB): DRAM
I (1402) heap_alloc_caps: At 3FFE8000 len 00018000 (96 KiB): D/IRAM
I (1466) heap_alloc_caps: At 4009B920 len 000046E0 (17 KiB): IRAM
I (1528) cpu_start: Pro cpu up.
I (1563) cpu_start: Single core mode
I (1603) cpu_start: Pro cpu start user code
I (2378) phy: phy_version: 258, Nov 29 2016, 15:51:07, 0, 0
I (3454) cpu_start: Starting scheduler on PRO CPU.
tcpip_task_hdlxxx : 3ffc4f38, prio:18,stack:2048
I (3467) wifi: frc2_timer_task_hdl:3ffc69cc, prio:22, stack:2048
I (3467) wifi: pp_task_hdl : 3ffc9218, prio:23, stack:8192
I (3467) mqtt: Setting WiFi configuration SSID cjzc-wifi...
I (3477) wifi: mode : sta (24:0a:c4:04:04:e4)
mg_connect(www.xxxx.com:1883) OK.
Guru Meditation Error of type IllegalInstruction occurred on core  0. Exception was unhandled.
Register dump:
PC      : 0x40104746  PS      : 0x00060430  A0      : 0x00000000  A1      : 0x3ffcaae0  
A2      : 0x00000000  A3      : 0x00000000  A4      : 0x00000000  A5      : 0x00000000  
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x80104746  A9      : 0x3ffcaa90  
A10     : 0x00000025  A11     : 0x3f406e60  A12     : 0x3f406e30  A13     : 0x00000000  
A14     : 0x00000000  A15     : 0x00000004  SAR     : 0x00000012  EXCCAUSE: 0x00000000  
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffa  

Backtrace: 0x40104746:0x3ffcaae0 0x00000000:0x3ffcab20

                                                      Rebooting...

And openocd side, it is keep printing reset messages like below:

Code: Select all

Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Debug controller was reset (pwrstat=0x4F, after clear 0x0F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Debug controller was reset (pwrstat=0x4F, after clear 0x0F).
Info : esp32.cpu0: Debug controller was reset (pwrstat=0x4F, after clear 0x0F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).
Info : esp32.cpu0: Core was reset (pwrstat=0x1F, after clear 0x1F).

What is the right approach to debug core dump issues like this?

[jeromeh]

Can someone from ESP comment on this?

[ESP_Sprite]

Strange. Have you enabled the "Make exception and panic handlers JTAG/OCD aware" option? That should make the esp32 halt where the exception happened instead of panic'ing.

Edit: I've heard that other people also get this behaviour, even with the option I mentioned enabled. Will look into this, maybe it's a bug that snuck in when we added the backtrace functionality.

[ESP_Sprite]

Gotcha, seems a bug snuck into esp-idf when we added the backtrace code. Quick fix: open components/esp32/cpu_util.c and add

Code: Select all

#include "sdkconfig.h"

near the other includes. We'll also fix this in the main esp-idf branch.

[Ritesh]

Hi Espressif System Developers,

We had started to use JTAG debugging over ESP32 Core V2 board and we have successfully connected and started JTAG debugging initially without any issue.

But, After sometime, We are getting below prints into both ESP32 and OpenOCD console as well.

Error: cpu0: esp32_write_dirty_registers (line 311): DSR (FFFFFFFF) indicates target still busy!
Error: cpu0: esp32_write_dirty_registers (line 311): DSR (FFFFFFFF) indicates DIR instruction generated an exception!
Error: cpu0: esp32_write_dirty_registers (line 311): DSR (FFFFFFFF) indicates DIR instruction generated an overrun!
Error: cpu1: esp32_write_dirty_registers (line 311): DSR (FFFFFFFF) indicates target still busy!
Error: cpu1: esp32_write_dirty_registers (line 311): DSR (FFFFFFFF) indicates DIR instruction generated an exception!
Error: cpu1: esp32_write_dirty_registers (line 311): DSR (FFFFFFFF) indicates DIR instruction generated an overrun!
Error: cpu0: xtensa_resume (line 431): DSR (FFFFFFFF) indicates target still busy!
Error: cpu0: xtensa_resume (line 431): DSR (FFFFFFFF) indicates DIR instruction generated an exception!
Error: cpu0: xtensa_resume (line 431): DSR (FFFFFFFF) indicates DIR instruction generated an overrun!
Error: cpu1: xtensa_resume (line 431): DSR (FFFFFFFF) indicates target still busy!
Error: cpu1: xtensa_resume (line 431): DSR (FFFFFFFF) indicates DIR instruction generated an exception!
Error: cpu1: xtensa_resume (line 431): DSR (FFFFFFFF) indicates DIR instruction generated an overrun!
Error: cpu0: esp32_fetch_all_regs (line 163): DSR (FFFFFFFF) indicates target still busy!
Error: cpu0: esp32_fetch_all_regs (line 163): DSR (FFFFFFFF) indicates DIR instruction generated an exception!
Error: cpu0: esp32_fetch_all_regs (line 163): DSR (FFFFFFFF) indicates DIR instruction generated an overrun!
Error: cpu0: esp32_fetch_all_regs (line 190): DSR (FFFFFFFF) indicates target still busy!
Error: cpu0: esp32_fetch_all_regs (line 190): DSR (FFFFFFFF) indicates DIR instruction generated an exception!
Error: cpu0: esp32_fetch_all_regs (line 190): DSR (FFFFFFFF) indicates DIR instruction generated an overrun!

So, Would you please let me know that are we doing something wrong into firmware side or at OpenOCD debugger side?

I have attached full logs of both ESP32 and OpenOCD console into attachment.

Let me know if you need any informations or anything else from my side.

[ESP_Sprite]

Are you sure you aren't reconfiguring (one of) the JTAG pins as normal GPIO or peripheral pins in your application?

[Ritesh]

Hi,

Yes. I am sure that we haven't used any JTAG pins for other purpose. So, still let me check with sample xample with one or two tasks with WiFi enabled and connected with rouer.

But, do you have any idea or any clue regarding this issue? Are we doing something wrong into application code?

[Ritesh]

Are you sure you aren't reconfiguring (one of) the JTAG pins as normal GPIO or peripheral pins in your application?

Also, Are you sure that if any of JTAG pin is being used with any GPIO or any peripheral then this type of issue will be occurred?