Not able to Deocde Coredump but Can decode Backtrace

[GeorgeIoak]

I've tried many different formats and testing but so far I have only been able to get partial information decoding the coredump. If I take the Backtrace that is printed in the serial console I can decode that.

As an example, here's the output from decoding the backtrack addresses:

Code: Select all

c:\as_debug>SET BACKTRACE=0x4008bee8:0x3ffda5e0 0x400f4cc4:0x3ffda5f0 0x400f4d6d:0x3ffdae80 0x400f4df6:0x3ffdaea0 0x4010a9b9:0x3ffdaec0 0x4010a9e2:0x3ffdaf10 0x4010b0f1:0x3ffdaf40 0x4010b26d:0x3ffdaf80 0x401b58bf:0x3ffdafa0 0x401b590e:0x3ffdafd0 0x401b5951:0x3ffdaff0 0x401b5a9a:0x3ffdb010 0x401b5b17:0x3ffdb030

c:\as_debug>xtensa-esp32-elf-addr2line -e "C:\AquaStat\.pio\build\esp32dev\firmware.elf" %BACKTRACE%
/builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/machine/xtensa/strcmp.S:467
C:\AquaStat/src/WebSocketHandler.cpp:112
C:\AquaStat/src/WebSocketHandler.cpp:139
C:\AquaStat/src/WebSocketHandler.cpp:61
c:\users\george\.platformio\packages\toolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
C:\AquaStat/.pio/libdeps/esp32dev/ESP Async WebServer/src/AsyncWebSocket.cpp:864
C:\AquaStat/.pio/libdeps/esp32dev/ESP Async WebServer/src/AsyncWebSocket.cpp:684
C:\AquaStat/.pio/libdeps/esp32dev/ESP Async WebServer/src/AsyncWebSocket.cpp:483
c:\users\george\.platformio\packages\toolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
C:\AquaStat/.pio/libdeps/esp32dev/AsyncTCP@src-2154f270fe2afd37e50c673663dbc05d/src/AsyncTCP.cpp:934
C:\AquaStat/.pio/libdeps/esp32dev/AsyncTCP@src-2154f270fe2afd37e50c673663dbc05d/src/AsyncTCP.cpp:1210
C:\AquaStat/.pio/libdeps/esp32dev/AsyncTCP@src-2154f270fe2afd37e50c673663dbc05d/src/AsyncTCP.cpp:162
C:\AquaStat/.pio/libdeps/esp32dev/AsyncTCP@src-2154f270fe2afd37e50c673663dbc05d/src/AsyncTCP.cpp:197

If I try to decode the actual coredump, I get this:

Code: Select all

py c:\Users\george\esp\esp-idf\components\espcoredump\espcoredump.py info_corefile "C:\AquaStat\.pio\build\esp32dev\firmware.elf" --core "C:\Users\george\OneDrive\Downloads\coredump_349454b62818_0.6.2_230612_1128.bin" --core-format raw
WARNING: C:\AquaStat\.pio\build\esp32dev\project_description.json does not exist. Please build the app with "idf.py build"
===============================================================
==================== ESP32 CORE DUMP START ====================

Crashed task handle: 0x3ffdb0ec, name: '', GDB name: 'process 1073590508'

================== CURRENT THREAD REGISTERS ===================
exccause       0x3 (LoadStoreErrorCause)
excvaddr       0x401cb114
epc1           0x40130241
epc2           0x0
epc3           0x0
epc4           0x0
epc5           0x4008319e
epc6           0x0
eps2           0x0
eps3           0x0
eps4           0x0
eps5           0x60120
eps6           0x0
pc             0x4008beeb          0x4008beeb
lbeg           0x4008cdcd          1074318797
lend           0x4008cddd          1074318813
lcount         0xfffffffd          4294967293
sar            0x13                19
ps             0x60720             395040
threadptr      <unavailable>
br             <unavailable>
scompare1      <unavailable>
acclo          <unavailable>
acchi          <unavailable>
m0             <unavailable>
m1             <unavailable>
m2             <unavailable>
m3             <unavailable>
expstate       <unavailable>
f64r_lo        <unavailable>
f64r_hi        <unavailable>
f64s           <unavailable>
fcr            <unavailable>
fsr            <unavailable>
a0             0x800f4ca3          -2146480989
a1             0x3ffda5f0          1073587696
a2             0x401cb114          1075622164
a3             0x3ffb3064          1073426532
a4             0xff                255
a5             0xff00              65280
a6             0xff0000            16711680
a7             0xff000000          -16777216
a8             0x78                120
a9             0x73                115
a10            0x3                 3
a11            0x8                 8
a12            0x1                 1
a13            0x8                 8
a14            0x65                101
a15            0x65                101

==================== CURRENT THREAD STACK =====================
#0  0x4008beeb in ?? ()
#1  0x400f4ca3 in ?? ()
#2  0x400f4d4c in ?? ()
#3  0x400f4dd5 in ?? ()
#4  0x4010a85c in ?? ()
#5  0x4010a885 in ?? ()
#6  0x4010af94 in ?? ()
#7  0x4010b110 in ?? ()
#8  0x401b52e6 in ?? ()
#9  0x401b5335 in ?? ()
#10 0x401b5378 in ?? ()
#11 0x401b54c1 in ?? ()
#12 0x401b553e in ?? ()

======================== THREADS INFO =========================
  Id   Target Id          Frame
* 1    process 1073590508 0x4008beeb in ?? ()
  2    process 1073447304 0x40084020 in ?? ()
  3    process 1073469484 0x401ce9ea in ?? ()
  4    process 1073469124 0x401ce9ea in ?? ()
  5    process 1073433656 0x40084020 in ?? ()
  6    process 1073438184 0x40084020 in ?? ()
  7    process 1073437824 0x40084020 in ?? ()
  8    process 1073445616 0x40084008 in ?? ()
  9    process 1073443916 0x40084020 in ?? ()
  10   process 1073572072 0x40084020 in ?? ()
  11   process 1073536024 0x40084020 in ?? ()
  12   process 1073466592 0x4000bff0 in ?? ()

==================== THREAD 1 (TCB: 0x3ffdb0ec, name: '') =====================
#0  0x4008beeb in ?? ()
#1  0x400f4ca3 in ?? ()
#2  0x400f4d4c in ?? ()
#3  0x400f4dd5 in ?? ()
#4  0x4010a85c in ?? ()
#5  0x4010a885 in ?? ()
#6  0x4010af94 in ?? ()
#7  0x4010b110 in ?? ()
#8  0x401b52e6 in ?? ()
#9  0x401b5335 in ?? ()
#10 0x401b5378 in ?? ()
#11 0x401b54c1 in ?? ()
#12 0x401b553e in ?? ()

==================== THREAD 2 (TCB: 0x3ffb8188, name: '') =====================
#0  0x40084020 in ?? ()
#1  0x40092760 in ?? ()
#2  0x400f50c8 in ?? ()
#3  0x400f7e2c in ?? ()
#4  0x401241e8 in ?? ()

==================== THREAD 3 (TCB: 0x3ffbd82c, name: '') =====================
#0  0x401ce9ea in ?? ()
#1  0x401314e0 in ?? ()
#2  0x4009164b in ?? ()

==================== THREAD 4 (TCB: 0x3ffbd6c4, name: '') =====================
#0  0x401ce9ea in ?? ()
#1  0x401314e0 in ?? ()
#2  0x4009164b in ?? ()

==================== THREAD 5 (TCB: 0x3ffb4c38, name: '') =====================
#0  0x40084020 in ?? ()
#1  0x40090fd2 in ?? ()
#2  0x4014dcae in ?? ()
#3  0x4013e126 in ?? ()

==================== THREAD 6 (TCB: 0x3ffb5de8, name: '') =====================
#0  0x40084020 in ?? ()
#1  0x40090fd2 in ?? ()
#2  0x400fbd2d in ?? ()

==================== THREAD 7 (TCB: 0x3ffb5c80, name: '') =====================
#0  0x40084020 in ?? ()
#1  0x40090fd2 in ?? ()
#2  0x401d6464 in ?? ()
#3  0x401d65eb in ?? ()

==================== THREAD 8 (TCB: 0x3ffb7af0, name: '') =====================
#0  0x40084008 in ?? ()
#1  0x40084020 in ?? ()
#2  0x40091138 in ?? ()
#3  0x40086c77 in ?? ()

==================== THREAD 9 (TCB: 0x3ffb744c, name: '') =====================
#0  0x40084020 in ?? ()
#1  0x40091138 in ?? ()
#2  0x40086c77 in ?? ()

==================== THREAD 10 (TCB: 0x3ffd68e8, name: '') =====================
#0  0x40084020 in ?? ()
#1  0x40090fd2 in ?? ()
#2  0x4015943e in ?? ()

==================== THREAD 11 (TCB: 0x3ffcdc18, name: '') =====================
#0  0x40084020 in ?? ()
#1  0x40090fd2 in ?? ()
#2  0x4013ba98 in ?? ()
#3  0x401d2558 in ?? ()

==================== THREAD 12 (TCB: 0x3ffbcce0, name: '') =====================
#0  0x4000bff0 in ?? ()
#1  0x40093688 in ?? ()
#2  0x40092cb9 in ?? ()
#3  0x40153d99 in ?? ()


======================= ALL MEMORY REGIONS ========================
Name   Address   Size   Attrs
.coredump.tasks.data 0x3ffdb0ec 0x158 RW
.coredump.tasks.data 0x3ffda530 0xba0 RW
.coredump.tasks.data 0x3ffb8188 0x158 RW
.coredump.tasks.data 0x3ffb2150 0x1c0 RW
.coredump.tasks.data 0x3ffbd82c 0x158 RW
.coredump.tasks.data 0x3ffcea00 0x180 RW
.coredump.tasks.data 0x3ffbd6c4 0x158 RW
.coredump.tasks.data 0x3ffbd530 0x180 RW
.coredump.tasks.data 0x3ffb4c38 0x158 RW
.coredump.tasks.data 0x3ffb4a40 0x1e0 RW
.coredump.tasks.data 0x3ffb5de8 0x158 RW
.coredump.tasks.data 0x3ffaf6e0 0x1b0 RW
.coredump.tasks.data 0x3ffb5c80 0x158 RW
.coredump.tasks.data 0x3ffb5a70 0x1f0 RW
.coredump.tasks.data 0x3ffb7af0 0x158 RW
.coredump.tasks.data 0x3ffb7910 0x1c0 RW
.coredump.tasks.data 0x3ffb744c 0x158 RW
.coredump.tasks.data 0x3ffb7290 0x1a0 RW
.coredump.tasks.data 0x3ffd68e8 0x158 RW
.coredump.tasks.data 0x3ffd66f0 0x1e0 RW
.coredump.tasks.data 0x3ffcdc18 0x158 RW
.coredump.tasks.data 0x3ffd0970 0x1d0 RW
.coredump.tasks.data 0x3ffbcce0 0x158 RW
.coredump.tasks.data 0x3ffbcb20 0x1a0 RW

===================== ESP32 CORE DUMP END =====================
===============================================================
Exception in thread Thread-1 (_readerthread):
Traceback (most recent call last):
  File "C:\Users\george\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-2 (_readerthread):
Traceback (most recent call last):
  File "C:\Users\george\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\george\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run
    self.run()
  File "C:\Users\george\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\george\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1499, in _readerthread
    self._target(*self._args, **self._kwargs)
  File "C:\Users\george\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1499, in _readerthread
    buffer.append(fh.read())
    buffer.append(fh.read())
OSError: [Errno 22] Invalid argument
OSError: [Errno 22] Invalid argument
Done!

I have tried MANY different ways to get the coredump and also tried feeding in different `--core-format` since it looks like the default is actually elf but raw seems to give me a little more results.

[GeorgeIoak]

I think this might be useful information for others (but I'm still looking to find why coredump can't be analyzed).

You can use this function to retrieve the backtrace AFTER the system reboots:

Code: Select all

void print_backtrace_info(const esp_core_dump_summary_t *coredump_summary)
{
  if (coredump_summary != NULL)
  {
    esp_core_dump_bt_info_t bt_info = coredump_summary->exc_bt_info;

    char results[256]; // Assuming a maximum of 256 characters for the backtrace string
    int offset = snprintf(results, sizeof(results), "Backtrace:");

    for (int i = 0; i < bt_info.depth; i++)
    {
      uintptr_t pc = bt_info.bt[i]; // Program Counter (PC)
      int len = snprintf(results + offset, sizeof(results) - offset, " 0x%08X", pc);
      if (len >= 0 && offset + len < sizeof(results))
      {
        offset += len;
      }
      else
      {
        break; // Reached the limit of the results buffer
      }
    }

    LOG_NOTICE("[backtrace]: %s", results);
    LOG_NOTICE("[backtrace]Backtrace Depth: %u", bt_info.depth);
    LOG_NOTICE("[backtrace]Backtrace Corrupted: %s", bt_info.corrupted ? "Yes" : "No");
    LOG_NOTICE("[backtrace]Program Counter: %d", coredump_summary->exc_pc);
    LOG_NOTICE("[backtrace]Coredump Version: %d", coredump_summary->core_dump_version);
  }
  else
  {
    LOG_NOTICE("Invalid core dump summary");
  }
}

and in my main.cpp setup() I have:

Code: Select all

  esp_core_dump_summary_t *coredump_summary = (esp_core_dump_summary_t *)malloc(sizeof(esp_core_dump_summary_t));
  if (coredump_summary != NULL)
  {
    esp_err_t err = esp_core_dump_get_summary(coredump_summary);
    if (err == ESP_OK)
    {
      print_backtrace_info(coredump_summary);
    }
    else
    {
      Serial.printf("Failed to get core dump summary: %d\n", err);
    }
    free(coredump_summary); // Free the allocated memory
  }
  else
  {
    Serial.println("Failed to allocate memory for core dump summary");
  }

This only retrieves the PC and in the serial console the panic routine is able to print the PC:SP pair (The backtrace line contains PC:SP pairs, where PC is the Program Counter and SP is Stack Pointer, for each stack frame of the current task).

BUT, what I found is that you ONLY need to feed the PC array (along with the ELF) to xtensa-esp32-elf-addr2line utility. I guess the SP are used more in debug sessions but for me I got the same results whether or not I fed xtensa-esp32-elf-addr2line the PC:SP pairs or just the PC.

[GeorgeIoak]

Can anybody confirm that they've successfully decoded a coredump?

[HitecSmartHome]

Can anybody confirm that they've successfully decoded a coredump?

Still trying