Can't access core dump with flash encryption and custom partition table

[jcolebaker]

I've turned on core dumps with the file stored to a custom partition on the flash. We use a custom partition table. Here's the partition table entry for the core dumps:

Code: Select all

coredump,data,coredump,0x50000,64K,encrypted

We have moved the partition table location to 0xE000 (because the bootloader needed to be bigger, IIRC):

Code: Select all

CONFIG_PARTITION_TABLE_OFFSET=0xE000

Here are the SDK config options for core dumps:

Code: Select all

CONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=y
CONFIG_ESP_COREDUMP_ENABLE_TO_UART=n
CONFIG_ESP_COREDUMP_ENABLE_TO_NONE=n
CONFIG_ESP_COREDUMP_DATA_FORMAT_BIN=n
CONFIG_ESP_COREDUMP_DATA_FORMAT_ELF=y
CONFIG_ESP_COREDUMP_CHECKSUM_CRC32=y
CONFIG_ESP_COREDUMP_CHECK_BOOT=y
CONFIG_ESP_COREDUMP_ENABLE=y
CONFIG_ESP_COREDUMP_MAX_TASKS_NUM=32

When I create a crash, I see this at boot, which looks good:

Code: Select all

I (879) esp_core_dump_flash: Init core dump to flash
I (884) esp_core_dump_flash: Found partition 'coredump' @ 50000 65536 bytes
I (1135) esp_core_dump_flash: Core dump data checksum is correct
I (1136) esp_core_dump_flash: Found core dump 41284 bytes in flash @ 0x50000

We are using ESP-IDF 4.4.4 - see: https://docs.espressif.com/projects/esp ... _dump.html

However, I have been unable to download and check the core dump via the debug UART. When I try to read and analyze this core dump with "espcoredump.py" tool (as per doc page), I get an error.

Command:

Code: Select all

python C:\Users\Jeremycb\esp-idf\components\espcoredump\espcoredump.py -p COM4 info_corefile --core-format elf --off 327680 .\build\wireless-controller.elf

Output:

Code: Select all

espcoredump.py v0.4-dev
INFO: Invoke esptool to read image.
INFO: Retrieving core dump partition offset and size...
Traceback (most recent call last):
  File "C:\Users\Jeremycb\esp-idf\components\partition_table\parttool.py", line 365, in <module>
    main()
  File "C:\Users\Jeremycb\esp-idf\components\partition_table\parttool.py", line 334, in main
    target = ParttoolTarget(**target_args)
  File "C:\Users\Jeremycb\esp-idf\components\partition_table\parttool.py", line 116, in __init__
    partition_table = gen.PartitionTable.from_binary(f.read())
  File "C:\Users\Jeremycb\esp-idf\components\partition_table\gen_esp32part.py", line 282, in from_binary
    result.append(PartitionDefinition.from_binary(data))
  File "C:\Users\Jeremycb\esp-idf\components\partition_table\gen_esp32part.py", line 434, in from_binary
    res.name = res.name.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 2: invalid start byte
ERROR: parttool get partition info failed with err 1
ERROR: Check if the coredump partition exists in partition table.
ERROR: esptool script execution failed with err 1
INFO: b'esptool.py v3.3.2\r\nSerial port COM4\r\nConnecting.....\r\nDetecting chip type... Unsupported detection protocol, switching and trying again...\r\nConnecting.....\r\nDetecting chip type... ESP32\r\nChip is ESP32-D0WDR2-V3-V3 (revision 3)\r\nFeatures: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None\r\nCrystal is 40MHz\r\nMAC: c4:de:e2:1d:2a:10\r\nUploading stub...\r\nRunning stub...\r\nStub running...\r\n3072 (100 %)\r\n3072 (100 %)\r\nRead 3072 bytes at 0x8000 in 
0.3 seconds (80.9 kbit/s)...\r\nHard resetting via RTS pin...\r\nRunning C:\\Users\\Jeremycb\\.espressif\\python_env\\idf4.4_py3.9_env\\Scripts\\python.exe C:\\Users\\Jeremycb\\esp-idf\\components\\esptool_py\\esptool\\esptool.py --port COM4 read_flash 32768 3072 C:\\Users\\Jeremycb\\AppData\\Local\\Temp\\tmptdbs79fu...\r\n'
ERROR: Error during the subprocess execution
Traceback (most recent call last):
  File "C:\Users\Jeremycb\esp-idf\components\espcoredump\espcoredump.py", line 350, in <module>
    temp_core_files = info_corefile()
  File "C:\Users\Jeremycb\esp-idf\components\espcoredump\espcoredump.py", line 149, in info_corefile
    core_elf_path, target, temp_files = get_core_dump_elf(e_machine=exe_elf.e_machine)
  File "C:\Users\Jeremycb\esp-idf\components\espcoredump\espcoredump.py", line 65, in get_core_dump_elf
    loader = ESPCoreDumpFlashLoader(args.off, args.chip, port=args.port, baud=args.baud)
  File "C:\Users\Jeremycb\esp-idf\components\espcoredump\corefile\loader.py", line 439, in __init__
    self.target = self._load_core_src()
  File "C:\Users\Jeremycb\esp-idf\components\espcoredump\corefile\loader.py", line 164, in _load_core_src
    _header = EspCoreDumpV1Header.parse(coredump_bytes)  # first we use V1 format to get version
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 288, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 300, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 1982, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 2440, in _parse
    return self.subcon._parsereport(stream, context, path)
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 1020, in _parse
    data = stream_read(stream, self.length, path)
  File "C:\Users\Jeremycb\.espressif\python_env\idf4.4_py3.9_env\lib\site-packages\construct\core.py", line 92, in stream_read
    raise StreamError("stream read less than specified amount, expected %d, found %d" % (length, len(data)), path=path)
construct.core.StreamError: Error in path (parsing) -> tot_len
stream read less than specified amount, expected 4, found 0

It looks like it is ignoring the "--off" option and trying to read the partition table at offset 0x8000 (32768) (the default partition table location?).

This used to work, but that was without flash encryption or custom partition table, and with an older version of ESP-IDF...

[ESP_igrr]

I'm afraid in this case the host tool would be unable to decide the core dump, even if it used the correct partition table offset. Both the partition table and the core dump would be encrypted, and typically the encryption key is not known on the host side.

If the scenario is to use core dumps in a deployed product, you can modify your application to send the core dump to a server, and decode the core dump there. In this case, core dump will be transparently decrypted on device side.

If you only want to use core dumps during development, then it's probably simpler to enable core dumps to UART. They will be decoded automatically by idf_monitor.