secure boot + flash encryption in release mode [IDFGH-4332]

[newhobby]

Hi,

I was able to successfully place an ESP32 on secure boot + flash encryption in release mode.
After I flashed the flash_encryption example, everything looked good and I confirmed on the log that it was indeed in release mode and that the encryption worked.

Code: Select all

Adding "flash"'s dependency "all" to list of commands with default set of options.
Executing action: all (aliases: build)
Running ninja in directory /home/owner/esp/esp-idf/examples/security/flash_encryption/build
Executing "ninja all"...
[7/1022] cd /home/owner/esp/esp-idf/examples/security/fl...*******************************************************"
Partition table binary generated. Contents:
*******************************************************************************
# Espressif ESP32 Partition Table
# Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x11000,24K,
storage,data,255,0x17000,4K,encrypted
factory,app,factory,0x20000,1M,
*******************************************************************************
[11/1022] Performing build step for 'bootloader'
ninja: no work to do.
[549/1020] Building C object esp-idf/mbedtls/mbedtls/lib...keFiles/mbedcrypto.dir/__/__/port/esp32/esp_bignum.c.obj
/home/owner/esp/esp-idf/components/mbedtls/port/esp32/esp_bignum.c:254:5: warning: no previous prototype for 'esp_mpi_mul_mpi_mod' [-Wmissing-prototypes]
 int esp_mpi_mul_mpi_mod(mbedtls_mpi *Z, const mbedtls_mpi *X, const mbedtls_mpi *Y, const mbedtls_mpi *M)
     ^~~~~~~~~~~~~~~~~~~
[1019/1020] Generating binary image from built executable
esptool.py v2.9-dev
Generated /home/owner/esp/esp-idf/examples/security/flash_encryption/build/flash_encryption-unsigned.bin
[1020/1020] Generating signed binary image
espsecure.py v2.9-dev
Signed 159024 bytes of data from /home/owner/esp/esp-idf/examples/security/flash_encryption/build/flash_encryption-unsigned.bin with key /home/owner/esp/esp-idf/examples/security/flash_encryption/secure_boot_signing_key_private.pem
Generated signed binary image /home/owner/esp/esp-idf/examples/security/flash_encryption/build/flash_encryption.bin from /home/owner/esp/esp-idf/examples/security/flash_encryption/build/flash_encryption-unsigned.bin
Executing action: flash
Choosing default port b'/dev/ttyUSB0' (use '-p PORT' option to set a specific serial port)
Running esptool.py in directory /home/owner/esp/esp-idf/examples/security/flash_encryption/build
Executing "/home/owner/.espressif/python_env/idf4.1_py3.6_env/bin/python /home/owner/esp/esp-idf/components/esptool_py/esptool/esptool.py -p /dev/ttyUSB0 -b 460800 --before default_reset --after no_reset --chip esp32 write_flash @flash_project_args"...
esptool.py -p /dev/ttyUSB0 -b 460800 --before default_reset --after no_reset --chip esp32 write_flash 0x10000 partition_table/partition-table.bin 0x20000 flash_encryption.bin
esptool.py v2.9-dev
Serial port /dev/ttyUSB0
Connecting......
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 26MHz
MAC: 84:0d:8e:17:84:cc
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 3140 bytes to 175...
Wrote 3140 bytes (175 compressed) at 0x00010000 in 0.0 seconds (effective 3224.5 kbit/s)...
Hash of data verified.
Compressed 159092 bytes to 84173...
Wrote 159092 bytes (84173 compressed) at 0x00020000 in 2.1 seconds (effective 597.5 kbit/s)...
Hash of data verified.

Leaving...
Staying in bootloader.
Executing action: monitor
Running idf_monitor in directory /home/owner/esp/esp-idf/examples/security/flash_encryption
Executing "/home/owner/.espressif/python_env/idf4.1_py3.6_env/bin/python /home/owner/esp/esp-idf/tools/idf_monitor.py -p /dev/ttyUSB0 -b 115200 --toolchain-prefix xtensa-esp32-elf- /home/owner/esp/esp-idf/examples/security/flash_encryption/build/flash_encryption.elf -m '/home/owner/.espressif/python_env/idf4.1_py3.6_env/bin/python' '/home/owner/esp/esp-idf/tools/idf.py'"...
--- idf_monitor on /dev/ttyUSB0 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4
load:0x3fff0034,len:11024
ho 0 tail 12 room 4
load:0x40078000,len:21044
load:0x40080400,len:3896
0x40080400: _init at ??:?

entry 0x40080688
I (40) boot: ESP-IDF v4.1-dirty 2nd stage bootloader
I (40) boot: compile time 09:40:35
I (40) boot: chip revision: 1
I (51) boot.esp32: SPI Speed      : 40MHz
I (51) boot.esp32: SPI Mode       : DIO
I (52) boot.esp32: SPI Flash Size : 4MB
I (56) boot: Enabling RNG early entropy source...
I (62) boot: Partition Table:
I (65) boot: ## Label            Usage          Type ST Offset   Length
I (73) boot:  0 nvs              WiFi data        01 02 00011000 00006000
I (80) boot:  1 storage          Unknown data     01 ff 00017000 00001000
I (88) boot:  2 factory          factory app      00 00 00020000 00100000
I (95) boot: End of partition table
I (99) esp_image: segment 0: paddr=0x00020020 vaddr=0x3f400020 size=0x06610 ( 26128) map
I (118) esp_image: segment 1: paddr=0x00026638 vaddr=0x3ffbdb60 size=0x021e8 (  8680) load
I (122) esp_image: segment 2: paddr=0x00028828 vaddr=0x40080000 size=0x00404 (  1028) load
0x40080000: _WindowOverflow4 at /home/owner/esp/esp-idf/components/freertos/xtensa_vectors.S:1778

I (126) esp_image: segment 3: paddr=0x00028c34 vaddr=0x40080404 size=0x073e4 ( 29668) load
I (148) esp_image: segment 4: paddr=0x00030020 vaddr=0x400d0020 size=0x13d98 ( 81304) map
0x400d0020: _stext at ??:?

I (179) esp_image: segment 5: paddr=0x00043dc0 vaddr=0x400877e8 size=0x02f44 ( 12100) load
0x400877e8: vListInsert at /home/owner/esp/esp-idf/components/freertos/list.c:196

I (185) esp_image: Verifying image signature...
I (529) boot: Loaded app from partition at offset 0x20000
I (530) esp_image: segment 0: paddr=0x00001020 vaddr=0x3fff0030 size=0x00004 (     4) 
I (533) esp_image: segment 1: paddr=0x0000102c vaddr=0x3fff0034 size=0x02b10 ( 11024) 
I (546) esp_image: segment 2: paddr=0x00003b44 vaddr=0x40078000 size=0x05234 ( 21044) 
I (558) esp_image: segment 3: paddr=0x00008d80 vaddr=0x40080400 size=0x00f38 (  3896) 
0x40080400: _init at ??:?

I (560) secure_boot_v1: Generating new secure boot key...
I (576) secure_boot_v1: Generating secure boot digest...
I (627) secure_boot_v1: Digest generation complete.
I (627) boot: Checking flash encryption...
I (627) flash_encrypt: Generating new flash encryption key...
I (633) flash_encrypt: Read & write protecting new key...
I (638) flash_encrypt: Setting CRYPT_CONFIG efuse to 0xF
I (644) flash_encrypt: Disable UART bootloader encryption...
I (650) flash_encrypt: Disable UART bootloader decryption...
I (657) flash_encrypt: Disable UART bootloader MMU cache...
I (663) flash_encrypt: Disable JTAG...
I (667) flash_encrypt: Disable ROM BASIC interpreter fallback...
I (685) esp_image: segment 0: paddr=0x00001020 vaddr=0x3fff0030 size=0x00004 (     4) 
I (686) esp_image: segment 1: paddr=0x0000102c vaddr=0x3fff0034 size=0x02b10 ( 11024) 
I (698) esp_image: segment 2: paddr=0x00003b44 vaddr=0x40078000 size=0x05234 ( 21044) 
I (710) esp_image: segment 3: paddr=0x00008d80 vaddr=0x40080400 size=0x00f38 (  3896) 
0x40080400: _init at ??:?

I (1301) flash_encrypt: Encrypting partition 1 at offset 0x17000...
I (1355) esp_image: segment 0: paddr=0x00020020 vaddr=0x3f400020 size=0x06610 ( 26128) map
I (1365) esp_image: segment 1: paddr=0x00026638 vaddr=0x3ffbdb60 size=0x021e8 (  8680) 
I (1368) esp_image: segment 2: paddr=0x00028828 vaddr=0x40080000 size=0x00404 (  1028) 
0x40080000: _WindowOverflow4 at /home/owner/esp/esp-idf/components/freertos/xtensa_vectors.S:1778

I (1372) esp_image: segment 3: paddr=0x00028c34 vaddr=0x40080404 size=0x073e4 ( 29668) 
I (1392) esp_image: segment 4: paddr=0x00030020 vaddr=0x400d0020 size=0x13d98 ( 81304) map
0x400d0020: _stext at ??:?

I (1423) esp_image: segment 5: paddr=0x00043dc0 vaddr=0x400877e8 size=0x02f44 ( 12100) 
0x400877e8: vListInsert at /home/owner/esp/esp-idf/components/freertos/list.c:196

I (1428) esp_image: Verifying image signature...
I (1767) flash_encrypt: Encrypting partition 2 at offset 0x20000...
I (15194) flash_encrypt: Setting FLASH_CRYPT_CNT for permanent encryption
I (15205) flash_encrypt: Flash encryption completed
I (15206) boot: Checking secure boot...
I (15206) secure_boot_v1: Read & write protecting new key...
I (15211) secure_boot_v1: blowing secure boot efuse...
I (15216) secure_boot_v1: Disable JTAG...
I (15221) secure_boot_v1: Disable ROM BASIC interpreter fallback...
I (15239) secure_boot_v1: secure boot is now enabled for bootloader image
I (15239) boot: Resetting with flash encryption enabled...
ets Jun  8 2016 00:22:57

rst:0x3 (SW_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4
load:0x3fff0034,len:11024
ho 0 tail 12 room 4
load:0x40078000,len:21044
load:0x40080400,len:3896
0x40080400: _init at ??:?

entry 0x40080688
I (93) boot: ESP-IDF v4.1-dirty 2nd stage bootloader
I (93) boot: compile time 09:40:35
I (93) boot: chip revision: 1
I (105) boot.esp32: SPI Speed      : 40MHz
I (105) boot.esp32: SPI Mode       : DIO
I (106) boot.esp32: SPI Flash Size : 4MB
I (111) boot: Enabling RNG early entropy source...
I (116) boot: Partition Table:
I (120) boot: ## Label            Usage          Type ST Offset   Length
I (127) boot:  0 nvs              WiFi data        01 02 00011000 00006000
I (135) boot:  1 storage          Unknown data     01 ff 00017000 00001000
I (142) boot:  2 factory          factory app      00 00 00020000 00100000
I (150) boot: End of partition table
I (154) esp_image: segment 0: paddr=0x00020020 vaddr=0x3f400020 size=0x06610 ( 26128) map
I (173) esp_image: segment 1: paddr=0x00026638 vaddr=0x3ffbdb60 size=0x021e8 (  8680) load
I (177) esp_image: segment 2: paddr=0x00028828 vaddr=0x40080000 size=0x00404 (  1028) load
0x40080000: _WindowOverflow4 at /home/owner/esp/esp-idf/components/freertos/xtensa_vectors.S:1778

I (181) esp_image: segment 3: paddr=0x00028c34 vaddr=0x40080404 size=0x073e4 ( 29668) load
I (203) esp_image: segment 4: paddr=0x00030020 vaddr=0x400d0020 size=0x13d98 ( 81304) map
0x400d0020: _stext at ??:?

I (235) esp_image: segment 5: paddr=0x00043dc0 vaddr=0x400877e8 size=0x02f44 ( 12100) load
0x400877e8: vListInsert at /home/owner/esp/esp-idf/components/freertos/list.c:196

I (241) esp_image: Verifying image signature...
I (586) boot: Loaded app from partition at offset 0x20000
I (586) secure_boot_v1: bootloader secure boot is already enabled. No need to generate digest. continuing..
I (591) boot: Checking flash encryption...
I (596) flash_encrypt: flash encryption is enabled (0 plaintext flashes left)
I (604) boot: Checking secure boot...
I (608) secure_boot_v1: bootloader secure boot is already enabled, continuing..
I (616) boot: Disabling RNG early entropy source...
I (622) cpu_start: Pro cpu up.
I (625) cpu_start: Application information:
I (630) cpu_start: Project name:     flash_encryption
I (636) cpu_start: App version:      v4.1-dirty
I (641) cpu_start: Compile time:     Nov 25 2020 09:42:31
I (647) cpu_start: ELF file SHA256:  f91d98a93f9f6599...
I (653) cpu_start: ESP-IDF:          v4.1-dirty
I (658) cpu_start: Starting app cpu, entry point is 0x400810a4
0x400810a4: call_start_cpu1 at /home/owner/esp/esp-idf/components/esp32/cpu_start.c:271

I (0) cpu_start: App cpu up.
D (669) memory_layout: Checking 7 reserved memory ranges:
D (674) memory_layout: Reserved memory range 0x3ffae000 - 0x3ffae6e0
D (680) memory_layout: Reserved memory range 0x3ffbdb60 - 0x3ffc0548
D (687) memory_layout: Reserved memory range 0x3ffe0000 - 0x3ffe0440
D (693) memory_layout: Reserved memory range 0x3ffe3f20 - 0x3ffe4350
D (699) memory_layout: Reserved memory range 0x40070000 - 0x40078000
D (706) memory_layout: Reserved memory range 0x40078000 - 0x40080000
0x40080000: _WindowOverflow4 at /home/owner/esp/esp-idf/components/freertos/xtensa_vectors.S:1778

D (712) memory_layout: Reserved memory range 0x40080000 - 0x4008a72c
0x40080000: _WindowOverflow4 at /home/owner/esp/esp-idf/components/freertos/xtensa_vectors.S:1778

D (719) memory_layout: Building list of available memory regions:
D (725) memory_layout: Available memory region 0x3ffae6e0 - 0x3ffb0000
D (731) memory_layout: Available memory region 0x3ffb0000 - 0x3ffb8000
D (738) memory_layout: Available memory region 0x3ffb8000 - 0x3ffbdb60
D (745) memory_layout: Available memory region 0x3ffc0548 - 0x3ffc2000
D (751) memory_layout: Available memory region 0x3ffc2000 - 0x3ffc4000
D (758) memory_layout: Available memory region 0x3ffc4000 - 0x3ffc6000
D (764) memory_layout: Available memory region 0x3ffc6000 - 0x3ffc8000
D (771) memory_layout: Available memory region 0x3ffc8000 - 0x3ffca000
D (778) memory_layout: Available memory region 0x3ffca000 - 0x3ffcc000
D (784) memory_layout: Available memory region 0x3ffcc000 - 0x3ffce000
D (791) memory_layout: Available memory region 0x3ffce000 - 0x3ffd0000
D (797) memory_layout: Available memory region 0x3ffd0000 - 0x3ffd2000
D (804) memory_layout: Available memory region 0x3ffd2000 - 0x3ffd4000
D (811) memory_layout: Available memory region 0x3ffd4000 - 0x3ffd6000
D (817) memory_layout: Available memory region 0x3ffd6000 - 0x3ffd8000
D (824) memory_layout: Available memory region 0x3ffd8000 - 0x3ffda000
D (830) memory_layout: Available memory region 0x3ffda000 - 0x3ffdc000
D (837) memory_layout: Available memory region 0x3ffdc000 - 0x3ffde000
D (844) memory_layout: Available memory region 0x3ffde000 - 0x3ffe0000
D (850) memory_layout: Available memory region 0x3ffe0440 - 0x3ffe3f20
D (857) memory_layout: Available memory region 0x3ffe4350 - 0x3ffe8000
D (863) memory_layout: Available memory region 0x3ffe8000 - 0x3fff0000
D (870) memory_layout: Available memory region 0x3fff0000 - 0x3fff8000
D (877) memory_layout: Available memory region 0x3fff8000 - 0x3fffc000
D (883) memory_layout: Available memory region 0x3fffc000 - 0x40000000
D (890) memory_layout: Available memory region 0x4008a72c - 0x4008c000
D (896) memory_layout: Available memory region 0x4008c000 - 0x4008e000
D (903) memory_layout: Available memory region 0x4008e000 - 0x40090000
D (910) memory_layout: Available memory region 0x40090000 - 0x40092000
D (916) memory_layout: Available memory region 0x40092000 - 0x40094000
D (923) memory_layout: Available memory region 0x40094000 - 0x40096000
D (929) memory_layout: Available memory region 0x40096000 - 0x40098000
D (936) memory_layout: Available memory region 0x40098000 - 0x4009a000
D (943) memory_layout: Available memory region 0x4009a000 - 0x4009c000
D (949) memory_layout: Available memory region 0x4009c000 - 0x4009e000
D (956) memory_layout: Available memory region 0x4009e000 - 0x400a0000
I (962) heap_init: Initializing. RAM available for dynamic allocation:
D (970) heap_init: New heap initialised at 0x3ffae6e0
I (975) heap_init: At 3FFAE6E0 len 0000F480 (61 KiB): DRAM
D (981) heap_init: New heap initialised at 0x3ffc0548
I (986) heap_init: At 3FFC0548 len 0001FAB8 (126 KiB): DRAM
I (992) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (999) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
D (1005) heap_init: New heap initialised at 0x4008a72c
I (1010) heap_init: At 4008A72C len 000158D4 (86 KiB): IRAM
I (1017) cpu_start: Pro cpu start user code
D (1029) clk: RTC_SLOW_CLK calibration value: 3153861
D (1038) intr_alloc: Connected src 46 to int 2 (cpu 0)
D (1039) efuse: coding scheme 0
D (1039) efuse: In EFUSE_BLK0__DATA0_REG is used 1 bits starting with 2 bit
E (1044) flash_encrypt: Flash encryption & Secure Boot together requires FLASH_CRYPT_CNT efuse to be write protected. Fixing now...
D (1057) efuse: coding scheme 0
D (1060) efuse: In EFUSE_BLK0__DATA0_REG is used 1 bits starting with 2 bit
D (1067) efuse: coding scheme 0
D (1070) efuse: In EFUSE_BLK0__DATA0_REG is used 1 bits starting with 2 bit
D (1077) efuse: coding scheme 0
D (1080) efuse: coding scheme 0
D (1084) efuse: coding scheme 0
D (1098) efuse: coding scheme 0
D (1098) efuse: In EFUSE_BLK0__DATA0_REG is used 1 bits starting with 2 bit
D (1098) efuse: coding scheme 0
D (1102) efuse: In EFUSE_BLK0__DATA6_REG is used 1 bits starting with 9 bit
D (1110) efuse: coding scheme 0
D (1113) efuse: In EFUSE_BLK0__DATA6_REG is used 1 bits starting with 7 bit
D (1120) efuse: coding scheme 0
D (1123) efuse: In EFUSE_BLK0__DATA6_REG is used 1 bits starting with 8 bit
I (1130) flash_encrypt: Flash encryption mode is RELEASE
D (1136) intr_alloc: Connected src 57 to int 3 (cpu 0)
D (1142) intr_alloc: Connected src 24 to int 9 (cpu 0)
D (1147) FLASH_HAL: extra_dummy: 1
D (1150) spi_flash: trying chip: issi
D (1154) spi_flash: trying chip: gd
D (1157) spi_flash: trying chip: generic
I (1161) spi_flash: detected chip: generic
I (1166) spi_flash: flash io: dio
D (1170) chip_generic: set_io_mode: status before 0x200
D (1184) chip_generic: set_io_mode: status after 0x200
I (1184) cpu_start: Starting scheduler on PRO CPU.
D (0) intr_alloc: Connected src 25 to int 2 (cpu 1)
I (0) cpu_start: Starting scheduler on APP CPU.
D (1196) heap_init: New heap initialised at 0x3ffe0440
D (1206) heap_init: New heap initialised at 0x3ffe4350
D (1216) intr_alloc: Connected src 16 to int 12 (cpu 0)

Example to check Flash Encryption status

Test
D (1226) efuse: coding scheme 0
D (1226) efuse: In EFUSE_BLK0__DATA3_REG is used 1 bits starting with 15 bit
D (1236) efuse: coding scheme 0
D (1236) efuse: In EFUSE_BLK0__DATA5_REG is used 1 bits starting with 20 bit
This is ESP32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 1, 4MB external flash
D (1246) efuse: coding scheme 0
D (1256) efuse: In EFUSE_BLK0__DATA0_REG is used 7 bits starting with 20 bit
FLASH_CRYPT_CNT eFuse value is 127
D (1266) efuse: coding scheme 0
D (1266) efuse: In EFUSE_BLK0__DATA0_REG is used 1 bits starting with 2 bit
D (1276) efuse: coding scheme 0
D (1276) efuse: In EFUSE_BLK0__DATA6_REG is used 1 bits starting with 9 bit
D (1286) efuse: coding scheme 0
D (1286) efuse: In EFUSE_BLK0__DATA6_REG is used 1 bits starting with 7 bit
D (1296) efuse: coding scheme 0
D (1296) efuse: In EFUSE_BLK0__DATA6_REG is used 1 bits starting with 8 bit
Flash encryption feature is enabled in RELEASE mode
D (1306) partition: Loading the partition table
Erasing partition "storage" (0x1000 bytes)
Writing data with esp_partition_write:
I (1366) example: 0x3ffb1e40   00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  |................|
I (1366) example: 0x3ffb1e50   10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  |................|
Reading with esp_partition_read:
I (1376) example: 0x3ffb1e20   00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  |................|
I (1386) example: 0x3ffb1e30   10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  |................|
Reading with spi_flash_read:
I (1396) example: 0x3ffb1e20   1b 6b 2d 7e 16 ad f1 4d  4b 63 ed 16 02 2b 99 e8  |.k-~...MKc...+..|
I (1406) example: 0x3ffb1e30   7c 28 cb d9 4c 71 dd 5c  42 6f ba 62 ed 7a a6 30  ||(..Lq.\Bo.b.z.0|

Done

Then, I tried to flash the same code again through UART with idf.py encrypted-flash
It went through the motion of trying to upload but failed with the following:

Code: Select all

Adding "encrypted-flash"'s dependency "all" to list of commands with default set of options.
Executing action: all (aliases: build)
Running ninja in directory /home/owner/esp/esp-idf/examples/security/flash_encryption/build
Executing "ninja all"...
[1/4] cd /home/owner/esp/esp-idf/examples/security/flash...*******************************************************"
Partition table binary generated. Contents:
*******************************************************************************
# Espressif ESP32 Partition Table
# Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x11000,24K,
storage,data,255,0x17000,4K,encrypted
factory,app,factory,0x20000,1M,
*******************************************************************************
[2/4] Performing build step for 'bootloader'
ninja: no work to do.
Executing action: encrypted-flash
Choosing default port b'/dev/ttyUSB0' (use '-p PORT' option to set a specific serial port)
Running esptool.py in directory /home/owner/esp/esp-idf/examples/security/flash_encryption/build
Executing "/home/owner/.espressif/python_env/idf4.1_py3.6_env/bin/python /home/owner/esp/esp-idf/components/esptool_py/esptool/esptool.py -p /dev/ttyUSB0 -b 460800 --before default_reset --after no_reset --chip esp32 write_flash @flash_encrypted_project_args"...
esptool.py -p /dev/ttyUSB0 -b 460800 --before default_reset --after no_reset --chip esp32 write_flash --encrypt 0x10000 partition_table/partition-table.bin 0x20000 flash_encryption.bin
esptool.py v2.9-dev
Serial port /dev/ttyUSB0
Connecting......
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 26MHz
MAC: 84:0d:8e:17:84:cc
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 4MB

WARNING: - compress and encrypt options are mutually exclusive 
Will flash uncompressed
Wrote 16384 bytes at 0x00010000 in 0.4 seconds (340.7 kbit/s)...

A fatal error occurred: Timed out waiting for packet header
esptool.py failed with exit code 2

Now, it won't allow me to flash anything anymore through UART, which is what I was expecting.
Am I correct on my thinking?
The problem is that it also corrupted the partition table. idf.py monitor gives the following:

Code: Select all

Executing action: monitor
Choosing default port b'/dev/ttyUSB0' (use '-p PORT' option to set a specific serial port)
Running idf_monitor in directory /home/owner/esp/esp-idf/examples/security/flash_encryption
Executing "/home/owner/.espressif/python_env/idf4.1_py3.6_env/bin/python /home/owner/esp/esp-idf/tools/idf_monitor.py -p /dev/ttyUSB0 -b 115200 --toolchain-prefix xtensa-esp32-elf- /home/owner/esp/esp-idf/examples/security/flash_encryption/build/flash_encryption.elf -m '/home/owner/.espressif/python_env/idf4.1_py3.6_env/bin/python' '/home/owner/esp/esp-idf/tools/idf.py'"...
--- idf_monitor on /dev/ttyUSB0 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4
load:0x3fff0034,len:11024
ho 0 tail 12 room 4
load:0x40078000,len:21044
load:0x40080400,len:3896
0x40080400: _init at ??:?

entry 0x40080688
I (48) boot: ESP-IDF v4.1-dirty 2nd stage bootloader
I (48) boot: compile time 09:40:35
I (48) boot: chip revision: 1
I (60) boot.esp32: SPI Speed      : 40MHz
I (60) boot.esp32: SPI Mode       : DIO
I (60) boot.esp32: SPI Flash Size : 4MB
I (65) boot: Enabling RNG early entropy source...
E (70) flash_parts: partition 0 invalid magic number 0x236c
E (76) boot: Failed to verify partition table
E (81) boot: load partition table error!

And it keeps rebooting and giving the same error.
Is any attempt to flash anything through UART just going to brick the chip?
I am pretty sure I cannot recover from this now, right?

[WiFive]

Usually in release mode an efuse gets burned to disable encrypted flashing but seems like it tries anyway and corrupts the encrypted partition table

[newhobby]

That's what I thought and was expecting....
After the eFuse was burned, no UART flashing was allowed anymore.
The fact that it is corrupting the partition table is very concerning....
If anyone simply tries to do it, even by mistake, it will brick the module and leave me with a $400 paper weight because I won't be able to use that board anymore.

[WiFive]

I think in rev 3 you can prevent this it has additional efuses

[newhobby]

Which module would have rev3?
Is wrover-e a rev3 module?

[WiFive]

Yes e series has rev3

[ESP_Angus]

Hi newhobby,

Sorry for the inconvenience this has caused. Yes, when Flash Encryption is in release mode the encrypted-flash target can't be used (if it was possible to reflash the board in this mode then an attacker could flash a new bootloader that dumped the rest of the flash). esptool.py should not be trying to write to flash in this mode but it does and it fails at the hardware level, leaving the flash erased or otherwise corrupted. We will fix this.

If the flash encryption key was generated by the device and you don't have it then there is no way to recover this module, sorry.

The efuse in ESP32 V3 that WiFive is referring to is "Disable UART DL mode". There isn't a config option for this in ESP-IDF v4.1 yet (one may be added in a bugfix release), but you can use espefuse.py to burn it manually:

Code: Select all

esepefuse.py -p PORT burn_efuse UART_DOWNLOAD_DIS 1

Note that burning this efuse will disable all UART download mode functions - ie no more esptool, espefuse, etc. at all.

[newhobby]

If I can't use esptool after burning this eFuse, what is the purpose of it?
Would I be able to recover the module and flash a new encrypted partition table and app?

[ESP_Angus]

If I can't use esptool after burning this eFuse, what is the purpose of it?

The purpose of all of these security features is to lock down the chip against unauthorized access.

Disabling UART download mode is an option to reduce the amount of access. It would have prevented the "encrypted-flash" target from writing to flash, as it prevents all flashing attempts, but I'm afraid it's not a solution to recover your module, or a fix for the root cause bug (the root cause bug: esptool.py should have returned an error when it saw the chip was not in Development mode, instead of trying to flash anyhow. but it still wouldn't have been possible to flash successfully.)

If your plan is to continue to develop a firmware while having flash encryption enabled, the approach we recommend is to configure in Development Mode before the first flash, and then you can use the encrypted-flash target safely for additional flashes. This mode is not secure, but it's the same as Release mode in all other ways - then you can change over to Release Mode when you go to production.

However you will need to do this on a new module, a chip which was previously in Release Mode can't be switched to Development Mode (as this would be a security vulnerability).

Would I be able to recover the module and flash a new encrypted partition table and app?

If the flash encryption is enabled on the device in Release mode then this is probably not possible, sorry.

To be 100% certain, you can try running the "espefuse.py summary" command to print the full eFuse status and post the output. However the purpose of Flash Encryption Release Mode is to prevent access, so if none of the "Potentially Insecure Options" were set then it's probably permanently locked.

EDIT: I've updated the thread title to track the bug in esptool.

[newhobby]

Thanks for explanation.
I didn't plan on developing on release mode. I was testing it to ensure everything would be locked when I move forward with production. It did exactly what I was expecting except for this bug. If it had just returned an error and left the partition table intact as you said, I would have been happy with my test.
The problem with this bug is that it exposes my end product to be bricked by any attempt of trying to flash anything, even if it is by mistake. This would include any person on my contract manufacturing facility, any one of my resellers, any of my repair technicians if the unit comes back for repair and even any of my end users.
But if I understood correctly, with this bug identified, I can prevent this by ensuring a few things:
- make sure I use a wrover-e module and never an older one
- flash the module in release mode
- burn UART_DOWNLOAD_DIS eFuse manually
By doing this, any attempt to flash anything would leave my partition table still intact and prevent a bricked module.
Did I get it right?