Problems Flashing Firmware With Secure Boot and Flash Encryption

[jcolebaker]

Hi, I am trying to get flash encryption and secure boot working in "development" mode so that we can run and test all our code with these options on, but still be able to erase the flash and re-program (e.g. with "idf.py erase-flash") during development.

I have generated an RSA key for code signing and secure boot.
Here are the config options I turned on:

Code: Select all

(Top) → Component config → ESP32-specific
  Minimum Supported ESP32 Revision (Rev 3) --->

(Top) → Security features
    App Signing Scheme (RSA)  --->
[*] Enable hardware Secure Boot in bootloader (READ DOCS FIRST)
        Select secure boot version (Enable Secure Boot version 2)  --->
[*] Sign binaries during build
    (signing_key/secure_boot_v2_signing_key.pem) Secure boot private signing key
[*] Allow potentially insecure options
[*] Enable flash encryption on boot (READ DOCS FIRST)
        Enable usage mode (Development (NOT SECURE))  --->

	Potentially insecure options --->
    [ ] Leave ROM BASIC Interpreter available on reset (NEW)
    [ ] Allow JTAG Debugging (NEW)
    [ ] Allow app partition length not 64KB aligned (NEW)
    [*] Allow additional read protecting of efuses
    -*- Leave UART bootloader encryption enabled (NEW)
    [*] Leave UART bootloader decryption enabled
    [*] Leave UART bootloader flash cache enabled
    [ ] Require flash encryption to be already enabled

[ ] Check Flash Encryption enabled on app startup (NEW)
UART ROM download mode (UART ROM download mode (Enabled (not recommended))) --->

The first time I installed this build, I saw log messages about encrypting the flash partitions, and querying the efuses etc. showed that secure boot and flash encryption had been enabled in "developer" mode. Everything worked well.

HOWEVER, I then erased the flash:

Code: Select all

idf.py -p COM4 erase-flash

And tried to flash the firmware again, but it no longer boots.

According to my understanding, with these options, it should be possible to re-write the firmware even with encryption enabled because UART bootloader encryption and decryption are enabled, and the bootloader can access the encrypted flash transparently.

I have tried various options, e.g. this sequence:

Code: Select all

idf.py -p COM4 erase-flash
idf.py -p COM4 bootloader
idf.py -p COM4 bootloader-flash
idf.py -p COM4 encrypted-flash

But I typically get this when I use "monitor":

Code: Select all

ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
invalid header: 0xa239a46e
invalid header: 0xa239a46e
invalid header: 0xa239a46e
invalid header: 0xa239a46e
invalid header: 0xa239a46e
invalid header: 0xa239a46e
invalid header: 0xa239a46e
ets Jul 29 2019 12:21:46

There are instructions here which say "idf.py bootloader will produce a pre-digested bootloader and secure boot digest file for flashing at offset 0x0. When following the plaintext serial re-flashing steps it is necessary to re-flash this file before flashing other plaintext data." I think this is supposed to generate "bootloader-reflash-digest.bin" ???, but my build doesn't create this file and I am not sure why it is needed. The instructions also talk about a "Reflashable" option ("SECURE_BOOTLOADER_REFLASHABLE") but I don't see this option in the security configuration.

Is there any way to recover my board? What is the process to re-flash the firmware (including 2nd stage bootloader and partition table) with these options? Or have I got the options wrong?

Code: Select all

ESP32 (ESP32-D0WD-V3, revision 3)
ESP-IDF 4.4

PS C:\Users\Jeremycb\esp\wireless-controller> espefuse -p COM4 summary
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting.....
Detecting chip type... ESP32
espefuse.py v3.2-dev
EFUSE_NAME (Block) Description  = [Meaningful Value] [Readable/Writeable] (Hex Value)
----------------------------------------------------------------------------------------
Calibration fuses:
BLK3_PART_RESERVE (BLOCK0):                        BLOCK3 partially served for ADC calibration data   = False R/W (0b0)
ADC_VREF (BLOCK0):                                 Voltage reference calibration                      = 1093 R/W (0b10001)

Config fuses:
XPD_SDIO_FORCE (BLOCK0):                           Ignore MTDI pin (GPIO12) for VDD_SDIO on reset     = False R/W (0b0)
XPD_SDIO_REG (BLOCK0):                             If XPD_SDIO_FORCE, enable VDD_SDIO reg on reset    = False R/W (0b0)
XPD_SDIO_TIEH (BLOCK0):                            If XPD_SDIO_FORCE & XPD_SDIO_REG                   = 1.8V R/W (0b0)
CLK8M_FREQ (BLOCK0):                               8MHz clock freq override                           = 55 R/W (0x37)
SPI_PAD_CONFIG_CLK (BLOCK0):                       Override SD_CLK pad (GPIO6/SPICLK)                 = 0 R/W (0b00000)
SPI_PAD_CONFIG_Q (BLOCK0):                         Override SD_DATA_0 pad (GPIO7/SPIQ)                = 0 R/W (0b00000)
SPI_PAD_CONFIG_D (BLOCK0):                         Override SD_DATA_1 pad (GPIO8/SPID)                = 0 R/W (0b00000)
SPI_PAD_CONFIG_HD (BLOCK0):                        Override SD_DATA_2 pad (GPIO9/SPIHD)               = 0 R/W (0b00000)
SPI_PAD_CONFIG_CS0 (BLOCK0):                       Override SD_CMD pad (GPIO11/SPICS0)                = 0 R/W (0b00000)
DISABLE_SDIO_HOST (BLOCK0):                        Disable SDIO host                                  = False R/W (0b0)

Efuse fuses:
WR_DIS (BLOCK0):                                   Efuse write disable mask                           = 384 R/W (0x0180)
RD_DIS (BLOCK0):                                   Efuse read disable mask                            = 1 R/W (0x1)
CODING_SCHEME (BLOCK0):                            Efuse variable block length scheme
   = NONE (BLK1-3 len=256 bits) R/W (0b00)
KEY_STATUS (BLOCK0):                               Usage of efuse block 3 (reserved)                  = False R/W (0b0)

Identity fuses:
MAC (BLOCK0):                                      Factory MAC Address
   = 34:ab:95:49:47:dc (CRC 0xac OK) R/W
MAC_CRC (BLOCK0):                                  CRC8 for factory MAC address                       = 172 R/W (0xac)
CHIP_VER_REV1 (BLOCK0):                            Silicon Revision 1                                 = True R/W (0b1)
CHIP_VER_REV2 (BLOCK0):                            Silicon Revision 2                                 = True R/W (0b1)
CHIP_VERSION (BLOCK0):                             Reserved for future chip versions                  = 2 R/W (0b10)
CHIP_PACKAGE (BLOCK0):                             Chip package identifier                            = 1 R/W (0b001)
MAC_VERSION (BLOCK3):                              Version of the MAC field                           = 0 R/W (0x00)

Security fuses:
FLASH_CRYPT_CNT (BLOCK0):                          Flash encryption mode counter                      = 1 R/W (0b0000001)
UART_DOWNLOAD_DIS (BLOCK0):                        Disable UART download mode (ESP32 rev3 only)       = False R/W (0b0)
FLASH_CRYPT_CONFIG (BLOCK0):                       Flash encryption config (key tweak bits)           = 15 R/W (0xf)
CONSOLE_DEBUG_DISABLE (BLOCK0):                    Disable ROM BASIC interpreter fallback             = True R/W (0b1)
ABS_DONE_0 (BLOCK0):                               Secure boot V1 is enabled for bootloader image     = False R/W (0b0)
ABS_DONE_1 (BLOCK0):                               Secure boot V2 is enabled for bootloader image     = True R/W (0b1)
JTAG_DISABLE (BLOCK0):                             Disable JTAG                                       = True R/W (0b1)
DISABLE_DL_ENCRYPT (BLOCK0):                       Disable flash encryption in UART bootloader        = False R/W (0b0)
DISABLE_DL_DECRYPT (BLOCK0):                       Disable flash decryption in UART bootloader        = False R/W (0b0)
DISABLE_DL_CACHE (BLOCK0):                         Disable flash cache in UART bootloader             = False R/W (0b0)
BLOCK1 (BLOCK1):                                   Flash encryption key
   = ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? -/-
BLOCK2 (BLOCK2):                                   Secure boot key
   = 84 d7 c4 4d 6b 57 85 e3 1b bf 88 65 89 61 88 16 07 78 7a 22 66 c1 a5 b2 75 42 53 43 28 d9 2d 7c R/-
BLOCK3 (BLOCK3):                                   Variable Block 3
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W

Flash voltage (VDD_SDIO) determined by GPIO12 on reset (High for 1.8V, Low/NC for 3.3V).

[ESP_igrr]

Could you please try "idf.py encrypted-bootloader-flash" instead of "idf.py bootloader-flash"?

(Normally, "idf.py flash" or "idf.py encrypted-flash will also flash the bootloader; however this doesn't happen when the Secure Boot feature is enabled, so the bootloader needs to be flashed manually.)

[jcolebaker]

Thankyou, that worked. I used the following:

Code: Select all

idf.py -p COM4 encrypted-bootloader-flash
idf.py -p COM4 encrypted-flash

I am using IDF4.4, and the "idf.py" command doesn't list "encrypted-bootloader-flash" as one of the command options in its help screen.

Can you describe what "encrypted-bootloader-flash" and "encrypted-flash" do differently to the normal "flash" command? I would like to have an easy way for testers to set up their devices, but they don't have the build tools installed and so don't have access to "idf.py".

Thanks!

[ESP_igrr]

Basically, the "encrypted-" versions of flashing targets use "--encrypt-files" argument of esptool.py to enable encryption during flashing.
Note that this only works in the development mode of flash encryption.

[jcolebaker]

Does that mean the encryption is done by esptool.py during the download process? So it has to read the secret flash encryption key from the efuses? I thought the encryption happened transparently when the bootloader wrote the new files to the flash.

From the "SECURE_FLASH_ENCRYPTION_MODE" config option help:
"By default Development mode is enabled which allows ROM download mode to perform flash encryption
operations (plaintext is sent to the device, and it encrypts it internally and writes ciphertext
to flash.)"

But that would mean we wouldn't need a different "esptool" download command.

[ESP_igrr]

When CONFIG_SECURE_FLASH_ENCRYPTION_MODE is enabled, encryption process indeed happens on the chip.

However esptool.py still has control whether encryption on the chip will be performed or if the plain text data will be flashed.

This is necessary, for instance, because some of the data partitions (like NVS or SPIFFS) need to be flashed in plain text. Therefore there is a command line flag which is used to tell esptool.py whether to perform the encryption or not.

[robert-t]

Hi, whitout opening a new topic, I am on idf 4.0.*, secure boot v1 and flashencryption enabled. I flashed once the ESP32. (patition table, ota partition, app, nvs_key). I need to flash now only the app, again, with commands I should use? should I flash it only the app at the same address?

I think I have done something wrong, because now the module keeps rebooting (flash read err, 1000).

[jcolebaker]

Hi, @robert-t

Have a look at "Possible Failures" on this page:
https://docs.espressif.com/projects/esp ... e-failures

It sounds like you have flashed the bootloader in normal / plain text mode at some point.

This should fix it (using idf.py):

Code: Select all

idf.py -p <PORT> erase-flash
idf.py -p <PORT> encrypted-bootloader-flash
idf.py -p <PORT> encrypted-flash

Once the system is working again, if you just want to flash the app, use:

Code: Select all

idf.py -p <PORT> encrypted-app-flash

You can also use "esptool" (exe or python script) to do the same thing. Note the use of "--encrypted" option:

Code: Select all

python esptool.py --chip esp32 -p <PORT> -b 460800 --before=default_reset --after=no_reset write_flash --flash_mode dio --flash_freq 40m --flash_size 4MB --encrypt 0x1000 bootloader/bootloader.bin 0x180000 my-firmware.bin 0xe000 partition_table/partition-table.bin 0x16000 ota_data_initial.bin

You would need to change the bin names and install addresses as required to match your application.

[robert-t]

Thanks a lot

@jcolebaker

I will try