Problem enabling SecureBoot and FlashEncryption

[vsimionescu]

Hello

I am working on a project that uses the ESP32 (wroom32) and I am doing an "auto config/build/flash" tool and i have some problems with the security options.

I am using ESP32 with the ESP-IDF environment for building. I am trying to enable SecureBoot (V1, as the chips seem too old for V2) and Flash encryption.

I have configured with idf.py menuconfig the following: SecureBoot one time flash and FlashEncryption in developer mode (I don't want the OTA updates). I am using for the secure boot a openssl generated key and for the flash a key generated with the espsecure.py generate_flash_encryption_key.

After having the keys generated and the config done I write the flash key (using espefuse.py burn_key flash_encryption); I do a full clean (noticed that otherwise the signing certificate is not updated or something was not working with the secure boot), build the bootloader (idf.py bootloader), build the rest of the images (idf.py build) and flash everything (in one command) not encrypted, in plain text, to the target (using esptool).

I expected that, as mentioned in https://docs.espressif.com/projects/esp ... ption.html (The image will include the firmware bootloader, partition table, application, and other partitions marked by the user as encrypted. These binaries will be written to flash memory unencrypted. Once the flashing is complete, your device will reset. On the next boot, the firmware bootloader encrypts the flash application partition and then resets. After that, the sample application is decrypted at runtime and executed.) the ESP32 will start the encryption process and this will in the end reset the device and after that it will work.

What actually happens is:
1) after the flash I get constant bootloader errors:

Code: Select all

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun  8 2016 00:22:57

2) i can see with espfuse that the fuse config was done for the flashencrypt

Code: Select all

EFUSE_NAME             Description = [Meaningful Value] [Readable/Writeable] (Hex Value)
----------------------------------------------------------------------------------------
Efuse fuses:
WR_DIS                 Efuse write disable mask                          = 128 R/W (0x80)
RD_DIS                 Efuse read disablemask                            = 1 R/W (0x1)
CODING_SCHEME          Efuse variable block length scheme                = 0 R/W (0x0)
KEY_STATUS             Usage of efuse block 3 (reserved)                 = 0 R/W (0x0)

Security fuses:
FLASH_CRYPT_CNT        Flash encryption mode counter                     = 0 R/W (0x0)
FLASH_CRYPT_CONFIG     Flash encryption config (key tweak bits)          = 15 R/W (0xf)
CONSOLE_DEBUG_DISABLE  Disable ROM BASIC interpreter fallback            = 1 R/W (0x1)
ABS_DONE_0             secure boot enabled for bootloader                = 0 R/W (0x0)
ABS_DONE_1             secure boot abstract 1 locked                     = 0 R/W (0x0)
JTAG_DISABLE           Disable JTAG                                      = 1 R/W (0x1)
DISABLE_DL_ENCRYPT     Disable flash encryption in UART bootloader       = 0 R/W (0x0)
DISABLE_DL_DECRYPT     Disable flash decryption in UART bootloader       = 1 R/W (0x1)
DISABLE_DL_CACHE       Disable flash cache in UART bootloader            = 1 R/W (0x1)
BLK1                   Flash encryption key
  = ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? -/-
BLK2                   Secure boot key
  = 9e 44 2f ed 44 8c 91 05 0a 6a 94 02 09 77 30 17 44 3a 1b 0b 88 8f 19 aa 8f a8 89 92 2f 83 76 d7 R/W
BLK3                   Variable Block 3
  = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W

Calibration fuses:
BLK3_PART_RESERVE      BLOCK3 partially served for ADC calibration data  = 0 R/W (0x0)
ADC_VREF               Voltage reference calibration                     = 1114 R/W (0x2)

Config fuses:
XPD_SDIO_FORCE         Ignore MTDI pin (GPIO12) for VDD_SDIO on reset    = 0 R/W (0x0)
XPD_SDIO_REG           If XPD_SDIO_FORCE, enable VDD_SDIO reg on reset   = 0 R/W (0x0)
XPD_SDIO_TIEH          If XPD_SDIO_FORCE & XPD_SDIO_REG, 1=3.3V 0=1.8V   = 0 R/W (0x0)
CLK8M_FREQ             8MHz clock freq override                          = 52 R/W (0x34)
SPI_PAD_CONFIG_CLK     Override SD_CLK pad (GPIO6/SPICLK)                = 0 R/W (0x0)
SPI_PAD_CONFIG_Q       Override SD_DATA_0 pad (GPIO7/SPIQ)               = 0 R/W (0x0)
SPI_PAD_CONFIG_D       Override SD_DATA_1 pad (GPIO8/SPID)               = 0 R/W (0x0)
SPI_PAD_CONFIG_HD      Override SD_DATA_2 pad (GPIO9/SPIHD)              = 0 R/W (0x0)
SPI_PAD_CONFIG_CS0     Override SD_CMD pad (GPIO11/SPICS0)               = 0 R/W (0x0)
DISABLE_SDIO_HOST      Disable SDIO host                                 = 0 R/W (0x0)

Identity fuses:
MAC                    Factory MAC Address
  = a4:cf:12:7c:a0:98 (CRC 0x9b OK) R/W
CHIP_VER_REV1          Silicon Revision 1                                = 1 R/W (0x1)
CHIP_VER_REV2          Silicon Revision 2                                = 0 R/W (0x0)
CHIP_VERSION           Reserved for future chip versions                 = 2 R/W (0x2)
CHIP_PACKAGE           Chip package identifier                           = 0 R/W (0x0)

Flash voltage (VDD_SDIO) determined by GPIO12 on reset (High for 1.8V, Low/NC for 3.3V).

To recover from this, the only solution I found was: to burn the FLASH_CRYPT_CNT (espefuse.py burn_efuse FLASH_CRYPT_CNT) and to make another flash after I have manually encrypted everything. Doing this the ESP32 works and it seems encrypted and with secure boot active but it seems that the process is not correct.

Questions:
1) Is this normal, did I misunderstand the auto encrypt with a self generated key process
2) Can the fullclean be easily avoided?
3) will idf.py build also build the bootloader if the SecureBoot is active; in the docs it says at least that idf.py flash will not write the bootloader (if secure boot is active), but for me it is not clear if it should be built, and if it will be signed like the images?
4) Is it normal that even with the secure boot enabled I can actually reflash the bootloader? If I for example flash it again with one encrypted with a different key the board stops working and sends errors like the one above until I flash again with one encrypted with the correct key. The docs say: "Remember this is a one time flash, you can’t change the bootloader after this!." so I thought the the flashing will just fail, not that it won't work afterwards.
5)What happens if I have a board with secure boot active and afterwards I try to enable flashencryption? From my personal exp it seems sw bricked, and the option is not working. If possible, what are the steps, if not may I suggest adding the info in the SecureBoot that if this is done flash encryption can't be activated anymore?
6) can someone pls write the required steps for this config: Secure boot one time only flash with flashencryption enabled in Dev mode.

[ESP_Angus]

Hi vsimionescu,

Thanks for the detailed report, please find answers to your questions:

1) Is this normal, did I misunderstand the auto encrypt with a self generated key process

The initial encryption process can take a long time (tens of seconds to maybe a minute for a very big firmware). The bootloader does the following in sequence (roughly):

1. Sets up the initial keys, flash encryption config in efuse
2. Encrypts all data in place
3. Burns FLASH_CRYPT_CNT to 1 to enable encryption
4. Enable Secure Boot (if configured)

If the board is reset (for example, by "idf.py monitor" before Step 2 shown here has completed then it will fail to boot, as the bootloader is partially or fully encrypted but the encryption engine isn't actually enabled.

The most common way around this is to re-flash with plaintext firmware again, as step (1) should automatically be skipped the second time. Steps 2 & 3 will run again. Enabling flash encryption manually and then flashing pre-encrypted firmware in Development mode, as described, should also be OK.

Will add a warning about this in the Possible Failures section of the docs.

2) Can the fullclean be easily avoided?

It should be avoidable, this sounds like a bug. You don't happen to recall the exact error that the build system reported?

3) will idf.py build also build the bootloader if the SecureBoot is active; in the docs it says at least that idf.py flash will not write the bootloader (if secure boot is active), but for me it is not clear if it should be built, and if it will be signed like the images?

If Secure Boot is on, the bootloader is still built by "idf.py build" but is not flashed by "idf.py flash". You can build the app by itself with "idf.py app" or the bootloader by itself with "idf.py bootloader", or build and flash the bootloader by itself (even with Secure Boot on) with "idf.py bootloader-flash".

For Secure Boot V1, the bootloader is not signed with an ECDSA signature the same way that the apps are. The bootloader is protected by a digest calculated during the first boot, using a secret key that's stored in efuse and read/write protected from software. This is the Secure Bootloader Key mentioned in the docs and it's different from the ECDSA signing key.

By the way, this is something I noticed that was unexpected in the "espefuse.py summary" posted - the Secure Boot key has been burned to efuse but is not read protected. If the secure boot key was generated by the bootloader on first boot, it should be read protected at the same time or Secure Boot V1 will not work effectively. Did you burn this key to efuse manually as well as the flash encryption key?

4) Is it normal that even with the secure boot enabled I can actually reflash the bootloader? If I for example flash it again with one encrypted with a different key the board stops working and sends errors like the one above until I flash again with one encrypted with the correct key. The docs say: "Remember this is a one time flash, you can’t change the bootloader after this!." so I thought the the flashing will just fail, not that it won't work afterwards.

Yes, this is expected - the flashing tool doesn't have a way to verify whether what you're flashing is going to be accepted or rejected by the Secure Boot configuration that's internal to the chip.

Secure Boot works by having the ROM verify the bootloader image is valid at boot time, and the bootloader verifies the app is signed correctly.

The verification of the bootloader works by generating a digest on first boot (stored at offset 0x0 in the flash) which is based on the efuse key and the bootloader data. Then the ROM compares this digest to the one calculated from the key and the bootloader data on each subsequent boot. Reflashing the bootloader without being able to generate a new digest will cause failure to boot, but if you have a copy of the Secure Boot key then you can generate a new matching digest and flash this as well ("Reflashable mode").

5)What happens if I have a board with secure boot active and afterwards I try to enable flashencryption? From my personal exp it seems sw bricked, and the option is not working. If possible, what are the steps, if not may I suggest adding the info in the SecureBoot that if this is done flash encryption can't be activated anymore?

We don't recommend using Secure Boot without Flash Encryption under any circumstances, it's not a secure configuration. We can make this clearer in the documentation, though.

The issue with enabling flash encryption later on is the same as described in the previous answer. If you have a copy of the Secure Bootloader Key stored in the eFuse then it's possible to generate a new digest to match the new bootloader, and then burn both the digest and the bootloader.

In your case it seems like you do have both keys (you mention that the flash encryption key was generated on the host, and the Secure Boot key seems to marked readable in efuse so it's available somewhere as well). So it would be possible to build the bootloader, generate the bootloader digest, flash the bootloader digest, and then boot this bootloader to enable flash encryption.

6) can someone pls write the required steps for this config: Secure boot one time only flash with flashencryption enabled in Dev mode.

Assuming a chip with no security efuses burned:

Follow all steps from here up to but not including the "idf.py flash" command, for either the "pre-generated key" or "ESP32 generated key" modes:
https://docs.espressif.com/projects/esp ... pment-mode

Follow steps 1-7 here:
https://docs.espressif.com/projects/esp ... ecure-boot

The flashing steps mentioned here (esptool.py command lines or idf.py flash targets) will automatically append the "--after no_reset" option to stop the chip resetting into the firmware automatically after flashing is done.

When running Step 8, this is the step that will reset the chip and trigger the self-encrypting. I recommend that instead of manually resetting the chip it's better to run "idf.py monitor" at this stage, this will allow you to see the log output from the first boot and capture any output.

Will add some clarifications to the docs about how to integrate these two sets of steps.

[axellin]

The flashing steps mentioned here (esptool.py command lines or idf.py flash targets) will automatically append the "--after no_reset" option to stop the chip resetting into the firmware automatically after flashing is done.

Hi,

Is it possible to avoid the automatically append the "--after no_reset" option?

I have a script to do erase flash first then download images.
1. erase flash
esptool.py -p /dev/ttyUSB0 -b 460800 --before default_reset --after hard_reset --chip esp32 erase_flash

2. esptool.py --chip esp32 -p /dev/ttyUSB0 -b 460800 --before=default_reset --after=hard_reset write_flash ....

But every time after erase flash is done, I have to press RST button.
Is it possible to make device reboot immediately after erase flash?

[vsimionescu]

Hello ESP_Angus,

First of all thank you for the very detailed answer.

I will try to also bring some further clarifications to you answer and potential optimization suggestions.

The initial encryption process can take a long time (tens of seconds to maybe a minute for a very big firmware). The bootloader does the following in sequence (roughly):

1. Sets up the initial keys, flash encryption config in efuse
2. Encrypts all data in place
3. Burns FLASH_CRYPT_CNT to 1 to enable encryption
4. Enable Secure Boot (if configured)

I am doing an automated script that is responsible for both flashing the ESP32 and another uC on my device and it is a bit problematic that I don't easily know when the encryption process has finished for the script to go to the next step. I guess the only option would be to continuously read the serial and wait for some message observed at first boot; as a suggestion it would be very nice if you could further develop one of the PY scripts that you already have to directly do this initial encryption.

For now I will just put the flags and flash the pre-encrypted images my self as it seems easier and according to your reply, it is not a hack

2) Can the fullclean be easily avoided?
It should be avoidable, this sounds like a bug. You don't happen to recall the exact error that the build system reported?

I was actually running my initial flash script on multiple boards and playing around with re-flashing them and I observed (at least twice, didn't test further tbh) that if I just do another build (without the clean) for a different board than before (different KEY for the secureboot) and just update the KEY file with the key of the new board the SW was not working. I could imagine that in the build process that is not noticed as a change and as such they are not signed again (the code itself was not changed, only the key). Hope this makes it a bit more clear. Related to the error, it was not a build error, it was the BL not being able to work; the same thing as when you would flash a wrongly signed bootloader while having secure boot active.

By the way, this is something I noticed that was unexpected in the "espefuse.py summary" posted - the Secure Boot key has been burned to efuse but is not read protected. If the secure boot key was generated by the bootloader on first boot, it should be read protected at the same time or Secure Boot V1 will not work effectively. Did you burn this key to efuse manually as well as the flash encryption key?

This is something quite easily reproduced: Just enable Secure bootloader (one time only in my case) and the flash encryption (developer mode). For the secure boot I have made my signing certificate with openSSL(for signing, not the one stored in BLK1, so I guess it is irelevant) and for the flash encryption I used the key generator provided by you. I built the images, and using esptool flashed the bootloader and all the other (app, partition table etc etc) together.Probably I was not patient enough, waiting for ~1min and reset the board. In doing so the key is left readable as you observed in the fuse report. After I did burn the CNT fuse and flashed the encrypted images all was good and the SecureBoot key became hidden. But before that you can just read it even if generated by the device.

Yes, this is expected - the flashing tool doesn't have a way to verify whether what you're flashing is going to be accepted or rejected by the Secure Boot configuration that's internal to the chip.

Ah, ok I see. I was expecting that the location of the BL will somehow be stored and when the flash scripts start that the ROM would be interrogated and not accept to write there again (not sure how the flash scripts do the writing so maybe I am completely off )

5)What happens if I have a board with secure boot active and afterwards I try to enable flashencryption? From my personal exp it seems sw bricked, and the option is not working. If possible, what are the steps, if not may I suggest adding the info in the SecureBoot that if this is done flash encryption can't be activated anymore?

We don't recommend using Secure Boot without Flash Encryption under any circumstances, it's not a secure configuration. We can make this clearer in the documentation, though.

When developing my scripts for some reason I choose SecureBoot first and it was not clear for me that afterwards I won;t be able to also flash encrypt (actually I had a hunch based on the "one time only flashing of the BL, but I was thinking maybe it would allow at least for the apps to be encrypted still and not the BL). But yes agreed both are needed but that was the history.

Follow steps 1-7 here:
https://docs.espressif.com/projects/esp ... ecure-boot

The flashing steps mentioned here (esptool.py command lines or idf.py flash targets) will automatically append the "--after no_reset" option to stop the chip resetting into the firmware automatically after flashing is done.

The last steps 1-7 say:

6)When you’re ready to flash the bootloader, run the specified command (you have to enter it yourself, this step is not performed by make) and then wait for flashing to complete. Remember this is a one time flash, you can’t change the bootloader after this!.

7)Run idf.py flash to build and flash the partition table and the just-built app image. The app image will be signed using the signing key you generated in step 4.

Is there a problem if instead of first flashing the BL and than the images, in 2 steps , I just flash all with the esptool in one go (with no reset after)?

Thank you again for the support and cool product. (I have been using a different one before stumbling upon the ESP and I can say my life became much easier )

[ESP_Angus]

Hi axellin,

Are you passing "--after no_reset" or "--after herd_reset"? It wasn't clear from your post.

Hard reset relies on the RTS pin of the serial interface being wired to the EN pin of the ESP32. If you don't have this then hard_reset is basically equivalent to no_reset.

Unfortunately esptool.py doesn't yet have a supported soft_reset feature on ESP32, to trigger a reset without the external connection.

[axellin]

Hi axellin,

Are you passing "--after no_reset" or "--after herd_reset"? It wasn't clear from your post.

Hard reset relies on the RTS pin of the serial interface being wired to the EN pin of the ESP32. If you don't have this then hard_reset is basically equivalent to no_reset.

Unfortunately esptool.py doesn't yet have a supported soft_reset feature on ESP32, to trigger a reset without the external connection.

Hi @ESP_Angus,
Just checked with our hardware guys and confirm It's h/w issue on my board, not IDF issue.
Thanks.