Flashing plain text firmware after encryption while development

[vinodstanur]

Hi,

I am trying the flash encryption now. I am not using secure boot for now.

I am using esp-idf 3.3 (latest stable) in linux.

In menuconfig -> Security features I enabled Enable flash encryption on boot (READ DOCS FIRST)

Since I am developing I entered into Potentially insecure options ---> and enabled

[*] Allow JTAG Debugging
[*] Leave UART bootloader encryption enabled
[*] Leave UART bootloader decryption enabled
[*] Leave UART bootloader flash cache enabled



Now I did make flash and the firmware is downloaded and encrypted by the chip on first boot.
a
Now I am unable to load plain text firmware for development purpose, because I thought if I check [*] Leave UART bootloader encryption enabled , [*] Leave UART bootloader decryption enabled and [*] Leave UART bootloader flash cache enabled then I can force the esp32 to encrypt and save the bootloader firmware loaded via serial bootloader. But this is not the case. Chip is on bootloop reset after doing subsequent "make flash".

Code: Select all

[15:21:05:366] rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)␍␊
[15:21:05:366] flash read err, 1000␍␊
[15:21:05:366] ets_main.c 371 ␍␊
[15:21:05:762] ets Jun  8 2016 00:22:57␍␊

For development, I can insert pre-defined 32bit encryption key, but in this case I haven't done that by assuming that if I prevent DISABLE_DL_ENCRYPT, DISABLE_DL_DECRYPT and DISABLE_DL_CACHE to be burned '1'.

So what exactly is the function of these bits ?


My espefuse.py summary is as below.

Code: Select all

espefuse.py v2.7-dev
Connecting........_
EFUSE_NAME             Description = [Meaningful Value] [Readable/Writeable] (Hex Value)
----------------------------------------------------------------------------------------
Security fuses:
FLASH_CRYPT_CNT        Flash encryption mode counter                     = 7 R/W (0x7)
FLASH_CRYPT_CONFIG     Flash encryption config (key tweak bits)          = 15 R/W (0xf)
CONSOLE_DEBUG_DISABLE  Disable ROM BASIC interpreter fallback            = 1 R/W (0x1)
ABS_DONE_0             secure boot enabled for bootloader                = 0 R/W (0x0)
ABS_DONE_1             secure boot abstract 1 locked                     = 0 R/W (0x0)
JTAG_DISABLE           Disable JTAG                                      = 0 R/W (0x0)
DISABLE_DL_ENCRYPT     Disable flash encryption in UART bootloader       = 0 R/W (0x0)
DISABLE_DL_DECRYPT     Disable flash decryption in UART bootloader       = 0 R/W (0x0)
DISABLE_DL_CACHE       Disable flash cache in UART bootloader            = 0 R/W (0x0)
BLK1                   Flash encryption key                              
  = ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? -/- 
BLK2                   Secure boot key                                   
  = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 
BLK3                   Variable Block 3                                  
  = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 

Efuse fuses:
WR_DIS                 Efuse write disable mask                          = 128 R/W (0x80)
RD_DIS                 Efuse read disablemask                            = 1 R/W (0x1)
CODING_SCHEME          Efuse variable block length scheme                = 0 R/W (0x0)
KEY_STATUS             Usage of efuse block 3 (reserved)                 = 0 R/W (0x0)

Config fuses:
XPD_SDIO_FORCE         Ignore MTDI pin (GPIO12) for VDD_SDIO on reset    = 0 R/W (0x0)
XPD_SDIO_REG           If XPD_SDIO_FORCE, enable VDD_SDIO reg on reset   = 0 R/W (0x0)
XPD_SDIO_TIEH          If XPD_SDIO_FORCE & XPD_SDIO_REG, 1=3.3V 0=1.8V   = 0 R/W (0x0)
CLK8M_FREQ             8MHz clock freq override                          = 54 R/W (0x36)
SPI_PAD_CONFIG_CLK     Override SD_CLK pad (GPIO6/SPICLK)                = 0 R/W (0x0)
SPI_PAD_CONFIG_Q       Override SD_DATA_0 pad (GPIO7/SPIQ)               = 0 R/W (0x0)
SPI_PAD_CONFIG_D       Override SD_DATA_1 pad (GPIO8/SPID)               = 0 R/W (0x0)
SPI_PAD_CONFIG_HD      Override SD_DATA_2 pad (GPIO9/SPIHD)              = 0 R/W (0x0)
SPI_PAD_CONFIG_CS0     Override SD_CMD pad (GPIO11/SPICS0)               = 0 R/W (0x0)
DISABLE_SDIO_HOST      Disable SDIO host                                 = 0 R/W (0x0)

Identity fuses:
MAC                    Factory MAC Address                               
  = 39:54:a4:c3:65:1c (CRC e8 OK) R/W 
CHIP_VER_REV1          Silicon Revision 1                                = 0 R/W (0x0)
CHIP_VERSION           Reserved for future chip versions                 = 0 R/W (0x0)
CHIP_PACKAGE           Chip package identifier                           = 0 R/W (0x0)

Calibration fuses:
BLK3_PART_RESERVE [Codebox=bash file=Untitled.bsh][/Codebox]     BLOCK3 partially served for ADC calibration data  = 0 R/W (0x0)
ADC_VREF               Voltage reference calibration                     = 1100 R/W (0x0)

Flash voltage (VDD_SDIO) determined by GPIO12 on reset (High for 1.8V, Low/NC for 3.3V).

[WiFive]

At this stage if user wants to update modified plaintext application image to flash in encrypted format it can be done using following command:

Code: Select all

idf.py encrypted-app-flash monitor

If all partitions needs to be updated in encrypted format, it can be done as

Code: Select all

idf.py encrypted-flash monitor

https://docs.espressif.com/projects/esp ... ption.html

[WiFive]

Try just adding --encrypt option to your esptool.py command.

[vinodstanur]

At this stage if user wants to update modified plaintext application image to flash in encrypted format it can be done using following command:

Code: Select all

idf.py encrypted-app-flash monitor

If all partitions needs to be updated in encrypted format, it can be done as

Code: Select all

idf.py encrypted-flash monitor

https://docs.espressif.com/projects/esp ... ption.html

Thanks for the reply. But it is showing that option is not available.

idf.py: error: argument actions: invalid choice: 'encrypted-flash'

Code: [Select all] [Expand/Collapse]

1. Note: You are using Python 3.6.5. Python 3 support is new, please report any problems you encounter. Search for 'Setting the Python Interpreter' in the ESP-IDF docs if you want to use Python 2.7.
2. usage: idf.py [-h] [-p PORT] [-b BAUD] [-C PROJECT_DIR] [-B BUILD_DIR]
3.               [-G {Ninja,Unix Makefiles}] [-n] [-v]
4.               [-D DEFINE_CACHE_ENTRY [DEFINE_CACHE_ENTRY ...]] [--no-ccache]
5.               {all,build,clean,fullclean,reconfigure,menuconfig,defconfig,confserver,size,size-components,size-files,bootloader,bootloader-clean,bootloader-flash,app,app-flash,efuse_common_table,efuse_custom_table,show_efuse_table,partition_table,partition_table-flash,flash,erase_flash,monitor,erase_otadata,read_otadata}
6.               [{all,build,clean,fullclean,reconfigure,menuconfig,defconfig,confserver,size,size-components,size-files,bootloader,bootloader-clean,bootloader-flash,app,app-flash,efuse_common_table,efuse_custom_table,show_efuse_table,partition_table,partition_table-flash,flash,erase_flash,monitor,erase_otadata,read_otadata} ...]
7. idf.py: error: argument actions: invalid choice: 'encrypted-flash' (choose from 'all', 'build', 'clean', 'fullclean', 'reconfigure', 'menuconfig', 'defconfig', 'confserver', 'size', 'size-components', 'size-files', 'bootloader', 'bootloader-clean', 'bootloader-flash', 'app', 'app-flash', 'efuse_common_table', 'efuse_custom_table', 'show_efuse_table', 'partition_table', 'partition_table-flash', 'flash', 'erase_flash', 'monitor', 'erase_otadata', 'read_otadata')

GeSHi © Codebox Plus Extension

[vinodstanur]

Try just adding --encrypt option to your esptool.py command.

Thanks it worked!!

I have few questions and doubts about this overall setup. I am typing those as point by point below.
-------------------------------------
1. That means if I keep this JTAG_DISABLE, DISABLE_DL_ENCRYPT, DISABLE_DL_ENCRYPT and DISABLE_DL_CACHE backdoors opened (0), I can serial boot multiple number of times while development, am I right ?
-------------------------------------

2. Is this feature an alternative to user defined flash encryption key setup ?
-------------------------------------

3. Also if I enabled those bits in the production sample, we won't be able to flash/read the firmware via any serial wire or Jtag, am I right ?
-------------------------------------

4. For production, how will I force the FLASH_CRYPT_CNT to 0xff in one shot ?
-------------------------------------

5. If I know the flash encryption key in the device, then there is no way to prevent me to load the encrypted binary to the chip via serial bootloader ? Is this right ? Is this assumption correct ?
-------------------------------------

6. Basically the only option is to disable the serial bootloader flash write and read encryption so that even if we write/read, it won't execute because it will be a corrupted bin while executing / encrypted bin respectively. Am I right?

------------------------------------

7. My OTA bootloader performance becomes almost 4+ x slower after enabling flash encryption. Is this expected?
I am using bluetooth classic for OTA and writing 1024 bytes packets to flash one by one.

[ESP_Angus]

Between IDF v3.3 and v4.x we changed the flash encryption process to support "development mode" and encrypt-while-flashing.

This is why not everything described in the "latest" docs works on v3.3. The v3.3 docs are here:
https://docs.espressif.com/projects/esp ... ption.html

But by passing "--encrypt" to esptool.py and making the config changes you've made, you're essentially recreating the v4.x "development mode" on v3.3. Which should be fine, it's just not documented/supported.

-------------------------------------
1. That means if I keep this JTAG_DISABLE, DISABLE_DL_ENCRYPT, DISABLE_DL_ENCRYPT and DISABLE_DL_CACHE backdoors opened (0), I can serial boot multiple number of times while development, am I right ?
-------------------------------------

Correct. You can flash with "--encrypt" as many times as you need to.

2. Is this feature an alternative to user defined flash encryption key setup ?
-------------------------------------

Yes. If you compare the v3.x docs with the v4.x docs then you'll see this is why we made this change, to createa more flexible alternative.

3. Also if I enabled those bits in the production sample, we won't be able to flash/read the firmware via any serial wire or Jtag, am I right ?
-------------------------------------

Yes. Although please note: https://www.espressif.com/en/news/Secur ... rotections

4. For production, how will I force the FLASH_CRYPT_CNT to 0xff in one shot ?
-------------------------------------

The easiest option is to wait for ESP-IDF v3.3.1 and then enable this config option:
https://docs.espressif.com/projects/esp ... -plaintext

Otherwise you can add this code to app_main(), shown near the bottom of this advisory:
https://www.espressif.com/en/news/Espre ... ecure_Boot

5. If I know the flash encryption key in the device, then there is no way to prevent me to load the encrypted binary to the chip via serial bootloader ? Is this right ? Is this assumption correct ?
-------------------------------------

Correct. Although if Secure Boot is enabled then you will need to also know the secure boot key, or the ESP32 wil refuse to boot the new firmware.

6. Basically the only option is to disable the serial bootloader flash write and read encryption so that even if we write/read, it won't execute because it will be a corrupted bin while executing / encrypted bin respectively. Am I right?

That's right. The idea is that this level of access is the same as if you purchased a SPI flash programmer device and wired it up directly to the flash chip.

7. My OTA bootloader performance becomes almost 4+ x slower after enabling flash encryption. Is this expected?
I am using bluetooth classic for OTA and writing 1024 bytes packets to flash one by one.

Encrypted writes to flash are much slower than plaintext writes. For the best performance, write a buffer whose length is 32 byte aligned to a 32 byte aligned address each time you call the flash API (if you're writing 1024 byte aligned then this will already be the case.)

[vinodstanur]

Between IDF v3.3 and v4.x we changed the flash encryption process to support "development mode" and encrypt-while-flashing.

This is why not everything described in the "latest" docs works on v3.3. The v3.3 docs are here:
https://docs.espressif.com/projects/esp ... ption.html

But by passing "--encrypt" to esptool.py and making the config changes you've made, you're essentially recreating the v4.x "development mode" on v3.3. Which should be fine, it's just not documented/supported.

-------------------------------------
1. That means if I keep this JTAG_DISABLE, DISABLE_DL_ENCRYPT, DISABLE_DL_ENCRYPT and DISABLE_DL_CACHE backdoors opened (0), I can serial boot multiple number of times while development, am I right ?
-------------------------------------

Correct. You can flash with "--encrypt" as many times as you need to.

2. Is this feature an alternative to user defined flash encryption key setup ?
-------------------------------------

Yes. If you compare the v3.x docs with the v4.x docs then you'll see this is why we made this change, to createa more flexible alternative.

3. Also if I enabled those bits in the production sample, we won't be able to flash/read the firmware via any serial wire or Jtag, am I right ?
-------------------------------------

Yes. Although please note: https://www.espressif.com/en/news/Secur ... rotections

4. For production, how will I force the FLASH_CRYPT_CNT to 0xff in one shot ?
-------------------------------------

The easiest option is to wait for ESP-IDF v3.3.1 and then enable this config option:
https://docs.espressif.com/projects/esp ... -plaintext

Otherwise you can add this code to app_main(), shown near the bottom of this advisory:
https://www.espressif.com/en/news/Espre ... ecure_Boot

5. If I know the flash encryption key in the device, then there is no way to prevent me to load the encrypted binary to the chip via serial bootloader ? Is this right ? Is this assumption correct ?
-------------------------------------

Correct. Although if Secure Boot is enabled then you will need to also know the secure boot key, or the ESP32 wil refuse to boot the new firmware.

6. Basically the only option is to disable the serial bootloader flash write and read encryption so that even if we write/read, it won't execute because it will be a corrupted bin while executing / encrypted bin respectively. Am I right?

That's right. The idea is that this level of access is the same as if you purchased a SPI flash programmer device and wired it up directly to the flash chip.

7. My OTA bootloader performance becomes almost 4+ x slower after enabling flash encryption. Is this expected?
I am using bluetooth classic for OTA and writing 1024 bytes packets to flash one by one.

Encrypted writes to flash are much slower than plaintext writes. For the best performance, write a buffer whose length is 32 byte aligned to a 32 byte aligned address each time you call the flash API (if you're writing 1024 byte aligned then this will already be the case.)

Thanks for your detailed reply.

I have few more questions:

1> I am unable to find equivalent ESP32-WROOM-32 modules with latest ESP32-D0WD-V3 chip. Is this chip revision not yet available in this esp32-wroom-32 module ? I am about to purchase the esp32-wroom-32 modules in bulk very soon and I need flash encryption also

2> After I enabled the flash encryption feature on the chip, I observed the overall bluetooth classic SPP throughput got reduced. Earlier I thought the OTA is slow because of flash encryption write of 1024 bytes chunk. But I commented out the flash write and trying my OTA speed, it got improved a bit but still less. So I am assuming the overall flash encryption is affecting the bluetooth SPP speed itself. I am still trying to debug it.

3> If I am using secure boot, I need to sign the binary from PC side before pushing to OTA server. But does it means the signed application binary is encrypted as-well or is like it just append some signature at the end of plain text firmware without encyrption ?

4> I am forced to move my factory (OTA bootloader) to 0x20000 because the bootloader size got increased after enabling flash encryption. So I pushed partition table offset + 0x1000 and pushed nvs, otadata and phy_init offsets accordingly. But I am forced to push factory also but it seems like it needs 64K aligned, so I moved it from 10000 to 20000. So I got a lot more space on top of it, so I increased the size of nvs, phy_init etc to 24K, 16K etc. I hope these wont create any problem or in the other way, it might improve the wear and tear issue as-well. Am I right ? or am I missing anything ?


5> The flash encryption for any write to the flash is hardware accelerated right ? So still will it add a lot of wait state for read and write from and to flash when encrypt is enabled in fuse ?