Secure boot + flash encryption + Arduino BIN file

[GreenEyedExplorer]

Hi. I've come to problem.

I have the latest combination of ESP-IDF (ESP-IDF Pre-release v3.2-beta3), MYSYS and arduino libraries -all setup working very well on Windows OS. All installed on 1st of march, 2019. I also use latest version of esptool 2.6.
I have made an ESP32 to run a fully operational app-ota example with secure boot + flash encryption. I have tried both "One-time Flash" and "Reflashable" secure boot variants. Both successfully.
Steps I've taken:
1. created .PEM file:

Code: Select all

espsecure.py generate_signing_key secure_boot_signing_key.pem

2. got .BIN from .PEM:

Code: Select all

espsecure.py digest_private_key --keyfile secure_boot_signing_key.pem --keylen 256 my_flash_encryption_key.bin

3. make menuconfig -> checked both boot and ecryption options; selected my partition table; +reduction of bootloader logging to only ERROR (0x7000 length limit - see partition).
sdkconfig:

Code: Select all

# Security features
CONFIG_SECURE_SIGNED_ON_BOOT=y
CONFIG_SECURE_SIGNED_ON_UPDATE=y
CONFIG_SECURE_SIGNED_APPS=y
CONFIG_SECURE_BOOT_ENABLED=y
CONFIG_SECURE_BOOTLOADER_ONE_TIME_FLASH=y
CONFIG_SECURE_BOOTLOADER_REFLASHABLE=
CONFIG_SECURE_BOOT_BUILD_SIGNED_BINARIES=y
CONFIG_SECURE_BOOT_SIGNING_KEY="secure_boot_signing_key.pem"
CONFIG_SECURE_BOOT_INSECURE=
CONFIG_FLASH_ENCRYPTION_ENABLED=y
CONFIG_FLASH_ENCRYPTION_INSECURE=

table:

Code: Select all

nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x140000,
app1,     app,  ota_1,   0x150000,0x140000,
eeprom,   data, 0x99,    0x290000,0x1000,
storage,  data, spiffs,  0x291000,0xA0000,

4. make bootloader
5.

Code: Select all

esptool.py --chip esp32 --port COM15 --baud 921600 --before default_reset --after no_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0x1000 bootloader.bin

6. make
7.

Code: Select all

esptool.py --chip esp32 --port COM15 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0xe000 ota_data_initial.bin 0x10000 app-ota-template.bin 0x8000 partitions.bin

8. WAIT 1 MINUTE before further action due to encryption.

This works all great.
Some more Arduino similarities: I use Arduino IDE for programming and to make .BIN files. I use the same partition table. The normal command for flashing Arduino without any security would look like:

Code: Select all

esptool.exe --chip esp32 --port COM13 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0xe000 partitions/boot_app0.bin 0x1000 tools/sdk/bin/bootloader_dio_80m.bin 0x10000 ArdBLA.ino.bin 0x8000 ArdBLA.ino.partitions.bin

this is almost the same as for ESP-IDF way above ().

The problem I have is, if I use my ArdBLA.ino.bin file at the step 7 for flashing at 0x10000 the program does not work.
No luck with both untouched(=unencrypted) and pre-encyrpted with command:

Code: Select all

espsecure.py encrypt_flash_data --keyfile my_flash_encryption_key.bin --address 0x10000 -o ArdBLA_encoded.bin ArdBLA.ino.bin
No luck.

I've run out of ideas. Any help would be appreciated on how to put Arduino .bin file onto ESP32 (using secure boot and flash encryption).

Rok Rodic alias GreenEyedExplorer
https://www.youtube.com/channel/UCIOIhh ... LB0azJmd8w
http://www.rodic.si

Some references I've used along the way:
https://docs.espressif.com/projects/esp ... t-started/
https://github.com/espressif/arduino-es ... mponent.md
https://docs.espressif.com/projects/esp ... -boot.html
https://docs.espressif.com/projects/esp ... ption.html
https://docs.espressif.com/projects/esp ... ables.html
https://github.com/espressif/esptool/wi ... read-error
http://bbs.esp32.com/viewtopic.php?t=6457
https://www.esp32.com/viewtopic.php?t=4994
https://github.com/espressif/arduino-esp32/issues/1387

[ESP_Angus]

Hi GreenEyedExplorer,

The short answer is that unfortunately Flash Encryption and Secure Boot are not supported with the Arduino IDE. You should be able to convert the .ino file to a .cpp file and use it in an ESP-IDF project with the configuration set as you describe.

The longer answer is that the Arduino IDE uses precompiled IDF libraries, and there are parts of IDF which are simply not compiled in when the flash encryption and signature verification options are disabled. Even if you managed to get an IDF bootloader configured with these features to boot a .bin file compiled from Arduino, certain features - like signature verification and support for encrypted partitions - won't be compiled into the app at all, so it would either be insecure (no signature verification on OTA) or simply broken (can't read certain partition types correctly).

[GreenEyedExplorer]

Hi. Thank you very much for your answer. I was hoping it would be easier.

Another question. Is there a way to make Arduino Serial.print and company of Serials work in ESP-IDF?

Rok Rodic alias GreenEyedExplorer
https://www.youtube.com/channel/UCIOIhh ... LB0azJmd8w
http://www.rodic.si

[ESP_Angus]

Another question. Is there a way to make Arduino Serial.print and company of Serials work in ESP-IDF?

It would be best to post this as a new thread in the Arduino forum but the short answer is yes. Arduino automatically generates #include lines for sketches but if you #include "HardwareSerial.h" in your .cpp file, you should be able to use these (provided Arduino component is added to the IDF project).

[GreenEyedExplorer]

Thank you ESP_Angus! Your answer helped a lot.

[GreenEyedExplorer]

Hi, ESP_Angus!
I have ported my Arduino code under ESP-IDF. I have recompiled Arduino, so it supports encryption under ESP-IDF. It calls those spi_flash_write_encrypted() and spi_flash_read_encrypted() now. So, when I enable one-time boot flashing and flash encryption the code works. Even EEPROM writes and SPIFFS both called via Arduino code. So, .
Well, the first flash (steps from 1-8 as described above) goes as described above and it works.
But when I re-run the make command and afterwards run (as the make command suggests):

Code: Select all

esptool.py --chip esp32 --port COM16 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0xe000 build/ota_data_initial.bin 0x10000 build/prog.bin 0x8000 build/partitions.bin 0x291000 build/storage.bin

...the program starts displaying:

Code: Select all

rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1012
load:0x40078000,len:13836
load:0x40080400,len:5692
entry 0x40080674

indefinately.
In the build folder there are two versions of prog.bin and partitions.bin. The other ones are prog-unsigned.bin and partitions-unsigned.bin. ota_data_initial.bin and storage.bin have none of unsigned variants.

ESPEFUSE reports:

Code: Select all

espefuse.py v2.6
Connecting....
Security fuses:
FLASH_CRYPT_CNT        Flash encryption mode counter                     = 1 R/W (0x1)
FLASH_CRYPT_CONFIG     Flash encryption config (key tweak bits)          = 15 R/W (0xf)
CONSOLE_DEBUG_DISABLE  Disable ROM BASIC interpreter fallback            = 1 R/W (0x1)
ABS_DONE_0             secure boot enabled for bootloader                = 1 R/W (0x1)
ABS_DONE_1             secure boot abstract 1 locked                     = 0 R/W (0x0)
JTAG_DISABLE           Disable JTAG                                      = 1 R/W (0x1)
DISABLE_DL_ENCRYPT     Disable flash encryption in UART bootloader       = 1 R/W (0x1)
DISABLE_DL_DECRYPT     Disable flash decryption in UART bootloader       = 1 R/W (0x1)
DISABLE_DL_CACHE       Disable flash cache in UART bootloader            = 1 R/W (0x1)
BLK1                   Flash encryption key
  = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -/-
BLK2                   Secure boot key
  = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -/-
BLK3                   Variable Block 3
  = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W

Efuse fuses:
WR_DIS                 Efuse write disable mask                          = 384 R/W (0x180)
RD_DIS                 Efuse read disablemask                            = 3 R/W (0x3)
CODING_SCHEME          Efuse variable block length scheme                = 0 R/W (0x0)
KEY_STATUS             Usage of efuse block 3 (reserved)                 = 0 R/W (0x0)

Config fuses:
XPD_SDIO_FORCE         Ignore MTDI pin (GPIO12) for VDD_SDIO on reset    = 0 R/W (0x0)
XPD_SDIO_REG           If XPD_SDIO_FORCE, enable VDD_SDIO reg on reset   = 0 R/W (0x0)
XPD_SDIO_TIEH          If XPD_SDIO_FORCE & XPD_SDIO_REG, 1=3.3V 0=1.8V   = 0 R/W (0x0)
SPI_PAD_CONFIG_CLK     Override SD_CLK pad (GPIO6/SPICLK)                = 0 R/W (0x0)
SPI_PAD_CONFIG_Q       Override SD_DATA_0 pad (GPIO7/SPIQ)               = 0 R/W (0x0)
SPI_PAD_CONFIG_D       Override SD_DATA_1 pad (GPIO8/SPID)               = 0 R/W (0x0)
SPI_PAD_CONFIG_HD      Override SD_DATA_2 pad (GPIO9/SPIHD)              = 0 R/W (0x0)
SPI_PAD_CONFIG_CS0     Override SD_CMD pad (GPIO11/SPICS0)               = 0 R/W (0x0)
DISABLE_SDIO_HOST      Disable SDIO host                                 = 0 R/W (0x0)

Identity fuses:
MAC                    MAC Address                                       = 80:7d:3a:c5:22:1c R/W
CHIP_VERSION           Chip version                                      = 10 -/W (0xa)
CHIP_PACKAGE           Chip package identifier                           = 0 -/W (0x0)

Flash voltage (VDD_SDIO) determined by GPIO12 on reset (High for 1.8V, Low/NC for 3.3V).

Any idea where the problem is? I have ruined half a dozen of ESP32s by now...
Is there an encryption key problem? Should I pre-flash the encryption key into ESP32 (i.e. espefuse.py --port COM16 burn_key flash_encryption my_flash_encryption_key.bin) before the bootloader and flash writes?

Thank you very much!!!

Rok Rodic alias GreenEyedExplorer
http://www.rodic.si

[WiFive]

The whole point of secure boot and encryption is to prevent reflashing. It is meant to be used with ota only. The way around this is to have a copy of both the flash encryption key and the secure boot signing key and to encrypt the signed binaries before flashing them. Although you can also disable encryption up to 3 times if you choose reflashable mode so you may be able to recover your modules. Study the docs.

[GreenEyedExplorer]

Hi WiFive, thank you for your time answering my question.
I have studied documentation and tons of furums. I know I have to preencypt my .bins (https://docs.espressif.com/projects/esp ... yption-key) to be able to flash via serial port (3rd dot at: https://docs.espressif.com/projects/esp ... ecure-boot). And it is not working? Why? I have stated I have my encoding keys and they are imported via make menuconfig... Any other idea?
So my questions remain: Why is it not working? Any idea where the problem is? Is there an encryption key problem? Should I manually pre-flash the encryption key into ESP32 (i.e. espefuse.py --port COM16 burn_key flash_encryption my_flash_encryption_key.bin) before the bootloader and flash writes?

Rok Rodic alias GreenEyedExplorer
http://www.rodic.si

[ESP_Angus]

Hi GreenEyedExplorer,

Should I manually pre-flash the encryption key into ESP32 (i.e. espefuse.py --port COM16 burn_key flash_encryption my_flash_encryption_key.bin) before the bootloader and flash writes?

Yes, if you plan to reflash with pre-encrypted binaries then you have to burn your flash encryption key onto the ESP32 before you do anything else. See this step under "Reflashing via Pregenerated Flash Encryption Key" instructions).

If the device has completed the "First Boot" with flash encryption enabled and no key pre-burned, it will generate a random flash encryption key and write this into efuse. You won't be able to pre-encrypt any binaries for this device, as you don't have the same key that it's using. However, you can probably rescue the ESP32 by burning FLASH_CRYPT_CNT one more time (to disable flash encryption) and then re-flash it with all plaintext (provided you do have a valid secure boot key, see below). Note that you can't change the flash encryption key, though, so any time flash encryption is on you'll be locked out.

The same general approach applies to Secure Boot if you plan to reflash the bootloader (Reflashable Software Bootloader Instructions). In this case the IDF build system will prompt you for the "esp_efuse.py burn_key ..." command to run, to burn the secure boot key before the first boot.

It's critically important that both these keys are manually burned to efuse before the first boot (ideally, before the first flash - to avoid the possibility of accidental booting), otherwise random keys will be generated and used.

[r_george]

Being a novice one simple question.

is it possible to have
Secure boot + flash encryption + Arduino BIN file