Rebuild flash encryption/decryption in software

[Dork507]

hello,

is it possible to build the flash encryption in software with the given aes-library?

Background:
I would like to send a file encrypted with the existing

Code: Select all

espsecure.py encrypt_flash_data

function and decrypt it on the ESP32.

What have I configure to achieve that?

Following my testcode:

Code: Select all

#include <Arduino.h>
#include "mbedtls/aes.h"

mbedtls_aes_context aes;

unsigned char key[32] ={
0x12, 0xA1, 0x38, 0xEB, 0xF5, 0x01, 0xD7, 0x89, 0x4C, 0x98, 0x12, 0x4E, 0x47, 0x88, 0xE9, 0xB5, 
0x79, 0x16, 0xF7, 0x7C, 0x82, 0x95, 0xBB, 0xD2, 0x14, 0x84, 0xE4, 0x38, 0xF4, 0x6C, 0x82, 0x9E
};
unsigned char iv[16]={0};
size_t iv_off=0;
unsigned char input [16]={"this is my test"};
unsigned char output[16];

size_t input_len = sizeof(input);
size_t output_len = 0;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);

  esp_aes_init(&aes);
  esp_aes_setkey( &aes, key, 256 );
  esp_aes_crypt_ecb( &aes, ESP_AES_ENCRYPT, input, output );


  Serial.println(String(output, 16));
}

void loop() {
}

[Dork507]

The main questions are, used the flash encryption the ecb mode or the cbc mode? When cbc, what is the initial vector iv?

[ESP_igrr]

The algorithm is different depending on the chip. ESP32 uses the ECB mode with key tweak, details are at https://docs.espressif.com/projects/esp ... -algorithm
ESP32-S2, ESP32-C3 and newer chips use XTS: https://docs.espressif.com/projects/esp ... -algorithm

[Dork507]

Thanks for your answer.
For my purposes I don't need the tweak in the Flash encrypting. And tried to adapt the algorithm of the Pythonscript.

I am despairing.
I'm just trying to encrypt a file with Python and with the ESP32 and expect the same result.
No matter if I use ECB or CBC mode. The result doesn't match. I don't see what I'm doing wrong.

Can someone help me?

Code: Select all

#include <Arduino.h>
#include "mbedtls/aes.h"

mbedtls_aes_context aes;

unsigned char key[32] ={0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x00,0x01,0x02 };
unsigned char iv[16]={0};
size_t iv_off=0;
unsigned char input [16]={"this is my test"};
unsigned char output[32];

size_t output_len = 0;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);

  Serial.println(String(key, 32)); 

  esp_aes_init(&aes);
  esp_aes_setkey( &aes, key, 256 );
  /* 
  esp_aes_crypt_ecb( &aes, ESP_AES_ENCRYPT, input, output );
  esp_aes_crypt_ecb( &aes, ESP_AES_DECRYPT, &input[16], &output[16] );
 */
  esp_aes_crypt_cbc(&aes, ESP_AES_ENCRYPT, sizeof(input), iv, input, output);

  Serial.println(String(output, sizeof(output)));
}

Python:

Code: Select all

def encrypt(output_file, input_file, keyfile):
    do_decrypt=False

    key = keyfile.read()
 
    backend = default_backend()
    iv=[0, 0, 0 , 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    iv= bytes(iv)

    cipher = None
    block_offs = 0x00
    while True:
        block = input_file.read(16)
        

        if len(block) == 0:
            break
        elif len(block) < 16:
            if do_decrypt:
                raise esptool.FatalError("Data length is not a multiple of 16 bytes")
            pad = 16 - len(block)
            block = block + os.urandom(pad)
            print("Note: Padding with %d bytes of random data (encrypted data must be multiple of 16 bytes long)" % pad)

        if block_offs % 32 == 0 or cipher is None:
            # each bit of the flash encryption key is XORed with tweak bits derived from the offset of 32 byte block of flash
            #block_key = _flash_encryption_tweak_key(key, block_offs, tweak_range)
            #block_key = key
            test=key
            print("key: %s" % test)
            print("Decrypt?: %s" % do_decrypt)

            if cipher is None:  # first pass
                cipher = Cipher(algorithms.AES(key), modes.CBC(iv), backend)

                # note AES is used inverted for flash encryption, so
                # "decrypting" flash uses AES encrypt algorithm and vice
                # versa. (This does not weaken AES.)
                actor = cipher.encryptor() if do_decrypt else cipher.decryptor()
            
        output = actor.update(block) + actor.finalize()

        #output_file.write(block[::-1])  # reverse output block byte order
        #print("Code: %s" % output)
        output_file.write(output) 
        block_offs += 16

key.txt

(32 Bytes) Downloaded 196 times

plaintext.txt

(16 Bytes) Downloaded 169 times

[ESP_Sprite]

For my purposes I don't need the tweak in the Flash encrypting.

No matter if I use ECB or CBC mode. The result doesn't match. I don't see what I'm doing wrong.

Regardless of if you need or do not need the tweak, the ESP32 is going to use it anyway. If you do not use it, you will en/decrypt with the wrong AES key and you won't get the same results.

[ESP_igrr]

Could you describe what you are trying to achieve, overall? It will help us understand why using the existing tools is not an option.

[Dork507]

How I told. I want to send an encrypted file to the ESP over html (no https) or the user downloads its himself, like a firmware.bin.
The firmware shall nobody use on foreign Hardware. Only my first installed Software has the key.


What ever. I found my solution:

Implementing this "tweak" in software was too time-consuming for me now.
Therefore I created an encrypt.py based on the espsecure.py file. It generates an encrypted file using AES256-CBC encryption. To do this, an other "crypto" library had to be integrated.
This procedure is consistent with the #include "mbedtls/aes.h" library.

The original library in espsecure.py was incompatible. I had never been able to generate the same values.

encrypt.py.txt

Python File to encrypt Files

(4.42 KiB) Downloaded 410 times

The fitting update Code for the esp32 on Arduino-Framework.

Code: Select all

void handlefirmwareUpload_encrypt(AsyncWebServerRequest *request, String filename, size_t index, uint8_t *data, size_t len, bool final)
{
    static uint8_t* overlape[16]={0}; //Überstand kann maximal 15 sein
    static size_t last_overlape_len=0;     
    uint8_t encrypted_data[2000]={0};
    uint8_t decrypted_data[2000]={0};
    int error_decrypt=0;
      
    if(!index){
      Serial.printf("Update Start: %s\n", filename.c_str());
      if(!Update.begin(0xFFFFFFFF)){
        Update.printError(Serial);
      }
    last_overlape_len=0; //Überlänge beim Start noch NULL
    }

    size_t data_len = len+last_overlape_len; //Länge der encryptbaren Daten
    size_t data_len16 = data_len -  (data_len % 16); //Länge auf 16Byte
    if(encrypted_data){
    memcpy(encrypted_data, overlape, last_overlape_len);//alten Überstand in neue Daten integrieren
    memcpy(encrypted_data+last_overlape_len,data,data_len);//neue Daten hinzufügen
    }

    last_overlape_len = data_len - data_len16; //neu Überlapplänge berechnen
    memcpy(overlape, encrypted_data+data_len16, last_overlape_len);//Überstand mit in neuen Aufruf übernehmen

    if (decrypted_data){
     error_decrypt = esp_aes_crypt_cbc(&aes, ESP_AES_DECRYPT, data_len16 , iv, encrypted_data, decrypted_data); //Decrypten
    }
    
     if(!Update.hasError() && !error_decrypt){
      Serial.print("-");
        if(Update.write(decrypted_data, data_len16) != data_len16){
          Update.printError(Serial);
        }
      } 

    if(final){
        Serial.printf("\r\nEnde: %u Byte\r\n", index+len);

       if(Update.end(true)){
        Serial.printf("Update Success: %uB\n", index+len);
        shouldReboot = true;


      } else {
        Update.printError(Serial);
      } 
    }
 
        
}

I wanted to use malloc() but a crash occursed. So I used a fix buffer for encrypted and decrypted data.

If flash-encryption enable, the bootloader encrypts with its own key and the software is safe.
I haven't found a comparable solution on the internet yet.

[Dork507]

For my purposes I don't need the tweak in the Flash encrypting.

No matter if I use ECB or CBC mode. The result doesn't match. I don't see what I'm doing wrong.

Regardless of if you need or do not need the tweak, the ESP32 is going to use it anyway. If you do not use it, you will en/decrypt with the wrong AES key and you won't get the same results.

I looked for a simple solution.

I don't understand. Found you a mistake on my code? I always uses the same key on both python and esp32.

[ESP_Sprite]

Regardless of if you need or do not need the tweak, the ESP32 is going to use it anyway. If you do not use it, you will en/decrypt with the wrong AES key and you won't get the same results.

I looked for a simple solution.

I don't understand. Found you a mistake on my code? I always uses the same key on both python and esp32.

The thing is: you seem to feed the flash key directly to the AES encryption. The ESP32 does not do that: instead it applies a tweak (=modification) to the key based on the address before feeding the key to the AES encryption. In other words: the sectors are encrypted with a *modified* flash encryption key. If your program doesn't also do that modification, the keys used for AES encryption are going to be different and the ESP32 won't be able to decode the flash.

[boarchuz]

How I told. I want to send an encrypted file to the ESP over html (no https) or the user downloads its himself, like a firmware.bin.
The firmware shall nobody use on foreign Hardware. Only my first installed Software has the key.

The title is confusing, then. It sounded like you wanted to implement the exact same encryption as ESP32's flash encryption, so that the resulting binary can be written directly (raw) to the correct flash offset as it is received.

It's now apparent that what you actually want is to encrypt the binary for the purposes of secure transport. You can use whatever encryption you like, since you're decrypting the incoming data before passing it along to Update.write anyway (which will then encrypt as it writes to flash, if enabled).

There's an example here: https://github.com/espressif/esp-idf/tr ... rypted_ota