Ordering problem with flash encryption

[schickb]

Having a problem performing operations with parttool.py after doing encrypted-flash. The error I get is as follows:

Code: Select all

Traceback (most recent call last):
  File "/Users/bschick/esp/esp-idf/components/partition_table/parttool.py", line 365, in <module>
    main()
  File "/Users/bschick/esp/esp-idf/components/partition_table/parttool.py", line 334, in main
    target = ParttoolTarget(**target_args)
  File "/Users/bschick/esp/esp-idf/components/partition_table/parttool.py", line 116, in __init__
    partition_table = gen.PartitionTable.from_binary(f.read())
  File "/Users/bschick/esp/esp-idf/components/partition_table/gen_esp32part.py", line 281, in from_binary
    result.append(PartitionDefinition.from_binary(data))
  File "/Users/bschick/esp/esp-idf/components/partition_table/gen_esp32part.py", line 433, in from_binary
    res.name = res.name.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf4 in position 0: invalid continuation byte

It seems like parttool.py is reading encrypted data and failing since it isn't decrypted before reading. More details:

Code: Select all

% idf.py --version
ESP-IDF v4.4-beta1-177-ge104dd7f27-dirty

MacOSx 12.1

target = esp32c3

I can reproduce this problem using the flash_encryption demo:

• Enable development mode flash encryption withg defaults (secure boot not enabled)
• Did a normal plaintext "idf.py flash", rebooted, went through onchip generated key encryption process
• App runs, everything works (except my user NVS partition is empty)
• Update application code and run "idf.py encrypted-flash"
• Boot and running the app still works fine (but no user NVS partition)
• Run "iparttool.py -o 0x9000 erase_partition --partition-name storage" (or any other parttool command) (get the error above)
• Update just the partition table with "idf.py partition-table-flash" (I assume this write unencrypted part table)
• "parttool.py -o 0x9000 erase_partition --partition-name storage" now works as expected, but application will not boot due to "partition 0 invalid magic number" which I assume happens because the partition table is not encrypted
• If I run "idf.py encrypted-flash" the app starts, but the parttool.py fails with the error above.

So how do I get an encrypted partition table and modify partitions with parttool? Seems like I need partool to work with an encrypted partition table, or I need an "encrypted-partition-table-flash" command so I can write the plaintext version, updates partitions, then write back the encrypted part table".

Thanks for suggestions.

[schickb]

I solved this problem. Two realizations got me there:

* I discovered the --partition-table-file option for parttool.py and similar commands. That let me update the flash even when the partition table is encrypted.

* Second, I realized that even though my partition table file lists the nvs_key partition as encrypted, parttool and esptool don't use the information and wrote to the partition without encryption. When I added the following flag to parttool to address that: --esptool-write-args encrypt

Using both of those, I've been able to create a pre-generated key based NVS partition, write that, and write an nvs_key partition and get it all working. I filed this to recommend a docs addition: https://github.com/espressif/esp-idf/issues/8196