esp32-idf parttool.py stops with error UnicodeDecodeError (IDF-2839)

[MarcRoma]

HI,

I've run out of ideas on this one and now my project is stuck - really appreciate if someone can help on this one.

When running parttool.py to upload a .bin nvs file I get "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 1: ordinal not in range(128)"

I am using esp-idf version 4.2 on Ubuntu 18.04. I've tried to remove and re-install esp-idf. Problem still there. Tried to run with Python 3.6 and 2.7. Still same problem. The command and error messages are shown below. Any ideas are welcomed.


~/esp/esp-idf/components/partition_table/parttool.py write_partition --partition-name=nvs_app --input station33.bin

Traceback (most recent call last):

File "$HOME/esp/esp-idf/components/partition_table/parttool.py", line 347, in <module>
main()

File "$HOME/esp/esp-idf/components/partition_table/parttool.py", line 316, in main
target = ParttoolTarget(**target_args)

File "$HOME/esp/esp-idf/components/partition_table/parttool.py", line 114, in __init__
partition_table = gen.PartitionTable.from_binary(f.read())

File "$HOME/esp/esp-idf/components/partition_table/gen_esp32part.py", line 231, in from_binary
result.append(PartitionDefinition.from_binary(data))

File "$HOME/esp/esp-idf/components/partition_table/gen_esp32part.py", line 381, in from_binary
res.name = res.name.decode()

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 1: ordinal not in range(128)

[ESP_Roland]

Hi MarcRoma,

Are you using a custom partition table? Please check the partition names. It seems that one of the partition names is starting with a weird character which cannot be decoded.

Also if you locate line "res.name = res.name.decode()" in "$HOME/esp/esp-idf/components/partition_table/gen_esp32part.py" then you can add a new line before it for diagnostics, e.g. "print('>>>>>>', res.name)" which could help you to narrow down which name causes the issue.

[MarcRoma]

Hi Roland,


Thx for the reply. Following your suggestion I've created a minimal key-value pair for nvs:

filename: test.csv
content:
key,type,encoding,value
default,namespace,,
key1,data,u8,1

I converted it to binary first, using:
nvs_partition_gen.py generate test.csv test.bin 0x80000.

and then I run the parttool.py command: parttool.py write_partition --partition_name=nvs_app --input test.bin

Got the very same stack trace/error message as before.

I also printed out the content of res.name - The output is binary and does not give me much hint (even when I run od -c)

[ESP_Roland]

I have trouble to reproduce this. We will look into this soon...

[MarcRoma]

As an additional info: I've run the same test after installing the pre-requisites and the esp-idf dev stack on a ubuntu docker image. Got the same error message.

[ESP_Shivani]

Hi MarcRoma ,


I went through this and took a look.

Can you provide me with a few more details:

1. Does your app boot properly ? (say when you run idf.py monitor)

2. Did you use idf.py to flash the app ?

3. What does your partitions.csv file look like ?
Ideally, the res.name line should be a bytestring which is easily readable, it tells us the name of the partition, offset, etc.
This is for ensuring your partitions offset's are not overlapping.

4. Can you send in the first line you get while running the tool, it will say something like `Running ....` (can you provide what you are getting for this line), I believe it is running using python2.7.

[ESP_Shivani]

@MarcRoma

Hi,

Also, just another point, sometimes some editors may add some non-printable characters like when you create an excel sheet with the LibreOffice editor and save it as .csv, it asks for the encoding format, a mismatch is possible to occur here.
So, somewhere while creating the partitions.csv file some non-printable characters may have been added at the start of the partition name.
Could you check the contents of the partitions.csv you are using in a hex editor ?

Thanks

[ESP_Shivani]

Hi MarcRoma,

Could you provide with the information mentioned above, so that we can help further to resolve the issue.

Thanks.