ota checksum failed

[imtiaz]

Hi Espressif ,

This looks like a hardware problem on our module - can someone help me as to the reason why this may happen. After doing an OTA update of the esp32 I get :

E (148645) esp_image: checksum failed. Calculated 0x8d read 0xff

The bootloader is updating through serial fine - so Im not sure what the issue could be in hardware. The same software works fine on off the shelf modules.

I've reduced my SPI speed to 20MHZ , DIO mode - the chip is a standard supported chip.

Regards
Imtiaz

[ESP_Sprite]

What OTA process do you use? Can you use the bootloader and esptool.py to read the OTA partition back over serial and compare that with what it should be? It sounds like your OTA image gets corrupted somewhere in the process...

[imtiaz]

Hi @ESP_Sprite,

The OTA process - download file over a tcp connection and write to OTA partition with the esp idf OTA API.

I am assuming that the TCP connection guarantees data integrity and order etc.

I have just tried storing the file in a another partition and calculating the MD5 hash of the received file - which is failing.

It seems that even though the file is sent successfully and received over tcp , the MD5 hash is failing - which I didn't think would be possible.

Any ideas - and this is only happens on our module which is quite badly tuned - RF wise ?

[ESP_Sprite]

TCP should indeed mostly guarantee integrity - in theory, there's about a 0.01% chance that a packet that gets corrupted will go through the checks in the TCP proto alone, and even less when you keep the entire IP/WiFi stack in mind.

Maybe there's something in the flash that throws off the OTA process? Can you see what happens if you do a 'make erase_flash' on the module, reflash and re-try?

[ESP_Angus]

I've seen this kind of problem with faulty or browning out flash. Can you please share some details about the exact hardware configuration you have? (ie Espressif or other branded module, or bare chip on a custom board, what flash voltage, etc.)

If you still have physical access, one thing you can do is read back the OTA partition contents with "esptool.py read_flash addr filename", convert the original .bin and the readback .bin to HEX files, and diff the contents.

[imtiaz]

So it looks like if the tcp data is correct because I did an on the fly md5 hash (as I receive the packets) and it passes - but fails the hash on the flash partition.

Hardware configuration attached - its our own module.

schematic

Capture.PNG (67.26 KiB) Viewed 31685 times

[imtiaz]

Problem found - it was our power supply - not handling the peak currents - thanks for your help

[Ritesh]

Problem found - it was our power supply - not handling the peak currents - thanks for your help

Hello,

We are facing somilar issue like device is powered up through invenrtor which works fine for few days and then suddenly stuck with following logs

[2022-03-31 14:20:34.371] rst:0x1 (POWERON_RESET),boot:0x12 (SPI_FAST_FLASH_BOOT)
[2022-03-31 14:20:34.371] configsip: 0, SPIWP:0xee
[2022-03-31 14:20:34.371] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[2022-03-31 14:20:34.371] mode:DIO, clock div:2
[2022-03-31 14:20:34.371] load:0x3fff0018,len:4
[2022-03-31 14:20:34.371] load:0x3fff001c,len:5840
[2022-03-31 14:20:34.371] ho 0 tail 12 room 4
[2022-03-31 14:20:34.371] load:0x40078000,len:7756
[2022-03-31 14:20:34.371] load:0x40080000,len:5876
[2022-03-31 14:20:34.371] entry 0x40080318
[2022-03-31 14:20:34.371] I (31) boot: ESP-IDF v3.1-beta1-216-gd91c42517 2nd stage bootloader
[2022-03-31 14:20:34.371] I (31) boot: compile time 11:47:03
[2022-03-31 14:20:34.371] I (32) boot: Enabling RNG early entropy source...
[2022-03-31 14:20:34.371] I (37) boot: SPI Speed : 40MHz
[2022-03-31 14:20:34.371] I (41) boot: SPI Mode : DIO
[2022-03-31 14:20:34.371] I (45) boot: SPI Flash Size : 16MB
[2022-03-31 14:20:34.371] I (50) boot: Partition Table:
[2022-03-31 14:20:34.371] I (53) boot: ## Label Usage Type ST Offset Length
[2022-03-31 14:20:34.371] I (60) boot: 0 nvs WiFi data 01 02 00009000 00004000
[2022-03-31 14:20:34.371] I (68) boot: 1 otadata OTA data 01 00 0000d000 00002000
[2022-03-31 14:20:34.371] I (75) boot: 2 phy_init RF data 01 01 0000f000 00001000
[2022-03-31 14:20:34.402] I (83) boot: 3 factory factory app 00 00 00010000 00200000
[2022-03-31 14:20:34.402] I (90) boot: 4 ota_0 OTA app 00 10 00210000 00200000
[2022-03-31 14:20:34.402] I (98) boot: 5 storage Unknown data 01 82 0081c000 00200000
[2022-03-31 14:20:34.402] I (105) boot: End of partition table
[2022-03-31 14:20:34.402] I (110) boot: Defaulting to factory image
[2022-03-31 14:20:34.402] I (114) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x2d578 (185720) map
[2022-03-31 14:20:34.439] I (188) esp_image: segment 1: paddr=0x0003d5a0 vaddr=0x3ffb0000 size=0x02a70 ( 10864) load
[2022-03-31 14:20:34.439] I (193) esp_image: segment 2: paddr=0x00040018 vaddr=0x400d0018 size=0x9fa98 (653976) map
[2022-03-31 14:20:34.648] I (424) esp_image: segment 3: paddr=0x000dfab8 vaddr=0x3ffb2a70 size=0x0117c ( 4476) load
[2022-03-31 14:20:34.697] I (426) esp_image: segment 4: paddr=0x000e0c3c vaddr=0x3ffb3bec size=0x00000 ( 0) load
[2022-03-31 14:20:34.697] I (430) esp_image: segment 5: paddr=0x000e0c44 vaddr=0x40080000 size=0x00400 ( 1024) load
[2022-03-31 14:20:34.697] I (440) esp_image: segment 6: paddr=0x000e104c vaddr=0x40080400 size=0x12340 ( 74560) load
[2022-03-31 14:20:34.783] I (478) esp_image: segment 7: paddr=0x000f3394 vaddr=0x400c0000 size=0x00000 ( 0) load
[2022-03-31 14:20:34.783] I (478) esp_image: segment 8: paddr=0x000f339c vaddr=0x50000000 size=0x00000 ( 0) load
[2022-03-31 14:20:34.783] E (485) esp_image: Checksum failed. Calculated 0xd2 read 0xc2

Then it is going to restart again and again.

We have dumped image from device and checked using image_info command

$ python esptool.py --chip esp32 image_info Shailesh.bin
esptool.py v2.5.0
Image version: 1
Entry point: 40081314
9 segments
Segment 1: len 0x2d578 load 0x3f400020 file_offs 0x00000018
Segment 2: len 0x02a70 load 0x3ffb0000 file_offs 0x0002d598
Segment 3: len 0x9fa98 load 0x400d0018 file_offs 0x00030010
Segment 4: len 0x0117c load 0x3ffb2a70 file_offs 0x000cfab0
Segment 5: len 0x00000 load 0x3ffb3bec file_offs 0x000d0c34
Segment 6: len 0x00400 load 0x40080000 file_offs 0x000d0c3c
Segment 7: len 0x12340 load 0x40080400 file_offs 0x000d1044
Segment 8: len 0x00000 load 0x400c0000 file_offs 0x000e338c
Segment 9: len 0x00000 load 0x50000000 file_offs 0x000e3394
Checksum: c2 (invalid - calculated d2)
Validation Hash: ebc71f84a4098082cbc2ef508c17859e2c4fb12472926687699fab36402c167e (invalid)

So, Does it due to power fluctuation? But strange thing is like if we reflash firmware using flash download tool then it is again working without any issue.

I am bit surpriced with the issue behaviour and recovery mechanism but didn't find anything for the same.

Let me know if you have anything or any clue for the same.

[Ritesh]

Problem found - it was our power supply - not handling the peak currents - thanks for your help

Hello,

We are facing somilar issue like device is powered up through invenrtor which works fine for few days and then suddenly stuck with following logs

[2022-03-31 14:20:34.371] rst:0x1 (POWERON_RESET),boot:0x12 (SPI_FAST_FLASH_BOOT)
[2022-03-31 14:20:34.371] configsip: 0, SPIWP:0xee
[2022-03-31 14:20:34.371] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[2022-03-31 14:20:34.371] mode:DIO, clock div:2
[2022-03-31 14:20:34.371] load:0x3fff0018,len:4
[2022-03-31 14:20:34.371] load:0x3fff001c,len:5840
[2022-03-31 14:20:34.371] ho 0 tail 12 room 4
[2022-03-31 14:20:34.371] load:0x40078000,len:7756
[2022-03-31 14:20:34.371] load:0x40080000,len:5876
[2022-03-31 14:20:34.371] entry 0x40080318
[2022-03-31 14:20:34.371] I (31) boot: ESP-IDF v3.1-beta1-216-gd91c42517 2nd stage bootloader
[2022-03-31 14:20:34.371] I (31) boot: compile time 11:47:03
[2022-03-31 14:20:34.371] I (32) boot: Enabling RNG early entropy source...
[2022-03-31 14:20:34.371] I (37) boot: SPI Speed : 40MHz
[2022-03-31 14:20:34.371] I (41) boot: SPI Mode : DIO
[2022-03-31 14:20:34.371] I (45) boot: SPI Flash Size : 16MB
[2022-03-31 14:20:34.371] I (50) boot: Partition Table:
[2022-03-31 14:20:34.371] I (53) boot: ## Label Usage Type ST Offset Length
[2022-03-31 14:20:34.371] I (60) boot: 0 nvs WiFi data 01 02 00009000 00004000
[2022-03-31 14:20:34.371] I (68) boot: 1 otadata OTA data 01 00 0000d000 00002000
[2022-03-31 14:20:34.371] I (75) boot: 2 phy_init RF data 01 01 0000f000 00001000
[2022-03-31 14:20:34.402] I (83) boot: 3 factory factory app 00 00 00010000 00200000
[2022-03-31 14:20:34.402] I (90) boot: 4 ota_0 OTA app 00 10 00210000 00200000
[2022-03-31 14:20:34.402] I (98) boot: 5 storage Unknown data 01 82 0081c000 00200000
[2022-03-31 14:20:34.402] I (105) boot: End of partition table
[2022-03-31 14:20:34.402] I (110) boot: Defaulting to factory image
[2022-03-31 14:20:34.402] I (114) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x2d578 (185720) map
[2022-03-31 14:20:34.439] I (188) esp_image: segment 1: paddr=0x0003d5a0 vaddr=0x3ffb0000 size=0x02a70 ( 10864) load
[2022-03-31 14:20:34.439] I (193) esp_image: segment 2: paddr=0x00040018 vaddr=0x400d0018 size=0x9fa98 (653976) map
[2022-03-31 14:20:34.648] I (424) esp_image: segment 3: paddr=0x000dfab8 vaddr=0x3ffb2a70 size=0x0117c ( 4476) load
[2022-03-31 14:20:34.697] I (426) esp_image: segment 4: paddr=0x000e0c3c vaddr=0x3ffb3bec size=0x00000 ( 0) load
[2022-03-31 14:20:34.697] I (430) esp_image: segment 5: paddr=0x000e0c44 vaddr=0x40080000 size=0x00400 ( 1024) load
[2022-03-31 14:20:34.697] I (440) esp_image: segment 6: paddr=0x000e104c vaddr=0x40080400 size=0x12340 ( 74560) load
[2022-03-31 14:20:34.783] I (478) esp_image: segment 7: paddr=0x000f3394 vaddr=0x400c0000 size=0x00000 ( 0) load
[2022-03-31 14:20:34.783] I (478) esp_image: segment 8: paddr=0x000f339c vaddr=0x50000000 size=0x00000 ( 0) load
[2022-03-31 14:20:34.783] E (485) esp_image: Checksum failed. Calculated 0xd2 read 0xc2

Then it is going to restart again and again.

We have dumped image from device and checked using image_info command

$ python esptool.py --chip esp32 image_info Shailesh.bin
esptool.py v2.5.0
Image version: 1
Entry point: 40081314
9 segments
Segment 1: len 0x2d578 load 0x3f400020 file_offs 0x00000018
Segment 2: len 0x02a70 load 0x3ffb0000 file_offs 0x0002d598
Segment 3: len 0x9fa98 load 0x400d0018 file_offs 0x00030010
Segment 4: len 0x0117c load 0x3ffb2a70 file_offs 0x000cfab0
Segment 5: len 0x00000 load 0x3ffb3bec file_offs 0x000d0c34
Segment 6: len 0x00400 load 0x40080000 file_offs 0x000d0c3c
Segment 7: len 0x12340 load 0x40080400 file_offs 0x000d1044
Segment 8: len 0x00000 load 0x400c0000 file_offs 0x000e338c
Segment 9: len 0x00000 load 0x50000000 file_offs 0x000e3394
Checksum: c2 (invalid - calculated d2)
Validation Hash: ebc71f84a4098082cbc2ef508c17859e2c4fb12472926687699fab36402c167e (invalid)

So, Does it due to power fluctuation? But strange thing is like if we reflash firmware using flash download tool then it is again working without any issue.

I am bit surpriced with the issue behaviour and recovery mechanism but didn't find anything for the same.

Let me know if you have anything or any clue for the same.

Hello WiFive and others,

Let me know if anyone has any clue as we are getting this type of issues into multiple sites where device is actually installed and facing issue randomly.

We have already provided one hardware to Espressif Team for further investigation.

[Ritesh]

Hello,

Anyone has any idea or clue regarding issue description which i have posted into last couple of thread? Because we are facing issues at customer site frequently for which current solution is to replace device or troubleshoot it and repair it.

Let me know if anyone can help us to at-least identify issue which will great for us.