ESP32 Forum

ESP32 Official Forum
http://esp32.io/

Is the OTA binary file size calculation wrong !?

http://esp32.io/viewtopic.php?f=13&t=24202

Page 1 of 1

Is the OTA binary file size calculation wrong !?

Posted: Fri Nov 05, 2021 11:39 am
by McDottie
---- EDIT ----
This is corrected in the latest version it was a problem with OpenSSl
---- EDIT ----

Hi, for the the project i'm working on i need to be able to calculate the MD5 of a binary downloaded using the ESP-IDF OTA library and i've encountered a problem.

To do this i need to know the size of the file wich is given by executing the function

Code: Select all

esp_https_ota_get_image_len_read
after the download is done. The problem i'm facing is that the image size, independatly of the real image size, is returning always 289 bytes over the actual size.
In my experiences the only thing that i observed was that the

Code: Select all

esp_https_ota_perform
function is being called every 289 bytes. Is there a bug where it counts in excess or am i reading this all wrong.

Sorry for the short ammount of details regards,
Jose

Re: Is the OTA binary file size calculation wrong !?

Posted: Mon Nov 08, 2021 4:58 am
by ESP_Mahavir
Hello,

Can you please try "advanced_https_ota" example once? Please see code along the lines of https://github.com/espressif/esp-idf/bl ... #L143-L150

Additionally, enabling debug logs could also help here. Please see https://github.com/espressif/esp-idf/bl ... ota.c#L161

Thanks.

Re: Is the OTA binary file size calculation wrong !?

Posted: Tue Nov 09, 2021 8:38 am
by McDottie
ESP_Mahavir wrote: ↑
Mon Nov 08, 2021 4:58 am
 Hello,

Can you please try "advanced_https_ota" example once? Please see code along the lines of https://github.com/espressif/esp-idf/bl ... #L143-L150

Additionally, enabling debug logs could also help here. Please see https://github.com/espressif/esp-idf/bl ... ota.c#L161

Thanks.
Sorry for the delayed response and thanks for the response. I have tried the example you mensioned, and the code i made was based on that, the read is preformed after

Code: Select all

esp_https_ota_perform
and in a loop.

After the debug log experiment i've observed the following:

Code: Select all

D (124982) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=790112
D (124992) TRANS_SSL: remain data in cache, need to read again
D (125002) HTTP_CLIENT: need_read=289, byte_to_read=289, rlen=289, ridx=0
D (125002) HTTP_CLIENT: http_on_body 289
D (125012) esp_https_ota: Written image length 790126

D (125022) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=790112
D (125032) TRANS_SSL: remain data in cache, need to read again
D (125032) HTTP_CLIENT: need_read=289, byte_to_read=289, rlen=275, ridx=0
D (125042) HTTP_CLIENT: http_on_body 275
D (125042) HTTP_CLIENT: http_on_message_complete, parser=3ffd4b30
D (125052) HTTP_CLIENT: is_data_remain=0, is_chunked=0, content_length=790112
D (125062) esp_https_ota: Written image length 790401


So as you can see the content_length as the expected value where as the written image length as 289 bytes over the actual value, moreover every read that is made is made every 289 bytes so i assume that the counting of the size is wrong. Is there a way to access the content-length directly beeing that that is what i need?

Thanks

Re: Is the OTA binary file size calculation wrong !?

Posted: Wed Nov 10, 2021 10:26 am
by ESP_Mahavir
I would suggest to use API `esp_https_ota_get_image_size` https://github.com/espressif/esp-idf/bl ... #L205-L218 instead. Since your logs shows that server is not sending data in chunked encoded fashion, this API shall directly return value from "Content-Length" header.

So as you can see the content_length as the expected value where as the written image length as 289 bytes over the actual value, moreover every read that is made is made every 289 bytes so i assume that the counting of the size is wrong.

We will check on this difference.

Hope this helps.

Re: Is the OTA binary file size calculation wrong !?

Posted: Wed Nov 10, 2021 10:53 am
by ESP_Shubham
Hi,

I tried running advanced_https_ota example on IDF master (v5.0-dev-258-gf8f49de735) and did not see any difference in value of written image length and content length. Please see debug logs below:

Code: Select all

D (35048) esp_https_ota: Written image length 145656
D (35058) advanced_https_ota_example: Image bytes read: 145656
D (35058) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=145905
D (35068) TRANSPORT_BASE: remain data in cache, need to read again
D (35068) HTTP_CLIENT: need_read=289, byte_to_read=289, rlen=249, ridx=0
D (35078) HTTP_CLIENT: http_on_body 249
D (35088) HTTP_CLIENT: http_on_message_complete, parser=3ffaf4bc
D (35088) HTTP_CLIENT: is_data_remain=0, is_chunked=0, content_length=145905
D (35098) esp_https_ota: Written image length 145905
D (35098) advanced_https_ota_example: Image bytes read: 145905
D (35108) HTTP_CLIENT: is_data_remain=0, is_chunked=0, content_length=145905
D (35118) esp_https_ota: Connection closed
If you are observing this issue on specific IDF version, could you share IDF version by running `git describe --tags`?

Thanks,
Shubham

Re: Is the OTA binary file size calculation wrong !?

Posted: Wed Nov 10, 2021 11:57 am
by McDottie
ESP_Shubham wrote: ↑
Wed Nov 10, 2021 10:53 am
 Hi,

I tried running advanced_https_ota example on IDF master (v5.0-dev-258-gf8f49de735) and did not see any difference in value of written image length and content length. Please see debug logs below:

Code: Select all

D (35048) esp_https_ota: Written image length 145656
D (35058) advanced_https_ota_example: Image bytes read: 145656
D (35058) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=145905
D (35068) TRANSPORT_BASE: remain data in cache, need to read again
D (35068) HTTP_CLIENT: need_read=289, byte_to_read=289, rlen=249, ridx=0
D (35078) HTTP_CLIENT: http_on_body 249
D (35088) HTTP_CLIENT: http_on_message_complete, parser=3ffaf4bc
D (35088) HTTP_CLIENT: is_data_remain=0, is_chunked=0, content_length=145905
D (35098) esp_https_ota: Written image length 145905
D (35098) advanced_https_ota_example: Image bytes read: 145905
D (35108) HTTP_CLIENT: is_data_remain=0, is_chunked=0, content_length=145905
D (35118) esp_https_ota: Connection closed
If you are observing this issue on specific IDF version, could you share IDF version by running `git describe --tags`?

Thanks,
Shubham
Hi, i am using version v4.3.

Re: Is the OTA binary file size calculation wrong !?

Posted: Wed Nov 10, 2021 12:00 pm
by McDottie
ESP_Mahavir wrote: ↑
Wed Nov 10, 2021 10:26 am
 I would suggest to use API `esp_https_ota_get_image_size` https://github.com/espressif/esp-idf/bl ... #L205-L218 instead. Since your logs shows that server is not sending data in chunked encoded fashion, this API shall directly return value from "Content-Length" header.

So as you can see the content_length as the expected value where as the written image length as 289 bytes over the actual value, moreover every read that is made is made every 289 bytes so i assume that the counting of the size is wrong.

We will check on this difference.

Hope this helps.
Hi, i was going to used it the problem is that verson 4.3 with i believe is the latest stable one does not provide that function

Re: Is the OTA binary file size calculation wrong !?

Posted: Thu Apr 27, 2023 12:10 pm
by Gardin
Any news on this? I'am having the exact same issue

Re: Is the OTA binary file size calculation wrong !?

Posted: Tue May 02, 2023 9:27 am
by hmalpani
Hello,
I tried the advanced_ota example on release/v4.3 branch and did not see any difference in the two values.
@Gardin, can you share the idf version as well as the test code which could reproduce this issue?
Thanks!
All times are UTC
Page 1 of 1
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/