HTTP Get file on amazon "S3"
-
- Posts: 11
- Joined: Sun Apr 03, 2022 10:39 am
HTTP Get file on amazon "S3"
Hello,
I'm using ESP32-C3 with ESP-AT firmware (v2.4...)
Another part has created account on Amazon. Then, We generate some device with certificate.
I convert these certificate (ca, cert and key) with the provided python script AtPKI.py.
Then I flash these files to the ESP32Flash on partition mqtt_ca, mqtt_key and mqtt_cert with the command AT+SYSFLASH.
Then I'm able to connect to the amazon server on MQTT with commands:
- AT+CIPSNTPCFG (set time)
- AT+MQTTUSERCFG
- AT + MQTTCONN..
After that I can publish message and subscribe to a topic.
This work properly (Tested with the MQTT client test on Amazon web interface)
Now I want be able to get a file.
To do that, I upload my file on "amazon S3" module.
This module allow me to upload a file and generate an url (for some couple of hour).
The generated URL is like this:
https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com/<myrep>/<myfile>.xxx?response-content-disposition=inline&X-Amz-Security-Token=IQoJb3JpZ2luX2V[....]Amz-Date=20221026T151421Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43200&X-Amz-Credential=ASIAWXYV5C2B6FEAXLEB%2F2022[....]
If I copy/paste this URL into my web browser, the file is downloaded. Then the link is good.
Now I want download this file with the ESP32.
Like the URL is > 256 Byte, I write the follwoing command:
- AT+HTTPURLCFG=1668
- >
- <myURL>
- AT+HTTPURLCFG? -> return my URL.
Follwing is my terminal view with different try:
17:42:13.407 [TX] - AT+HTTPGETSIZE=""
17:42:13.422 [RX] - AT+HTTPGETSIZE=""
17:42:13.998 [RX] -
ERROR
17:45:13.470 [RX] - AT+HTTPGETSIZE="https://<mysite>-ota.s3.eu-west-1.amazonaws.com/<myrep>/<myfile>.xxx"
17:45:14.046 [RX] - +HTTPGETSIZE:0
OK
17:42:40.587 [TX] - AT+HTTPCLIENT=2,0,"","https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com","/<myrep>",2
17:42:40.600 [RX] - AT+HTTPCLIENT=2,0,"","https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com","/<myrep>",2
17:42:41.228 [RX] - ERR CODE:0x010a7190
ERROR
17:43:24.569 [TX] - AT+HTTPCLIENT=2,0,"",,,2
17:43:24.585 [RX] - AT+HTTPCLIENT=2,0,"",,,2
17:43:25.128 [RX] - ERR CODE:0x010a7000
ERROR
Now if I create a local HTTP server on my computer, and just use AT+HTTPURLCFG=xx then My local url.
Then I get the size and then the file with following commands:
AT+HTTPCLIENT=2,0,"",,,2
AT+HTTPGETSIZE=""
All work properly.
In other word, I get error only with amazon file..
I think there are certificate problem?? But the url contains the key...
Where is my mistake?
Are they exemple to get HTTP file on amazon s3 server?
Regards
I'm using ESP32-C3 with ESP-AT firmware (v2.4...)
Another part has created account on Amazon. Then, We generate some device with certificate.
I convert these certificate (ca, cert and key) with the provided python script AtPKI.py.
Then I flash these files to the ESP32Flash on partition mqtt_ca, mqtt_key and mqtt_cert with the command AT+SYSFLASH.
Then I'm able to connect to the amazon server on MQTT with commands:
- AT+CIPSNTPCFG (set time)
- AT+MQTTUSERCFG
- AT + MQTTCONN..
After that I can publish message and subscribe to a topic.
This work properly (Tested with the MQTT client test on Amazon web interface)
Now I want be able to get a file.
To do that, I upload my file on "amazon S3" module.
This module allow me to upload a file and generate an url (for some couple of hour).
The generated URL is like this:
https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com/<myrep>/<myfile>.xxx?response-content-disposition=inline&X-Amz-Security-Token=IQoJb3JpZ2luX2V[....]Amz-Date=20221026T151421Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43200&X-Amz-Credential=ASIAWXYV5C2B6FEAXLEB%2F2022[....]
If I copy/paste this URL into my web browser, the file is downloaded. Then the link is good.
Now I want download this file with the ESP32.
Like the URL is > 256 Byte, I write the follwoing command:
- AT+HTTPURLCFG=1668
- >
- <myURL>
- AT+HTTPURLCFG? -> return my URL.
Follwing is my terminal view with different try:
17:42:13.407 [TX] - AT+HTTPGETSIZE=""
17:42:13.422 [RX] - AT+HTTPGETSIZE=""
17:42:13.998 [RX] -
ERROR
17:45:13.470 [RX] - AT+HTTPGETSIZE="https://<mysite>-ota.s3.eu-west-1.amazonaws.com/<myrep>/<myfile>.xxx"
17:45:14.046 [RX] - +HTTPGETSIZE:0
OK
17:42:40.587 [TX] - AT+HTTPCLIENT=2,0,"","https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com","/<myrep>",2
17:42:40.600 [RX] - AT+HTTPCLIENT=2,0,"","https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com","/<myrep>",2
17:42:41.228 [RX] - ERR CODE:0x010a7190
ERROR
17:43:24.569 [TX] - AT+HTTPCLIENT=2,0,"",,,2
17:43:24.585 [RX] - AT+HTTPCLIENT=2,0,"",,,2
17:43:25.128 [RX] - ERR CODE:0x010a7000
ERROR
Now if I create a local HTTP server on my computer, and just use AT+HTTPURLCFG=xx then My local url.
Then I get the size and then the file with following commands:
AT+HTTPCLIENT=2,0,"",,,2
AT+HTTPGETSIZE=""
All work properly.
In other word, I get error only with amazon file..
I think there are certificate problem?? But the url contains the key...
Where is my mistake?
Are they exemple to get HTTP file on amazon s3 server?
Regards
Re: HTTP Get file on amazon "S3"
i don't think it's a certificate issue, but a buffer size issue due to long url.
could you please compile esp-at firmware with latest master branch, and change DEFAULT_HTTP_BUF_SIZE to 4096 in esp-idf/components/esp_http_client/include/esp_http_client.h, and see if this issue still exists.
~
could you please compile esp-at firmware with latest master branch, and change DEFAULT_HTTP_BUF_SIZE to 4096 in esp-idf/components/esp_http_client/include/esp_http_client.h, and see if this issue still exists.
~
Re: HTTP Get file on amazon "S3"
Could you please use AT+HTTPCGET command instead.
like:
AT+HTTPURLCFG=<length>
AT+HTTPCGET="",<4096>,<2048>
like:
AT+HTTPURLCFG=<length>
AT+HTTPCGET="",<4096>,<2048>
-
- Posts: 11
- Joined: Sun Apr 03, 2022 10:39 am
Re: HTTP Get file on amazon "S3"
Hello
Thanl for your help.
The command AT+HTTPCGET work properly.
Regards
Thanl for your help.
The command AT+HTTPCGET work properly.
Regards
-
- Posts: 11
- Joined: Sun Apr 03, 2022 10:39 am
Re: HTTP Get file on amazon "S3"
Hello
I missed to check again the command AT+HTTPGETSIZE=""
So to get the file, I used the command AT+HTTPCGET, but before that, I would get the file size to erase the amount of neessary flash before receive the data flow.
Then I call the command
AT+HTTPCGET
But return "ERROR"
I missed to check again the command AT+HTTPGETSIZE=""
So to get the file, I used the command AT+HTTPCGET, but before that, I would get the file size to erase the amount of neessary flash before receive the data flow.
Then I call the command
AT+HTTPCGET
But return "ERROR"
Re: HTTP Get file on amazon "S3"
AT+HTTPGETSIZE returns ERROR, because of long url. you can change DEFAULT_HTTP_BUF_SIZE to 4096 in esp-idf/components/esp_http_client/include/esp_http_client.h.
i would like to add a tx parameter for AT+HTTPGETSIZE to support long url.
~
i would like to add a tx parameter for AT+HTTPGETSIZE to support long url.
~
-
- Posts: 11
- Joined: Sun Apr 03, 2022 10:39 am
Re: HTTP Get file on amazon "S3"
Hello
I havn't install all IDE to compile the source code.
I get directly the oficial release of ESP-AT, And I hope to keep this because is simplier for further update to only need to use the command AT+CIUPDATE...
Do you think is possible to add this option to make GETSIZE compatible with long URL?
I havn't install all IDE to compile the source code.
I get directly the oficial release of ESP-AT, And I hope to keep this because is simplier for further update to only need to use the command AT+CIUPDATE...
Do you think is possible to add this option to make GETSIZE compatible with long URL?
Re: HTTP Get file on amazon "S3"
plase test AT+HTTPGETSIZE=<"url">[,<tx_size>][,<rx_size>][,<timeout_ms>] feature with the firmware in the attachment.
such as AT+HTTPGETSIZE="https://blabla",2048,2048,5000
This feature will sync to GitHub within next two weeks.
~
such as AT+HTTPGETSIZE="https://blabla",2048,2048,5000
This feature will sync to GitHub within next two weeks.
~
- Attachments
-
- target-01.zip
- (859.98 KiB) Downloaded 297 times
-
- Posts: 11
- Joined: Sun Apr 03, 2022 10:39 am
Re: HTTP Get file on amazon "S3"
Ok,
THese two follocing command run correctly:
You means the release v2.5.0.0 which include this new functionality will be released within 2 weeks?
Thank for this additional function.
THese two follocing command run correctly:
So I deduce you increase the default buffer size?19:53:11.919 [TX] - AT+HTTPGETSIZE="",2048,2048,5000
19:53:11.927 [RX] - AT+HTTPGETSIZE="",2048,2048,5000
19:53:12.790 [RX] - +HTTPGETSIZE:28298
OK
19:53:19.230 [TX] - AT+HTTPGETSIZE=""
19:53:19.233 [RX] - AT+HTTPGETSIZE=""
19:53:20.480 [RX] - +HTTPGETSIZE:28298
OK
You means the release v2.5.0.0 which include this new functionality will be released within 2 weeks?
Thank for this additional function.
Re: HTTP Get file on amazon "S3"
please download latest esp-at firmware from GitHub Actions: https://github.com/espressif/esp-at/act ... 3368936467
See https://docs.espressif.com/projects/esp ... ithub.html for more details.
See https://docs.espressif.com/projects/esp ... ithub.html for more details.
Who is online
Users browsing this forum: No registered users and 13 guests