Downloading binary files with AT+HTTPCGET
Downloading binary files with AT+HTTPCGET
With ESP-AT, is it possible to download binary files using "AT+HTTPCGET"?
It seems that the "+AT+HTTPCGET" responses are ASCII with non-printing characters stripped out.
Is it possible to request ESP-AT to pass through the raw binary data escaped in some way?
I understand that I could use the passthrough mode to open a raw SSL socket to a webserver, but then my host processor would need to implement an HTTP client.
As a workaround, I am currently Base64 encoding the binary on the server and decoding on my host processor.
However, this adds overhead and means it's not possible to request arbitrary binary files from HTTP servers.
It seems that the "+AT+HTTPCGET" responses are ASCII with non-printing characters stripped out.
Is it possible to request ESP-AT to pass through the raw binary data escaped in some way?
I understand that I could use the passthrough mode to open a raw SSL socket to a webserver, but then my host processor would need to implement an HTTP client.
As a workaround, I am currently Base64 encoding the binary on the server and decoding on my host processor.
However, this adds overhead and means it's not possible to request arbitrary binary files from HTTP servers.
Re: Downloading binary files with AT+HTTPCGET
i think AT+HTTPCGET supports to download binary files. Could you please provide the reproduce steps, thanks.
~
~
Re: Downloading binary files with AT+HTTPCGET
I cannot receive binary data. The ESP-AT code appears to drop non-printing characters.
I produced a binary file using this short C program. It contains all of the characters from 0-255, repeated for 4 cycles:
I then request the file using:
Here is the data I get from ESP-AT, both in ASCII (RXLINE) and in hex (RXLINERAW).
I produced a binary file using this short C program. It contains all of the characters from 0-255, repeated for 4 cycles:
Code: Select all
#include <stdio.h>
#include <stdint.h>
void main(void) {
uint8_t c = 0;
int i;
for (i=0;i<1024;i++) {
printf("%c", c++);
}
}
Code: Select all
AT+HTTPCGET="http://chuck.local:8080/raw/foo.bin",2048,2048
Code: Select all
RXLINE: 'AT+HTTPCGET="http://chuck.local:8080/raw/foo.bin",2048,2048'
RXLINERAW:41542b48545450434745543d22687474703a2f2f636875636b2e6c6f63616c3a383038302f7261772f666f6f2e62696e222c323034382c32303438
RXLINE: '+HTTPCGET:1024,'
RXLINERAW:2b48545450434745543a313032342c
The first bytes received are 0x20, 0x21 - printable ASCII characters. The non-printing binary characters seem to be discarded.
Code: Select all
RXLINE: ' !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~'
RXLINERAW:202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e
RXLINE: ' !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~'
RXLINERAW:202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e
RXLINE: ' !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~'
RXLINERAW:202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e
...
Re: Downloading binary files with AT+HTTPCGET
Using these instructions, it's possible to place the ESP-AT device into a passthrough mode where there are no encoding issues.
https://github.com/espressif/esp-at/blo ... s.rst#L159
However, it's not clear how to detect the end of a file transfer. If the remote web server does not support HEAD requests (to fetch the expected Content-Length), how can I detect that the file transfer has completed?
https://github.com/espressif/esp-at/blo ... s.rst#L159
However, it's not clear how to detect the end of a file transfer. If the remote web server does not support HEAD requests (to fetch the expected Content-Length), how can I detect that the file transfer has completed?
Re: Downloading binary files with AT+HTTPCGET
Using the above instructions it's possible to have ESP-AT pass through most data, but it seems not all.
I have a webserver with a file containing the following 9 bytes 0d0a0d0a0d0a0a0d0a ("\r\n\r\n\r\n\n\r\n"). ESP-AT appears to be doing some filtering which ends the transfer when "\n\n" is encountered.
Returns only 4 bytes before ending the transfer
How can this be fixed? ESP-AT appears unable to transfer arbitrary binary files.
I have a webserver with a file containing the following 9 bytes 0d0a0d0a0d0a0a0d0a ("\r\n\r\n\r\n\n\r\n"). ESP-AT appears to be doing some filtering which ends the transfer when "\n\n" is encountered.
Code: Select all
AT+HTTPCGET=http://my.server.com/foo.bin,2048,2048
Code: Select all
0d0a
0d0a
0d0a4f4b0d0a ("\r\nOK\r\n")
Re: Downloading binary files with AT+HTTPCGET
could you please enable AT logs and compile AT firmware to see what happened:
python build.py mencuonfig -> Component config -> Log output -> Default log verbosity -> Info
python build.py mencuonfig -> Component config -> AT -> Enable ESP-AT Debug
python build.py mencuonfig -> Component config -> AT -> Enable ESP-AT Debug -> Enable Network Debug
https://docs.espressif.com/projects/esp ... le_it.html
If you have difficulties in compiling ESP-AT project, could you please provide your AT+GMR response and foo.bin here. i would like to look into it.
python build.py mencuonfig -> Component config -> Log output -> Default log verbosity -> Info
python build.py mencuonfig -> Component config -> AT -> Enable ESP-AT Debug
python build.py mencuonfig -> Component config -> AT -> Enable ESP-AT Debug -> Enable Network Debug
https://docs.espressif.com/projects/esp ... le_it.html
If you have difficulties in compiling ESP-AT project, could you please provide your AT+GMR response and foo.bin here. i would like to look into it.
Re: Downloading binary files with AT+HTTPCGET
Log file attached. The zip file contains the binary data I am attempting to download.'AT+GMR'
'AT version:3.5.0.0-dev(0f91d98 - ESP32C3 - Jul 19 2024 02:57:45)'
'SDK version:v5.0.6-dirty'
'compile time(5d1f30af):Aug 6 2024 09:58:55'
'Bin version:v3.4.0.0-dev(MINI-1)'
Thanks
Re: Downloading binary files with AT+HTTPCGET
thanks for your example, i can reproduce it now. will fix it within one week.
~
~
Re: Downloading binary files with AT+HTTPCGET
Hello, this issue should get fixed on master branch, could you please git pull the code and have a test.
Re: Downloading binary files with AT+HTTPCGET
Thank you. This is working in all of my tests so far.
Who is online
Users browsing this forum: No registered users and 23 guests