I randomly receive an incomplete responsebody when I use httpclient for a GET request
I'm using platformIO
Code: Select all
platform = espressif32@4.4.0
framework = arduino
Code: Select all
HTTPClient http;
http.begin("https://api.coingecko.com/api/v3/coins/markets.json?vs_currency=usd&order=market_cap_desc&per_page=6&page=0&sparkline=false&price_change_percentage=1h%2C24h%2C7d");
short httpResponseCode = http.GET();
String jsonBuffer = http.getString();
Serial.println("-------------------------------------------------");
Serial.println(jsonBuffer);
Serial.println("-------------------------------------------------");
http.end();And here is the logs:
Code: Select all
[112979][V][HTTPClient.cpp:252] beginInternal(): url: https://api.coingecko.com/api/v3/coins/markets.json?vs_currency=usd&order=market_cap_desc&per_page=6&page=0&sparkline=false&price_change_percentage=1h%2C24h%2C7d
[112987][D][HTTPClient.cpp:263] beginInternal(): unexpected protocol: https, expected http
[112995][V][HTTPClient.cpp:252] beginInternal(): url: https://api.coingecko.com/api/v3/coins/markets.json?vs_currency=usd&order=market_cap_desc&per_page=6&page=0&sparkline=false&price_change_percentage=1h%2C24h%2C7d
[113014][D][HTTPClient.cpp:303] beginInternal(): protocol: https, host: api.coingecko.com port: 443 url: /api/v3/coins/markets.json?vs_currency=usd&order=market_cap_desc&per_page=6&page=0&sparkline=false&price_change_percentage=1h%2C24h%2C7d
[113036][D][HTTPClient.cpp:598] sendRequest(): request type: 'GET' redirCount: 0
[113043][V][ssl_client.cpp:62] start_ssl_client(): Free internal heap before TLS 111792
[113051][V][ssl_client.cpp:68] start_ssl_client(): Starting socket
[113088][V][ssl_client.cpp:144] start_ssl_client(): Seeding the random number generator
[113089][V][ssl_client.cpp:153] start_ssl_client(): Setting up the SSL/TLS structure...
[113093][D][ssl_client.cpp:174] start_ssl_client(): WARNING: Skipping SSL Verification. INSECURE!
[113101][V][ssl_client.cpp:252] start_ssl_client(): Setting hostname for TLS session...
[113109][V][ssl_client.cpp:267] start_ssl_client(): Performing the SSL/TLS handshake...
[114480][V][ssl_client.cpp:288] start_ssl_client(): Verifying peer X.509 certificate...
[114481][V][ssl_client.cpp:296] start_ssl_client(): Certificate verified.
[114484][V][ssl_client.cpp:311] start_ssl_client(): Free internal heap after TLS 61136
[114492][D][HTTPClient.cpp:1156] connect(): connected to api.coingecko.com:443
[114499][V][ssl_client.cpp:364] send_ssl_data(): Writing HTTP request with 277 bytes...
[114551][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'HTTP/1.1 200 OK'
[114552][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Date: Thu, 23 Feb 2023 09:42:17 GMT'
[114557][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Content-Type: application/json; charset=utf-8'
[114566][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Transfer-Encoding: chunked'
[114574][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Connection: close'
[114581][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'X-Frame-Options: SAMEORIGIN'
[114589][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'X-XSS-Protection: 0'
[114597][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'X-Content-Type-Options: nosniff'
[114605][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'X-Download-Options: noopen'
[114613][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'X-Permitted-Cross-Domain-Policies: none'
[114622][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Referrer-Policy: strict-origin-when-cross-origin'
[114632][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Cache-Control: public, max-age=120'
[114640][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Access-Control-Allow-Origin: *'
[114649][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Access-Control-Allow-Methods: POST, PUT, DELETE, GET, OPTIONS'
[114660][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Access-Control-Request-Method: *'
[114669][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization'
[114682][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Access-Control-Expose-Headers: link, per-page, total'
[114692][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Vary: Accept-Encoding, Origin'
[114700][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'ETag: W/"b5f02132d05c1235fd84b57a4c261149"'
[114710][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'X-Request-Id: 3e3324e2-ae1d-4e76-a856-66b07e9ed7a9'
[114720][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'X-Runtime: 0.021611'
[114727][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Alternate-Protocol: 443:npn-spdy/2'
[114736][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'CF-Cache-Status: HIT'
[114743][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Age: 104'
[114749][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Expires: Thu, 23 Feb 2023 09:44:17 GMT'
[114760][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Set-Cookie: __cf_bm=xjRwMk1GuCU_Kf28_SdIUckyYIwP0kFoYO1cwe5ixT8-1677145337-0-ASiBmL8obVcySZF+B+Q7gUO5G0BTSSZSXiLNHk3dpaS37ZMz7Q3v08HospIacda9CsPyDMrcDXHIu2k1NBjIvLQ=; path=/; expires=Thu, 23-Feb-23 10:12:17 GMT; domain=.api.coingecko.com; HttpOnly; Secure; SameSite=None'
[114788][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'Server: cloudflare'
[114795][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: 'CF-RAY: 79df02b99b64bb09-MXP'
[114803][V][HTTPClient.cpp:1250] handleHeaderResponse(): RX: ''
[114808][D][HTTPClient.cpp:1307] handleHeaderResponse(): code: 200
[114814][D][HTTPClient.cpp:1314] handleHeaderResponse(): Transfer-Encoding: chunked
[114822][D][HTTPClient.cpp:628] sendRequest(): sendRequest code=200
[114828][D][HTTPClient.cpp:922] writeToStream(): read chunk len: 5756
[114857][D][HTTPClient.cpp:1399] writeToStreamDataBlock(): short write asked for 1369 but got 0 retry...
[114859][W][HTTPClient.cpp:1420] writeToStreamDataBlock(): short write asked for 1369 but got 0 failed.
[114865][W][HTTPClient.cpp:1469] returnError(): error(-10): Stream write error
[114874][D][HTTPClient.cpp:1471] returnError(): tcp stop
[114878][V][ssl_client.cpp:319] stop_ssl_socket(): Cleaning SSL connection.
-------------------------------------------------
[{"id":"bitcoin","symbol":"btc","name":"Bitcoin","image":"https://assets.coingecko.com/coins/images/1/large/bitcoin.png?1547033579","current_price":24395,"market_cap":471024395566,"market_cap_rank":1,"fully_diluted_valuation":512524558525,"total_volume":36978174287,"high_24h":24632,"low_24h":23634,"price_change_24h":329.53,"price_change_percentage_24h":1.36929,"market_cap_change
_24h":7546377722,"market_cap_change_percentage_24h":1.62821,"circulating_supply":19299587.0,"total_supply":21000000.0,"max_supply":21000000.0,"ath":69045,"ath_change_percentage":-64.66788,"ath_date":"2021-11-10T14:24:11.849Z","atl":67.81,"atl_change_percentage":35876.02077,"atl_date":"2013-07-06T00:00:00.000Z","roi":null,"last_updated":"2023-02-23T09:39:52.666Z","price_change_pe
rcentage_1h_in_currency":-0.20401220217204966,"price_change_percentage_24h_in_currency":1.369289598442974,"price_change_percentage_7d_in_currency":0.3583399263413353},{"id":"ethereum","symbol":"eth","name":"Ethereum","image":"https://assets.coingecko.com/coins/images/279/large/ethereum.png?1595348880","current_price":1668.4,"market_cap":200921160101,"market_cap_rank":2,"fully_di
luted_valuation":200921160101,"total_volume":9504860917,"high_24h":1676.81,"low_24h":1606.05,"price_change_24h":29.63,"price_change_percentage_24h":1.80796,"market_cap_change_24h":3896119216,"market_cap_change_percentage_24h":1.97747,"circulating_supply":120486590.021382,"total_supply":120486590.021382,"max_supply":null,"ath":4878.26,"ath_change_percentage":-65.79923,"ath_date":
"2021-11-10T14:24:19.604Z","atl":0.432979,"atl_change_percentage":385231.18829,"atl_date":"2015-10-20T00:00:00.000Z","roi":{"times":90.43367600357132,"currency":"btc","percentage":9043.36760035713},"last_updated":"2023-02-23T09:39:54.245Z","price_change_percentage_1h_in_currency":-0.13139591643635884,"price_change_percentage_24h_in_currency":1.807955309763373,"price_change_perce
ntage_7d_in_currency":-0.3855028900026271},{"id":"tether","symbol":"usdt","name":"Tether","image":"https://assets.coingecko.com/coins/images/325/large/Tether.png?1668148663","current_price":1.001,"market_cap":70538269814,"market_cap_rank":3,"fully_diluted_valuation":70538269814,"total_volume":47128805831,"high_24h":1.007,"low_24h":0.994305,"price_change_24h":0.00092146,"price_ch
ange_percentage_24h":0.09215,"market_cap_change_24h":60529210,"market_cap_change_percentage_24h":0.08588,"circulating_supply":70590595922.9975,"total_supply":70590595922.9975,"max_supply":null,"ath":1.32,"ath_change_percentage":-24.41864,"ath_date":"2018-07-24T00:00:00.000Z","atl":0.572521,"atl_change_percentage":74.66842,"atl_date":"2015-03-02T00:00:00.000Z","roi":null,"last_up
dated":"2023-02-23T09:40:23.676Z","price_change_percentage_1h_in_currency":0.06640935787535794,"price_change_percentage_24h_in_currency":0.09215153666281502,"price_change_percentage_7d_in_curr
-------------------------------------------------
[115152][D][HTTPClient.cpp:408] disconnect(): tcp is closed
[115191][V][ssl_client.cpp:319] stop_ssl_socket(): Cleaning SSL connection.
[115191][V][ssl_client.cpp:319] stop_ssl_socket(): Cleaning SSL connection.
As you can see the response is an incomplete JSON and I think this may can be caused by these logs line:
Code: Select all
114828][D][HTTPClient.cpp:922] writeToStream(): read chunk len: 5756
[114857][D][HTTPClient.cpp:1399] writeToStreamDataBlock(): short write asked for 1369 but got 0 retry...
[114859][W][HTTPClient.cpp:1420] writeToStreamDataBlock(): short write asked for 1369 but got 0 failed.
[114865][W][HTTPClient.cpp:1469] returnError(): error(-10): Stream write errorThis happen randomly and after a week of debugging I'm still struggling with this.
Many thanks for your help.