Search found 3 matches

by xwbxwb
Fri Dec 22, 2023 7:05 am
Forum: ESP-IDF
Topic: ESP-IDF precision of double limited to 7 decimal places?
Replies: 6
Views: 2448

Re: ESP-IDF precision of double limited to 7 decimal places?

I think the problem is not the decimal places but the total digits involved (integer part plus decimal part) that seams to be limited to 11 or 12 in total. I do not know about IDF but using Arduino IDE I had the following results: float QI=17.0/3.0; float RI=999999.999999/3.111; void setup() { Seri...
by xwbxwb
Fri Dec 22, 2023 7:04 am
Forum: ESP-IDF
Topic: ESP-IDF precision of double limited to 7 decimal places?
Replies: 6
Views: 2448

Re: ESP-IDF precision of double limited to 7 decimal places?

Check what the actual value of your double variable is. The base-10 string output may be truncated. Any tips on how to do this? I am already printing long float, and I also sent this value in a http REST response to a debugging portal I built and it has the same result of truncated decimal places. ...
by xwbxwb
Thu Dec 21, 2023 7:20 am
Forum: ESP-IDF
Topic: ESP-IDF precision of double limited to 7 decimal places?
Replies: 6
Views: 2448

ESP-IDF precision of double limited to 7 decimal places?

Dear all, I am working on a project that involves handling latitude and longitude information, and require a precision of at least 8 decimal places. It seems like doubles are strictly limited to 7 decimal places in ESP-IDF? Is there anyway to bypass this? In the following code, 1: we take in a latit...