Search found 3 matches
- Fri Dec 22, 2023 7:05 am
- Forum: ESP-IDF
- Topic: ESP-IDF precision of double limited to 7 decimal places?
- Replies: 6
- Views: 2834
Re: ESP-IDF precision of double limited to 7 decimal places?
I think the problem is not the decimal places but the total digits involved (integer part plus decimal part) that seams to be limited to 11 or 12 in total. I do not know about IDF but using Arduino IDE I had the following results: float QI=17.0/3.0; float RI=999999.999999/3.111; void setup() { Seri...
- Fri Dec 22, 2023 7:04 am
- Forum: ESP-IDF
- Topic: ESP-IDF precision of double limited to 7 decimal places?
- Replies: 6
- Views: 2834
Re: ESP-IDF precision of double limited to 7 decimal places?
Check what the actual value of your double variable is. The base-10 string output may be truncated. Any tips on how to do this? I am already printing long float, and I also sent this value in a http REST response to a debugging portal I built and it has the same result of truncated decimal places. ...
- Thu Dec 21, 2023 7:20 am
- Forum: ESP-IDF
- Topic: ESP-IDF precision of double limited to 7 decimal places?
- Replies: 6
- Views: 2834
ESP-IDF precision of double limited to 7 decimal places?
Dear all, I am working on a project that involves handling latitude and longitude information, and require a precision of at least 8 decimal places. It seems like doubles are strictly limited to 7 decimal places in ESP-IDF? Is there anyway to bypass this? In the following code, 1: we take in a latit...