[Question]: Wisdom of using ESP-IDF master as opposed to a release

[kolban]

Since I started playing with the ESP-IDF I have always been working against the Github "master" and for about 9 months now, had zero issues with that. Tonight I hit my first "puzzle" that seems to related to that practice.

The detailed technical story is at issue 857 - https://github.com/espressif/esp-idf/issues/857

But in summary, BLE server code that used to work was no longer working for me. I spent the evening studying to see where I might have gone wrong. My source code is housed on Github so I went back and started working from a version that I was very confident worked ... but now, using the current master, it doesn't.

Studying the release notes for ESP-IDF 2.1 found here (https://github.com/espressif/esp-idf/releases) I checked and checked for API changes that might be required but in every case, I seemed to have found them satisfied.

Running out of things to try, I went back in time and downloaded a month old version of ESP-IDF, cleaned and rebuilt all. My code (which used to work) now worked. I went back to master, cleaned and re-built all ... and it failed again in the same fashion as originally started me on this journey. Next I downloaded ESP-IDF Release v2.1 (released 2 days ago as per Github) ... and THIS build of ESP-IDF works. I also see that Github is reporting that master is 223 commits beyond this release.

Now ... all of the above are just numbers and raw information ... but what has me "spooked" is this first brush with "master" being potentially regressed from a release.

In my way of thinking, this should not be unexpected. Any development shop likely can't perform full regression testing everytime a commit is made. That would then explain "why" we have releases in the first place. We take a sprint of function, build it, test it and then release it. Any code that "happens" to be visible between releases "may" be broken with respect to a proper release.

And now the questions that I'd like to pose to the community:

1) What ESP-IDF do you use for building your apps? Do you work with an Espressif release or against master?
2) Assuming that the functions you are using "work" against master ... should we even be "attempting" to work against master?
3) Have you ever found master to be less than perfect in your code usage in the past? I have found things that were broken in releases but fixed in later masters (yeah!) but this is the first time I have seen the reverse.

[RRicky]

I guess we all want to use the latest greatest software release from the master, especially when the master is actively being expanded functionally. Downside, as you point out, regression is always a risk.

Due to past experiences I will always use the latest release, and occasionally check the master for bug fixes / features that are useful or needed for my product. When you cherrypick these changes from master you are forced to test the changes and you are able to estimate the impact of the change, greatly reducing the risk of regression.

Using a SDK that is actively developed, and integrating these changes "blindly" has always lead to regression, sooner or later, in my experience.

[ESP_Sprite]

With all that said, we very much would like it if people also develop or at least test with the master branch. For one, if no one does, then regressions we introduce but don't happen to test for may not get found and end up in a release. Also, if we release a new major version (version number change from x.n -> y.0) some API functions may change, and recompiling for master every now and then will help you catch occurrences of that early.