BUG: Provide makefile-based pythonless C/C++ development environment

[powerbroker]

Hi,

starting with bare-metal C/C++ ESP32 platform i am forced to wonder: why poothon-based?


having some experience with AVR, PIC and STM32, i was near in happy unawareness of poothon at all. the only rare cases i faced poothon was incompatibility issues of some strong poothon-depended software because of last poothon versions fail under OSes like Windows 7 and earlier(which are still actual in bare-metal context)...

but none of them - AVRs, STMs and PICs - had even a shadow of room for poothon dependency: just GCC + makefiles and a little native code for flashing/debugging. there is a common GCC and makefile based process of bare-metal development, so adding one more platform to it is never a headache. there are several IDEs supporting this way pretty good(Eclipse too, guys). just reuse tools and approaches already available, why not?!

but reinventing the wheel with poothon... agrrrrh!!!

[ESP_Sprite]

We're actually not reliant on Python for the build system itself - idf.py is simply a convenient wrapper around CMake. However, a lot of tools that aren't easily replaced by a Makefile are written in Python - for instance esptool.py, espefuse.py, the tool generating the partition table, size reporting tools, the serial monitor etc. We specifically chose Python for this as it has the largest chance of running on any host architecture and operating system; you'll note that most other SDKs either do not have the functionality that ESP-IDF does or only run on e.g. Windows on x86.

In general, we feel this is a good choice: we have encountered many more people who are happy with the added functionality, than people who dislike the choice of language so much that they feel the need to invent childish nicknames.

[ESP_Roland]

Suggestion poster here as well: https://github.com/espressif/esp-idf/issues/13839

[powerbroker]

We're actually not reliant on Python for the build system itself - idf.py is simply a convenient wrapper around CMake. However, a lot of tools that aren't easily replaced by a Makefile are written in Python - for instance esptool.py, espefuse.py, the tool generating the partition table, size reporting tools, the serial monitor etc. We specifically chose Python for this as it has the largest chance of running on any host architecture and operating system; you'll note that most other SDKs either do not have the functionality that ESP-IDF does or only run on e.g. Windows on x86.

In general, we feel this is a good choice: we have encountered many more people who are happy with the added functionality, than people who dislike the choice of language so much that they feel the need to invent childish nicknames.

guys, all this sounds great of course, if only a couple of little things:

1. i've bought the chips and don't feel i've got good enough for my money. i don't feel it even if someone of you dislike that.
2. your IDE is Eclipse-based.
   that means what? - out-of-the-box java. so, why not java - already available and not-as-challenging-as-poothon in running on different platforms?
   that way it would be beautiful: everywhere the IDE and compiler can run the ESP32 code can be built and programmed

or i am wrong?

BTW, is there any easy way to create ESP32 project in eclipse as usual C project? or, maybe, not usual, but ARM or so...

[ESP_Sprite]

i've bought the chips and don't feel i've got good enough for my money. i don't feel it even if someone of you dislike that.

Okay, but the fact that you *feel* like you're entitled to a Python-less IDE/SDK is hardly a business reason to change things on our side. As a fellow human being, I'm truly sorry you feel like this as I know from experience a surprise encounter with something that frustrates you is not a nice experience. However, as an Espressif employee, one person feeling like they don't get their moneys worth is not a reason to rewrite all the tools in our SDK. (Moreover: If you have things you dislike in an IDE so much that they're showstoppers for you, you could have easily downloaded the IDE and SDK beforehand to see if they contain anything you dislike, and even now you can probably return the chips to get your money back.)

your IDE is Eclipse-based.
that means what? - out-of-the-box java. so, why not java - already available and not-as-challenging-as-poothon in running on different platforms?
that way it would be beautiful: everywhere the IDE and compiler can run the ESP32 code can be built and programmed

We don't have *one* IDE you're forced to use. The base building block is ESP-IDF, and you can use that combined with Eclipse, Visual Studio, CLion, Visual GDB, or even use Vim (or any other text editor) and the command line to build things. Note that none of those, aside from Eclipse, are based on Java. (Plus, as runtimes go, I'm sure we'd get at least twice the complaints writing our tools in Java than in Python, given the reputation Java still has amongst some people.)

[MicroController]

Now I'm confused.
Did you or did you not get IDF with Python running?
If not, you could describe your problems and we could try and help. If yes, ... ok.

having some experience with AVR, PIC and STM32, i was near in happy unawareness...

Well, the ESPs are quite different beasts than what you've used before. Just to give you a heads-up, once you start programming the ESPs and notice that you cannot reasonably work with them without some understanding of multi-tasking concepts and concurrent programming (FreeRTOS) you may soon feel that Python hidden away in the build system is the very least of your concerns.

[powerbroker]

dear friend,

Okay, but the fact that you *feel* like you're entitled to a Python-less IDE/SDK is hardly a business reason to change things on our side. As a fellow human being, I'm truly sorry you feel like this as I know from experience a surprise encounter with something that frustrates you is not a nice experience. However, as an Espressif employee, one person feeling like they don't get their moneys worth is not a reason to rewrite all the tools in our SDK. (Moreover: If you have things you dislike in an IDE so much that they're showstoppers for you, you could have easily downloaded the IDE and SDK beforehand to see if they contain anything you dislike, and even now you can probably return the chips to get your money back.)

here i have some good news for you as Espressif employee

having this experience, next time starting a project i'll *buy* something else but *not* ESP32s. and now i proceed my way: i'll finally get the result with ESP32s already bought - it seems *with limitations* it's possible. this way my words 'it sucks' definitely have weight, since they're based on real experience: i can give examples and explain where it sucks, how and why. and people around are able to think substantively: ESP32 or something else.

in other words, the fact your customer *feels* bad about your "product"(well, not the "product" itself, but the support quality) exactly *is* the *business reason* to do things better: bad feelings have strong tend to turn into low sales. as Espressif employee you can easily see the logic: low sales => low salaries.

past years, when you could loan with lower and lower interest rate each time(so each new loan allowed you to pay previous one) customers with heir *feelings* was definitely nothing to worry about - the "profit" came from bank, but not from your production achievements and customers satisfaction. but now - in the new suck-o-sales era had become - such customer's *feelings* definitely worth attention.

We don't have *one* IDE you're forced to use. The base building block is ESP-IDF, and you can use that combined with Eclipse, Visual Studio, CLion, Visual GDB, or even use Vim (or any other text editor) and the command line to build things. Note that none of those, aside from Eclipse, are based on Java. (Plus, as runtimes go, I'm sure we'd get at least twice the complaints writing our tools in Java than in Python, given the reputation Java still has amongst some people.)

well, it's so interesting so i couldn't help researching dependencies of the mentioned IDEs:

• Visual Studio does not depend on poothon. AFAIK, it's a native windows application.
• CLion does not depend on poothon too. moreover, it depends on JetBrainsRuntime, which uses - surprise! surprise! - *Java* Chromium Embedded Framework.
• Visual GDB... looks like it also is a native windows application, as Visual Studio. there are no poothon in it's requirements, but some mentions that it *can manage* poothon installations, if any.

well, what exactly is so bad in java - strongly used by CLion and Eclipse, where poothon wins as much as it worths the additional dependency with all it's limitations?

[ESP_Sprite]

well, what exactly is so bad in java - strongly used by CLion and Eclipse, where poothon wins as much as it worths the additional dependency with all it's limitations?

I'm not saying Java is bad - I currently have no opinion on that. I'm saying there's enough people out there who do think it's bad and will complain in a similar fashion as you do now, and I'm also saying that your argument of 'eclipse comes with java so you should use that' does not hold water.

in other words, the fact your customer *feels* bad about your "product"(well, not the "product" itself, but the support quality) exactly *is* the *business reason* to do things better: bad feelings have strong tend to turn into low sales. as Espressif employee you can easily see the logic: low sales => low salaries.

You are absolutely right. The other side of the coin is that you cannot please everyone: some people want a Python-less environment, some people will want to be able to run ESP-IDF on a RiscV SOC or their FreeBSD-based PowerPC workstation; some people will want to use Visual Studio, some people want command-line integration. Some people want to integrate the ESP-IDF utilities in their automation workflow as well, and they'll have a hard time if we don't write those utilities in whatever language they use for their automation, whatever language we pick.

So ideally, we'll make the greatest common denominator of people happy with the least amount of effort on our side (so we can spend more effort on making our chips and software better in other ways). Using Python is one of those ways: it's perfectly adequate for the large majority of people, and it makes our lives easier so we can spend time on other things. Again, there's no way to make 100% of the people happy without an infinite amount of money and developer time: for instance, if we were to make a Python-less SDK, we'd need to take people away from other projects to develop that, meaning bugs won't be fixed and no advances would be made there. That would make others unhappy, meaning we still can't get to that 100%. So in conclusion: we need to strive to get most people (but not all!) happy for the least amount of work, and in this particular case, bits of the SDK being in Python helps us with that.

To put it very directly: you as a person do not fall in the 'most people' segment here. I'm gonna be very blunt: you're the very first person with such an amount of hate for Python I've ever encountered in the half a decade or so we've been integrating Python into our SDK. Even if you are the most vocal 1% of a group that thinks like you but stays silent, that is only a hundred people. As an indication: the ESP-IDF SDK has around 12K stars, so even compared to the amount of devs starring the esp-idf repo, you're only representing around 0.08%, and I think those calculations are very generous. On the other hand, it's not like you're asking for anything trivial: nothing less than the entire rewrite of the SDK and tools into anything that is not Python will do.

Again, I'm gonna be very blunt: with the incentives you put on the table (which at its worst seems to be 'I won't use my ESP32 unless you get rid of Python'), that is not gonna happen. If you feel like you can't use the ESP32 for that reason, then I will respect that and I'll honestly hope you'll have better luck with the SDKs our competitors provide.

[powerbroker]

Now I'm confused.
Did you or did you not get IDF with Python running?
If not, you could describe your problems and we could try and help. If yes, ... ok.

i did.

but it's not OK: even if we can develop at home, my team cannot do it in the office on office PCs now. our testers cannot just test - we all have to set up(and learn) linux to be able to build and flash as a team, not as group of individuals working from home for near one goal...

would be great if we can set up usual eclipse C/C++ projects for ESP32 development(ideally like ARM one, with all it's options) and set up a poothon-free build procedure(s) - so for development and test benches available hardware with pre-installed and configured Win7x64(and unsupported by Win10-11 without shamanic dances) can be used. there is a strong feeling that nothing except the poothon prevents from compiling under Win7x64, as well as flashing.

Well, the ESPs are quite different beasts than what you've used before. Just to give you a heads-up, once you start programming the ESPs and notice that you cannot reasonably work with them without some understanding of multi-tasking concepts and concurrent programming (FreeRTOS) you may soon feel that Python hidden away in the build system is the very least of your concerns.

well, still don’t see the relationship between poothon-based build procedure and concurrent threads running on MCU being programmed.
it may be wonderful: e.g. AtomThreads runs good enough on AVRs with 16K flash and 1K RAM, and mentioned FreeRTOS - absolutely fine on fat STM32s(didn't try any of that on PICs - they truly suck).
of course, all examples with no poothon at all.

BTW, why does Eclipse hung building yet simple ESP32 UART-SPI-PWM demo immediately after IDE startup on more than enough RAM and CPU power?(need to leave IDE alone for about 3 minutes to be able to build successfully)

multitasking most likely. REALLY BAD multitasking...

[powerbroker]

You are absolutely right. The other side of the coin is that you cannot please everyone: some people want a Python-less environment, some people will want to be able to run ESP-IDF on a RiscV SOC or their FreeBSD-based PowerPC workstation; some people will want to use Visual Studio, some people want command-line integration. Some people want to integrate the ESP-IDF utilities in their automation workflow as well, and they'll have a hard time if we don't write those utilities in whatever language they use for their automation, whatever language we pick.

So ideally, we'll make the greatest common denominator of people happy with the least amount of effort on our side (so we can spend more effort on making our chips and software better in other ways). Using Python is one of those ways: it's perfectly adequate for the large majority of people, and it makes our lives easier so we can spend time on other things. Again, there's no way to make 100% of the people happy without an infinite amount of money and developer time: for instance, if we were to make a Python-less SDK, we'd need to take people away from other projects to develop that, meaning bugs won't be fixed and no advances would be made there. That would make others unhappy, meaning we still can't get to that 100%. So in conclusion: we need to strive to get most people (but not all!) happy for the least amount of work, and in this particular case, bits of the SDK being in Python helps us with that.

To put it very directly: you as a person do not fall in the 'most people' segment here. I'm gonna be very blunt: you're the very first person with such an amount of hate for Python I've ever encountered in the half a decade or so we've been integrating Python into our SDK. Even if you are the most vocal 1% of a group that thinks like you but stays silent, that is only a hundred people. As an indication: the ESP-IDF SDK has around 12K stars, so even compared to the amount of devs starring the esp-idf repo, you're only representing around 0.08%, and I think those calculations are very generous. On the other hand, it's not like you're asking for anything trivial: nothing less than the entire rewrite of the SDK and tools into anything that is not Python will do.

Again, I'm gonna be very blunt: with the incentives you put on the table (which at its worst seems to be 'I won't use my ESP32 unless you get rid of Python'), that is not gonna happen. If you feel like you can't use the ESP32 for that reason, then I will respect that and I'll honestly hope you'll have better luck with the SDKs our competitors provide.

dear friend,

you caught the thing: i'm one who "with such an amount of hate for python" didn't just say once "f@ck-it", but despite the "hatred for python" gives a chance to the platform.

well... disassembling some fridge thermostat i see PIC in it. in universal battery charger "imax b6" - AVR or even MCS51. in some powerful UPS - PIC again. in receipt printer with fiscal device - NEC850 + AVR. in cash register - STM32. in laser printer - NXP LPC. in intercom and access control system - PICs and AVRs. in electric scooter - STM8 and STM32. in power meter - PIC. in JTAG adapter - STM32 or LPC...

you know where i saw ESP32? inside a "smart-ass" electrical outlet - with only feature: enable/disable remotely! no timer, no power measurements, no overload protection - nothing, just on and off from android app.
is ESP32 hardware too frail for e.g. power meter, intercom, UPS, battery charger or cash register? with all it's UARTs, ADC/DAC, PWM, SPI/I2C/..., on-board RAM volume, running frequency - well, 5-volt-tolerant 20-mA+ capable GPIOs would be really nice here, but it's not a blocker - more than good enough. definitely.

but we don't see it all there. why?

you've probably noticed: while CPU power and storage capabilities grow over time, software quality slowly and inevitably decreases. i had chances to see with own eyes how crappy software was pouring good and even outstanding hardware down the drain. no, i am not saying "you're doomed", especially when almost everyone does that way... you only missing the opportunity to offer something outstanding, nothing more.
so, using poothons(eager C/C++ indexing, implicit headers, "waterfall" online installation, ...) is perfectly adequate for the large majority of people...

... of people, developing "smart-ass" "on/off-remotely" electrical outlets.
and the other segments in which, due to the economic situation now, ESP32 *could replace* current platforms, we simply won't consider.