JavaScript on ESP32 - Duktape

[kolban]

Ive been tinkering with an implementation of a JavaScript interpreter called Duktape and it ported instantly to the ESP32. Very nice. Duktape can be found here:

http://duktape.org/

Its been an interesting journey so far. I started with Espruino JavaScript and that took "moderate work" to get running on ESP32. Then I was surprised to find JerryScript JavaScript and that ported with low work and seemed better to work with in our ESP32 environment. Now I've found Duktape and that took almost no work and may even be better than JerryScript for my needs.

A wealth of riches indeed. However its causing me to lay awake at night wondering about all this duplication of project efforts. The goal is to run JavaScript and we now seem to have (at least) three distinct significant projects with distinct design concepts and APIs.

Assume my goal is to assist in making a JavaScript environment for folks to use with less work ... how would someone like me choose which JavaScript environment to build upon?

[WiFive]

Which has best modularity and smallest memory footprint? Which has easiest c api? Which has cleanest source tree? Which has biggest developer community? Which has best oss license? Which is most stable/tested?

[kolban]

Status update: Duktape continues to be my current JavaScript engine of choice. The latest is that it now serves up a simple IDE.



We can enter scripts, run them on a click and see the console from the ESP32 all within a browser environment.

[WiFive]

Nice! So how is the memory optimization? What does it take to get an out of memory error?

[kolban]

With no optimization attempts and no examination of how the 512K is used, the ESP32 heap reports at about 110K with no scripts loaded. But RAM availability is still a mystery to me as a freshly booted ESP32 using a minimal ESP-IDF applications seems to report only about 150K free.

The complete flashable image clocks in at about 600K at this point including Mongoose web server and CURL HTTP client libraries. So tones of flash remaining.

[WiFive]

With no optimization attempts and no examination of how the 512K is used, the ESP32 heap reports at about 110K with no scripts loaded. But RAM availability is still a mystery to me as a freshly booted ESP32 using a minimal ESP-IDF applications seems to report only about 150K free.

The complete flashable image clocks in at about 600K at this point including Mongoose web server and CURL HTTP client libraries. So tones of flash remaining.

Yes overall ram optimization is supposedly still pending. So what is stack size you are using for duktape and how is its heap managed? How many js objects can be created? Are there any ram saving options in duktape?

Would love to see some kind of chart comparing js engines.

[kolban]

Howdy,
Finally got to reading some about the Duktape engine memory management and it looks quite promising and especially well documented see:

http://duktape.org/guide.html#memoryusage

The summary is that there is no specific heap for JS objects ... so the allocation comes from the normal C app heap. JS code is turned into bytecode which can be placed in flash and hence the size of the JS script itself doesn't appear to penalize the amount of storage available for data.

There also appears to be a raft of bewildering options that has good defaults:

https://github.com/svaarala/duktape/blo ... memory.rst

These seem to say that we can choose to exchange memory usage for performance on many features.

[kolban]

A very early binary (flashables) are now available for download with some basic instructions.

See:

https://github.com/nkolban/duktape-esp3 ... /README.md

[kolban]

2016-12-14: Status report on JavaScript on ESP32 story
Here is a current status report. The state of experiments with JavaScript on ESP32 makes leaps and bounds. A very interesting and important change has occurred that I'd like to tell you about.

We have a number of great JavaScript engines available for ESP32 today (as of this post date) including Duktape and JerryScript. I am focusing on Duktape just now but read on ...

Getting a raw JS environment working on ESP32 was not at all difficult. Duktape ran first compile and JerryScript took only a few hours. However, having a raw JavaScript interpreter, while great ... isn't that interesting to us in ESP32 land. In an ideal world, we would need full networking, web servers, MQTT, WebSockets, WiFi, BlueTooth and all the electronics protocols such as I2C, SPI etc. For the rest of this story, I'll call this "the framework". This is the layer of classes and libraries that are available to JavaScript authors to write JS applications that bring ESP32 to life. My first design pass at this was to source these functions as a variety of C libraries such as Mongoose for HTTP, Curl for outbound REST, and so on. As I started implementing this "hodge-podge" of parts, progress was solid and steady but they weren't lending themselves to the nature of JavaScript. When one is a JavaScript programmer, almost everything is asynchronronous in nature with registered callbacks.

For example (and this is pseudo code), to make a REST call in JavaScript one would *like* to call:

Code: Select all

var HTTP = require("http");
HTTP.request("http://myrestserver.example.com", function(response) {
   console.log("We got a response!");
   on("data", function(data) {
      console.log("Here is the data!");
   });
});

or to be a REST server, one might want:

Code: Select all

var HTTP = require("http");
var server = HTTP.createServer(function(request, response) {
   console.log("We got a new request");
   response.write("Here is the response");
   response.end();
});
server.listen(80);

This is all normal stuff to a programmer working in JS in Node.js on a desktop. However, the C libraries don't like to play in this
callback fashion. Most of them like to block until the work is done. This is fine in a multi-tasking environment but embracing JS and C level multitasking starts to get "weird". In addition, the sheer complexity starts to multiply as we are continually bridging from JavaScript to C and back-end. It wasn't going to be pretty.

This was when a new notion came to light. What if the framework were itself written in JavaScript? What if the HTTP calling, HTTP server, File I/O, electronics access and more were written in 100% JavaScript and run by the JavaScript engine of choice? (Duktape OR JerryScript). Could that be done? I looked at Node.JS and found that they didn't do that ... they wrote 100% of their interfacing in C so that wouldn't help as an Open Source base. So I asked myself ... what would the absolutely must have primitives be for a framework for JS written in JS itself? So far the list is small ... basically the sockets API (socket, listen, accept, close, send, recv, select) and the POSIX file I/O (open, close, read, write, stat). The electronics will also map to GPIO, SPI and I2C. From there ... we can build EVERYTHING on top of that.

So I spent a few evenings working on it ... and darn if it isn't functional. So ... we now have an HTTP stack written in JS that assumes only sockets APIs and a sockets stream abstraction beneath that. Its all written 100% in JS.

And Ive saved the best to last. Since the framework is in JavaScript and the framework is itself JS source files contained in the flash of the ESP32 which can be read and written through SPIFFS file system ... and we have a browser based IDE. The development of the JS framework for ESP32 is being done ON the ESP32 in JavaScript. No flashing is needed (after initial environment load) for the majority of changes. It's got a nice feel to it.

[shadow]

Very cool
I'm still trying to figure out how to program my board so a JS engine looks very nice.

At the same time I was reading about frontend choices for the associated cloud project and I've read a bit about ES5/ES6 and TypeScript and reactive programming, which also look cool, getting some type safety and replacing callbacks with promises, or, more recently, reactive code.
I don't know how appropriate these would be for an embedded environment...