JavaScript on ESP32 - Duktape

[kolban]

Project Status: 2016-12-31 - Bluetooth and serial interfacing

Over the last week Ive been studying Bluetooth Low Energy (BLE) and it looks good. It appears to be heavily event based and, to my way of thinking, lends itself very well to a functional programming language like JavaScript. As such, it is time to look at adding BLE support into ESP32-Duktape ... but there is a problem I foresee. It appears that switching on bluetooth is going to cost RAM ... and not a small amount of RAM ... it may be as high as 64K (of the approx 180K available to ESP32 apps). That's about 35% spent just by switching on bluetooth. In our present early Duktape state, we might not even fit!!!

Ok ... so what might be the solution? The immediate answer might be to switch off WiFi. While there might be some applications that want BOTH bluetooth AND WiFi, there will also be a large chunk of applications that want EITHER bluetooth OR WiFi but not both at the same time ... so if we can turn off WiFi in order to turn on bluetooth, that might not be so bad. Unfortunately, here is where we run-aground again. In the current design of ESP32-Duktape, we rely on WiFi network to access the ESP32 to push new scripts to it for execution. If we switch off WiFi, how then are we going to work with ESP32-Duktape? ... and that is the puzzle to be solved before we delve any further into bluetooth. My immediate thinking is to use UART. Let us assume that we don't have any networking, we will always have one or more UART interfaces. What if we "pushed" our scripts to be executed through UART to ESP32-Dukatpe? That would allow us to run with no networking at all and still be functional.

So ... that's what I'm focusing on now ... ESP32-Duktape and a 100% serial interface. While that will be useful in its own right, the bigger plan is that will be a stepping stone to bluetooth support.

[ajaymm58]

Neil, any news from the Duktape developer regarding executing bytecode directly from the Flash? It would be lot more effective to solve that issue, so that SRAM requirement will be reduced.

Additionally: I use the pre-prepared environment on MSYS32 per Step 1 of https://github.com/espressif/esp-idf/bl ... -setup.rst
Unfortunately the binary tools ~/bin/mkespfsimage and ~bin/mkspiff do not execute in that environment.
Will it be possible to provide some guidance on how to build them?
I did modify the Makefile as follows:

Code: Select all

images:
	echo "+--------------------+"
	echo "| Building espfs.img |"
	echo "+--------------------+"
	cd filesystem; find . -print | ../bin/mkespfsimage -c 0 > ../build/espfs.img
	echo "+---------------------+"
	echo "| Building spiffs.img |"
	echo "+---------------------+"
	../bin/mkspiffs -c filesystem -b 65536 -p 256 -s 524288 build/spiffs.img

[kolban]

At present, I'm going to focus on getting the core functions going before spending any efforts on polishing the development environment for enhancements. Of course, that can always change ... but before that I'd want to hear:

1. What is required from building on Windows that can't be achieved by building on Linux (even Linux on Virtual Box on Windows)?
2. What is the functional area that you are working on for ESP32-Duktape?

The ability to build ESP32-Duktape on Windows might itself be a sub-project you or another community member might like to attack?

The file systems (and hence the commands for building the file systems) are documented here:

https://github.com/nkolban/duktape-esp3 ... systems.md

As for running Duktape from flash so there is no byte code in RAM ... if it happens at all, I would not anticipate availability before summer 2017 at the earliest.

[andrew_p]

I'm really looking forward to try HTTP requests on Duktape.

How does the application part work? For example - I want to automatically run specific javascript code (application / HTTP request etc.) every time when esp32 powers on, how can I do that?

[kolban]

Howdy @andrew_p,

The high level design docs are just starting ... see: https://github.com/nkolban/duktape-esp3 ... 0design.md

However, the story is loosely as follows ... within the ESP32-Duktape environment, there is a read/write file system stored in flash. When the ESP32 boots, the application (ESP32-Duktape) starts and builds a JavaScript runtime environment. Once that is ready, it loads a JavaScript script from the on-board file system called "init.js". This provides "enough" of the JavaScript environment specific to ESP32 to start doing interesting things. Next, it looks for a JavaScript script called "start.js" and then runs that.

And that's it.

It doesn't have to be complicated. The current "start.js" causes the ESP32 to become a WebServer that serves up the ESP32-Duktape IDE to a browser and when you say you want to run a script (from the IDE), that script is sent via an HTTP PUSH and executed in a JavaScript "eval". What I'm working on now are more configuration options to the "supplied" start.js ... these use Non Volatile Storage to store settings but could just as easily use a JSON file stored in the file system. New options include the ability to NOT start a WebServer but instead watch a UART port for incoming commands/scripts. This means that we don't need to start the WiFi subsystem or networking (for example BT or no networking at all).

The majority of this project is not C programming on ESP32 at all ... but rather JavaScript frameworks to provide WebServers, Web Sockets and pass-throughs to the underlying ESP32 functions.

All of this is a technical description because at present, the cake is till in the mixing bowl and needs a lot of baking before it can be served to all ... however only by attempts to use by good folks like yourself will we know how it may taste. So feel free to ping me on IRC at #ESP32 if you can't get something working (or just post back here).

In a nutshell ... to run your own app at boot ... call your app "start.js" and put it in the file system ... and you are done.

[ajaymm58]

Neil,
Primary reason to use MSYS2 environment is the simplicity of installing it by extracting a single archive file provided and hopefully maintained by Expressif. Secondary reason is the ease of sharing the filesystem between windows environment and MSYS2. e.g. access to the filesystem is as easy as "cd /c/"
I agree that it is not a critical requirement at this time.

My interest is to evaluate if esp32-duktape can support JS programmable application layer gateway (ALG) for BLE devices to connect to IP. The other option is jerryscript and IoTJS. Hence Wi-Fi and BLE (Controller) operating concurrently is important to understand.

The reason I am interested in building from source is to get a better understanding of the code flow and memory usage.
I also want to explore if duktape bytecode files can be executed by reading them directly from the Flash (may require a different type of filesystem)

[Frida854]

When I send these 2 lines, from firmware yesterday:

Code: Select all

console.log("Hello World\n");
console.log("Settings initially: " + JSON.stringify(RMT.getState()) + "\n");

I got those errors:

Code: Select all

D (114596) modules: >> js_console_log called
D (114596) log: Console: Hello World

D (114598) log: ReferenceError: 4
D (114600) module_os: >> js_os_send

But it don't do it in the firrmware from: 'esp32-duktape-2016-12-24.tar.gz'
Just a reminder.

I have tryed the serial on 'rxPin: 15, txPin: 4', because I don't have all pins exposed, and it works.

[kolban]

Howdy,
Attention is being paid to memory usage and polish. The plan is to optimize memory usage over time. In the early releases, we literally added all modules into the namespace for JavaScript whether they were used or not. As our skills in JS are improving and techniques being learned, we are able to use the dynamic module loading capabilities better. What this means is that the "RMT" module is now no longer loaded when ESP32-Duktape starts. It will now only consume resources when needed through:

Code: Select all

var RMT = require("RMT");

The error message that is being logged is an attempt to use a variable called "RMT" which (at that point) doesn't exist. Add the above line to your script prior to referring to the RMT module and we should be back on track. LEDC/PWM is being added this week.

[ESP32_Forum]

Hello,
I am trying to flash the ESP32 with Duktape, using the binaries contained in the 12-24 zip file.
I am getting this message when rebooting:

ets Jun 8 2016 00:22:57<\r><\n>
<\r><\n>
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)<\r><\n>
ets Jun 8 2016 00:22:57<\r><\n>
<\r><\n>
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)<\r><\n>
configsip: 0, SPIWP:0x00<\r><\n>
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00<\r><\n>
mode:QIO, clock div:2<\r><\n>
load:0x3ffc0008,len:4<\r><\n>
load:0x3ffc000c,len:1544<\r><\n>
load:0x40078000,len:3408<\r><\n>
ho 0 tail 12 room 4<\r><\n>
load:0x40080000,len:260<\r><\n>
entry 0x40080034<\r><\n>
I (946) heap_alloc_caps: Initializing. RAM available for dynamic allocation:<\r><\n>
I (946) heap_alloc_caps: At 3FFC2004 len 0001DFFC (119 KiB): DRAM<\r><\n>
I (948) heap_alloc_caps: At 3FFE8000 len 00018000 (96 KiB): D/IRAM<\r><\n>
I (957) heap_alloc_caps: At 4009C2A0 len 00003D60 (15 KiB): IRAM<\r><\n>
I (965) cpu_start: Pro cpu up.<\r><\n>
I (970) cpu_start: Single core mode<\r><\n>
I (975) cpu_start: Pro cpu start user code<\r><\n>
V (1001) intr_alloc: esp_intr_alloc_intrstatus (cpu 0): checking args<\r><\n>
V (1001) intr_alloc: esp_intr_alloc_intrstatus (cpu 0): Args okay. Resulting flags 0xE<\r><\n>
D (1034) intr_alloc: Connected src 56 to int 2 (cpu 0)<\r><\n>
D (1082) nvs: nvs_flash_init_custom start=9 count=6<\r><\n>
D (1149) phy_init: loading PHY init data from application binary<\r><\n>
D (1174) nvs: nvs_open phy 0<\r><\n>
D (1198) nvs: nvs_get cal_version 4<\r><\n>
V (1228) phy_init: phy_get_rf_cal_version: 258<\r><\n>
<\r><\n>
D (1268) nvs: nvs_get_str_or_blob cal_mac<\r><\n>
D (1304) nvs: nvs_get_str_or_blob cal_data<\r><\n>
D (1345) nvs: nvs_close 1<\r><\n>
V (1359) phy_init: register_chipv7_phy, init_data=0x3f401c40, cal_data=0x3ffc2ffc, mode=0<\r><\n>
I (1564) phy: phy_version: 258, Nov 29 2016, 15:51:07, 0, 0<\r><\n>
D (1564) nvs: nvs_open phy 1<\r><\n>
V (1565) phy_init: phy_get_rf_cal_version: 258<\r><\n>
<\r><\n>
D (1574) nvs: nvs_set cal_version 4 258<\r><\n>
D (1709) nvs: nvs_set_blob cal_mac 6<\r><\n>
D (1713) nvs: nvs_set_blob cal_data 1904<\r><\n>
D (2691) nvs: nvs_close 2<\r><\n>
I (2692) cpu_start: Starting scheduler on PRO CPU.<\r><\n>
D (2693) duktape_main: Free heap at start: 208948<\r><\n>
tcpip_task_hdlxxx : 3ffc5a8c, prio:18,stack:2048<\r><\n>
D (2697) nvs: nvs_open misc 1<\r><\n>
D (2700) nvs: nvs_get_str_or_blob log<\r><\n>
I (2703) wifi: frc2_timer_task_hdl:3ffc74cc, prio:22, stack:2048<\r><\n>
D (2709) nvs: nvs_open nvs.net80211 1<\r><\n>
D (2712) nvs: nvs_get opmode 1<\r><\n>
D (2715) nvs: nvs_get country 1<\r><\n>
D (2718) nvs: nvs_get_str_or_blob sta.ssid<\r><\n>
D (2722) nvs: nvs_get_str_or_blob sta.mac<\r><\n>
D (2725) nvs: nvs_get sta.authmode 1<\r><\n>
D (2729) nvs: nvs_get_str_or_blob sta.pswd<\r><\n>
D (2732) nvs: nvs_get_str_or_blob sta.pmk<\r><\n>
D (2736) nvs: nvs_get sta.chan 1<\r><\n>
D (2739) nvs: nvs_get auto.conn 1<\r><\n>
D (2742) nvs: nvs_get bssid.set 1<\r><\n>
D (2745) nvs: nvs_get_str_or_blob sta.bssid<\r><\n>
D (2749) nvs: nvs_get sta.phym 1<\r><\n>
D (2752) nvs: nvs_get sta.phybw 1<\r><\n>
D (2755) nvs: nvs_get_str_or_blob sta.apsw<\r><\n>
D (2759) nvs: nvs_get_str_or_blob sta.apinfo<\r><\n>
D (2763) nvs: nvs_get_str_or_blob ap.ssid<\r><\n>
D (2767) nvs: nvs_get_str_or_blob ap.mac<\r><\n>
D (2770) nvs: nvs_get_str_or_blob ap.passwd<\r><\n>
D (2774) nvs: nvs_get_str_or_blob ap.pmk<\r><\n>
D (2778) nvs: nvs_get ap.chan 1<\r><\n>
D (2781) nvs: nvs_get ap.authmode 1<\r><\n>
D (2784) nvs: nvs_get ap.hidden 1<\r><\n>
D (2787) nvs: nvs_get ap.max.conn 1<\r><\n>
D (2790) nvs: nvs_get bcn.interval 2<\r><\n>
D (2793) nvs: nvs_get ap.phym 1<\r><\n>
D (2796) nvs: nvs_get ap.phybw 1<\r><\n>
D (2799) nvs: nvs_get ap.sndchan 1<\r><\n>
D (2803) nvs: nvs_set_blob sta.mac 6<\r><\n>
D (2806) nvs: nvs_set_blob ap.mac 6<\r><\n>
I (2810) wifi: pp_task_hdl : 3ffc9d28, prio:23, stack:8192<\r><\n>
D (2814) duktape_task: >> duktape_task<\r><\n>
D (2818) dukf_utils: duktape_task: heapSize=168776<\r><\n>
D (2822) duktape_spiffs: Loading SPIFFS at address 0x180000<\r><\n>
E (2827) duktape_spiffs: Error: rc from spiffs mount = -10050<\r><\n>
D (2866) dukf_utils: Heap after duk create heap: heapSize=133980<\r><\n>
D (2886) dukf_utils: Heap before after duk_module_duktape_init: heapSize=132708<\r><\n>
D (2886) espfs: rc from spi_flash_mmap: 258<\r><\n>
Guru Meditation Error of type LoadProhibited occurred on core 0. Exception was unhandled.<\r><\n>
Register dump:<\r><\n>
PC : 0x401296b6 PS : 0x00060c30 A0 : 0x80106408 A1 : 0x3ffcea10 <\r><\n>
A2 : 0x00000000 A3 : 0x00040000 A4 : 0x3f40820c A5 : 0x0000001f <\r><\n>
A6 : 0x00000001 A7 : 0x00000005 A8 : 0x801296b1 A9 : 0x3ffce9c0 <\r><\n>
A10 : 0x400fb288 A11 : 0x3ffbd4c0 A12 : 0x3f411308 A13 : 0x00000009 <\r><\n>
A14 : 0x00000040 A15 : 0x00000005 SAR : 0x00000004 EXCCAUSE: 0x0000001c <\r><\n>
EXCVADDR: 0x00000000 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xfffffffe <\r><\n>
<\n>Backtrace: 0x401296b6:0x3ffcea10 0x40106408:0x3ffcea40 0x40105224:0x3ffcea60 0x4010544b:0x3ffcea90<\n><\n>Entering gdb stub now.<\r><\n>
$T0b#e6

Will there be new binaries?

[Frida854]

I had to move RMT.js from the 'doc' folder to the 'filesystem' folder, then it works.

If I set useSerial = 1, and change rx, tx to my pins, how does the serial console work, or is it not ready yet?