Faster, optimized ESP-IDF fork + PSRAM Issues
Posted: Thu Dec 27, 2018 4:29 pm
During the development of low.js, a Node.JS port for ESP32 boards (https://www.lowjs.org/), we had a few challenges to overcome. I would like to give back to the ESP-IDF community with two things:
1. ESP-IDF modified to use dlmalloc
The default memory allocator in ESP-IDF is self made by Espressif (at least so it seems). It is not very fast, and becomes very slow when memory gets fragmented. This problem becomes evident when using SPI RAM.
esp-idf-dlmalloc is a fork of ESP-IDF which was modified to use dlmalloc, an industry standard memory allocator. It is almost twice as fast as the default memory allocator, and does not slow down notably with fragmented memory.
The fork has its own GitHub repository here: https://github.com/neonious/esp-idf-dlmalloc
Hopefully Espressif is interested to switch the memory allocator in the default branch of ESP-IDF too (and hey, Espressif, while you are here, maybe you want to take a look at low.js, might be interesting to offically back, too).
2. Still cache issues with PSRAM
This is not really a gift to the community, but might become one when our report helps fix this problem:
We stumbled upon the fact that cache issue with PSRAM still exist, even in the newest development environment. This can produce random crashes, even if the code is 100 % valid.
You can find the project showcasing the problem here: https://github.com/neonious/memcrash-esp32
Thomas
1. ESP-IDF modified to use dlmalloc
The default memory allocator in ESP-IDF is self made by Espressif (at least so it seems). It is not very fast, and becomes very slow when memory gets fragmented. This problem becomes evident when using SPI RAM.
esp-idf-dlmalloc is a fork of ESP-IDF which was modified to use dlmalloc, an industry standard memory allocator. It is almost twice as fast as the default memory allocator, and does not slow down notably with fragmented memory.
The fork has its own GitHub repository here: https://github.com/neonious/esp-idf-dlmalloc
Hopefully Espressif is interested to switch the memory allocator in the default branch of ESP-IDF too (and hey, Espressif, while you are here, maybe you want to take a look at low.js, might be interesting to offically back, too).
2. Still cache issues with PSRAM
This is not really a gift to the community, but might become one when our report helps fix this problem:
We stumbled upon the fact that cache issue with PSRAM still exist, even in the newest development environment. This can produce random crashes, even if the code is 100 % valid.
You can find the project showcasing the problem here: https://github.com/neonious/memcrash-esp32
Thomas