ESP32 RMT Issues: Passing large buffer to rmtWrite causes crash

[teknynja]

While attempting to use the Adafruit NeoPixel Library v1.12.2, I discovered that trying to drive more than 70+ pixels will cause the system to crash/boot-loop. I've verified this happens on both my ESP-WROOM-32 and ESP32-C3 boards. I found an issue on the Adafruit repository (https://github.com/adafruit/Adafruit_Ne ... issues/375) and posted details of my findings there. I suspect that it's not uncommon to need to drive 150 pixels (a standard full strip) or more, so the limitation of 70 poses quite a restriction.

Details:
* Arduino IDE 2.3.2
* Board Package: esp32 by Espressif v3.0.1
* Adafruit NeoPixel Library v 1.12.2
* ESP32-S3 and ESP32-C3 chips

When building with the ESP IDF v5.x.x, the Adafruit library simply invokes `rmtInit` then `rmtWrite` in it's driver during the `show()` method. When invoking the `rmtWrite` method with a large buffer, the chip crashes and reboots. I suspect there is some kind of buffer overrun occurring when more than around 1600-1700 RMT Items are passed to the `rmtWrite` method. I assume that there must be some kind of buffer managment going on behind the scenes, since that number of items is far larger than a single channel's block memory (or even all the channels combined). It is interesting to note that size limit is approximately the same on both the S3 vs the C3 chips (even though the C3 has much smaller block memory).

Ideally, not only should the crash be prevented for large item arrays, but also there needs to be a way to handle very large item buffers so that applications using 150, 300, or even 500 WS2812 devices can take advantage of the RMT hardware. Please let me know if there is any additional information or details I can provide to help resolve this issue. Thanks!

[teknynja]

Sample code to demonstrate issue:

Code: Select all

/*
    Sample code to demonstrate rmtWrite crash with large buffers

    This code is based on the ESP32 RMT Driver from the Adafruit
    NeoPixel Library at:

      https://github.com/adafruit/Adafruit_NeoPixel/blob/master/esp.c

    which itself seems to be directly based on the Espressif example
    code found at:

      https://docs.espressif.com/projects/arduino-esp32/en/latest/api/rmt.html
*/

// Setting this to > ~75 causes rmtWrite to crash
// Setting below < ~70 works
#define PIXEL_COUNT   100
#define PIXEL_PIN     12

#define ITEM_COUNT    (PIXEL_COUNT * 24)

void setup() {
  Serial.begin(115200);
  delay(1000);

  Serial.println("Demonstrate rmtWrite crash with large buffers");
  Serial.print("Chip: ");
  Serial.print(ESP.getChipModel());
  Serial.print(" Rev ");
  Serial.print(ESP.getChipRevision());
  Serial.print(", Arduino Core v");
  Serial.print(ESP.getCoreVersion());
  Serial.println();

  if (!rmtInit(PIXEL_PIN, RMT_TX_MODE, RMT_MEM_NUM_BLOCKS_1, 10000000)) {
    Serial.printf("Failed to init RMT TX mode on pin %d\r\n", PIXEL_PIN);
    while (true) delay(10000);
  }
}

void loop() {
  rmt_data_t items[ITEM_COUNT];

  for (int i = 0; i < ITEM_COUNT; i++) {
    items[i].level0 = 1;
    items[i].duration0 = 4;
    items[i].level1 = 0;
    items[i].duration1 = 8;
  }

  Serial.printf("Sending %d items...\r\n", ITEM_COUNT);
  rmtWrite(PIXEL_PIN, items, ITEM_COUNT, RMT_WAIT_FOR_EVER);

  delay(1000);
}

And here's the output of the crash:

Code: Select all

Demonstrate rmtWrite crash with large buffers
Chip: ESP32-D0WD Rev 301, Arduino Core v3.0.1
Guru Meditation Error: Core  1 panic'ed (Unhandled debug exception). 
Debug exception reason: Stack canary watchpoint triggered (loopTask) 
Core  1 register dump:
PC      : 0x400d172f  PS      : 0x00060836  A0      : 0x800d372f  A1      : 0x3ffafcf0  
A2      : 0x0000018c  A3      : 0xffffda80  A4      : 0x00000001  A5      : 0xffffffff  
A6      : 0xffffffff  A7      : 0xb33fffff  A8      : 0x00000960  A9      : 0x3ffb0320  
A10     : 0x00088004  A11     : 0x3ffb8188  A12     : 0x3ffb89fc  A13     : 0xfffffff0  
A14     : 0x3ffc1170  A15     : 0x0000000c  SAR     : 0x0000000c  EXCCAUSE: 0x00000001  
EXCVADDR: 0x00000000  LBEG    : 0x400d171c  LEND    : 0x400d1733  LCOUNT  : 0x000007d3  


Backtrace: 0x400d172c:0x3ffafcf0 0x400d372c:0x3ffb2290




ELF file SHA256: f8b8be2d5fc6043e

Rebooting...

[ESP_Sprite]

You're allocating the 'items' array on the stack of the 'loop' task, and that task only has a fixed amount of stack: if your array is larger than that stack size, you'll exceed that and you get this crash. Either make the array global (by placing it outside of the loop function) or allocate it dynamically (by e.g. using calloc()).

(Hint for next time: try to decode that backtrace. If you had, you'd've seen that the crashing instruction is one of the ones in the for loop in the 'loop' function that initializes the 'items' array, rather than somewhere inside rmtWrite.)

[teknynja]

That fixes it! (at least in the demo code above). I plan to take look at Adafruit's code to see how they are allocating the "items" buffer (their API does all the memory allocations internally, but I do instantiate their driver instance outside of any functions). I'm presuming they too are allocating the item's buffer on the stack, but need to verify.

Sorry about missing the backtrace, I'm not experienced in analyzing post-mortem data...

Thank you for you quick reply and for showing me the error of my ways!

[degnor]

This appears to be the relevant issue with the Adafruit library: https://github.com/adafruit/Adafruit_Ne ... issues/375