Stuttering in I2S_write

[The_YongGrand]

Hello ESP32 team,

I have encountered an issue where there is a stuttering in the I2S_write. Here are the flow of the code:

1.) On init, I2S dma buffer length = 256*2, no. of buffers = 8, I2S 16 bits.
2.) In the main task, process 256 samples of buffer that takes around 2.217ms (timed it with a scope).
3.) i2s_write called with 256 samples of that buffer.
4.) Process 2 and 3 repeats again and again.

However, after 3 seconds, the audio stuttered badly, and watchdog tripped. When I measured the I2S transmitting buffer time, it takes about 5.804ms, which is more than the processing time.

In other microcontrollers I worked with, I used a double buffering mode where two identical buffers with 256 samples being exchanged everytime the DMA block transfer complete is being set. This one works even for the PIC32 and the K210. Here in the ESP32-IDF, I have no control over what I2S_write was doing. There are no semaphores or task-notify for a transfer complete, or anything else. Once it is being sent, I do not know whether if that particular sample has being pushed or not. What I have seen during the init was the I2S functions already started pushing data even when the buffers are empty.

Is there an example of the I2S DMA with the simple double-buffering method, or other suggestions on how to effectively use the I2S_write?

[ESP_Sprite]

The I2S driver already implements a more generic version of the double-buffer idea you have; in this case the driver uses 8 buffers of 256 bytes each. The hardware will read these buffers in a round-robin fashion, and as soon as a buffer is read out, the driver will mark it as empty. The I2S_write routine will try to fill up all non-empty buffers, or will block ('wait') until one is free if all are filled. This way, you essentially have a buffer of at minimum 256*7=1792 samples as a buffer when i2s_write cannot be called for a while.

Now to guessing what causes the dropout... I'm not aware of the sample rate you're using, but say you're using 44KHz... then those 7 buffers will only last you 20mS or so. One of the reasons that most tasks stop is because of a flash erase/write cycle, and depending on the flash chip and other circumstances, these can take up to half a second, absolute worst case scenario. During that time, the buffers won't get written and the I2S peripheral will empty them all, causing the dropout you heard.

[The_YongGrand]

Hello Sprite,

Yes, my sample rate is 44.1kHZ as default.

The part is, after init the i2s module, and afterwards generating the samples and pumping the buffers into the i2s transmitter, the stuttering comes by after around 3 seconds. Even worse, the watchdog kept tripping non-stop too.

I have measured earlier that the time of generating samples is less than the time to push the samples to the i2s.

For the flash erase or write cycle, I believe that I have flashed the entire program into the Esp32 before the program played the music. I'm curious on how this would affect the i2s operation too.

I will put up that code snippet when I'm at my desk.

[The_YongGrand]

Here is a code snippet:

Code: Select all

static const i2s_config_t i2s_config = {
         .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_TX),
         .sample_rate = 44100,
         .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
         .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
         .communication_format = (i2s_comm_format_t)(I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
         .intr_alloc_flags = 0, // default interrupt priority
         .dma_buf_count = 8,
         .dma_buf_len = 256*2,
         .use_apll = false,
         .tx_desc_auto_clear = true

Code: [Select all] [Expand/Collapse]

1. void mainTask(void* pvParameter) {
2.     uint32_t bytesWritten = 0;
3.    
4.     memset(audio_buffer, 0x00, sizeof(audio_buffer));
5.     gpio_set_direction(GPIO_NUM_17, GPIO_MODE_OUTPUT);
6.     gpio_set_direction(GPIO_NUM_16, GPIO_MODE_OUTPUT);
7.     gpio_set_level(GPIO_NUM_16, 0);
8.     gpio_set_level(GPIO_NUM_17, 0);
9.    
10.     //vTaskDelay( 50 / portTICK_PERIOD_MS );
11.  
12.     while (1)
13.     {
14.         processSamples(sizeof(audio_buffer) / sizeof(int16_t), audio_buffer); // total time to process 256 samples: 2.218ms
15.         i2s_write(I2S_NUM_0, audio_buffer, sizeof(audio_buffer), &bytesWritten, portMAX_DELAY); // total time to stream 256 samples: 5.804ms
16.     }
17. }

GeSHi © Codebox Plus Extension

This is as simple as it gets - I took this from another example in the ESP32 IDF github. Adding anything after the i2s_write, especially delay wouldn't fix that either.

It also worked with my other example there, but the processing of the samples was not that complicated (the processing of the samples in the other example is simpler and shorter): https://github.com/uncle-yong/esp32-i2s-1. Unfortunately, that method doesn't work here.

Any ideas to get around it? And if this method in the code snippet doesn't work there, how could the ESP32 process a large block of, example, MP3 samples and then pump it into the I2S without jumping or stuttering?

P.S: About that,

One of the reasons that most tasks stop is because of a flash erase/write cycle, and depending on the flash chip and other circumstances, these can take up to half a second, absolute worst case scenario. During that time, the buffers won't get written and the I2S peripheral will empty them all, causing the dropout you heard.

, I have tried this on another ESP32 board and still the same result. I suspect the ESP32 isn't anymore write or erasing anything when the audio is playing.

[ESP_Sprite]

To be fair, the watchdog thing is weird and unexpected... does the watchdog error itself tell you more? Could it be that your sound generation code ends up in a big loop or something?

[The_YongGrand]

Hello Sprite,

Here's that message about the watchdog:

[0;31mE (8358) task_wdt: Task watchdog got triggered. The following tasks did n
ot reset the watchdog in time:[0m

[0;31mE (8358) task_wdt: - IDLE0 (CPU 0)[0m

[0;31mE (8358) task_wdt: Tasks currently running:[0m

[0;31mE (8358) task_wdt: CPU 0: mainTask[0m

[0;31mE (8358) task_wdt: CPU 1: IDLE1[0m

By the way, that particular sound generation code is MicroDexed: https://github.com/dcoredump/MicroDexed
This code is reported to be functioning well in Teensy 3.6, and I believe that the ESP32 has much more horsepower than the Teensy ones. Also, I'm using 240MHz and optimization at O3, so I'm not even sure why it wouldn't work well on that platform. Moreover, I have also replaced these ARM-specific "saturated add" with the ESP32 specific instructions, so I have bypassed some of the unoptimized code.

Kicking the watchdog in between the sound generation code and the i2s_write didn't work either. The very strange part is, the audio plays well for 3 seconds until the whole thing stuttered, even up to the point that the sound wouldn't play anymore.

The whole program is actually a small FM synth player playing some music. Here are the task roster:
1.) mainTask, core 0 = generate samples and push samples thru I2S_Write in a loop.
2.) playTask, core 1 = sequencer - interrupts every 1ms to switch notes and timings if there is a need to. If you are not sure of how it works, think of it as a MIDI sequencer.

Let me know if you need more info.

[ESP_Sprite]

So, what the watchdog timeout indicates is that *something* in the main task is using up CPU time for a long time (default is 5 seconds) without yielding. Normally, a call to i2s_write would do that: if all buffers are full, the call blocks and yields time to e.g. the idle task. The watchdog timeout indicates this does not happen; this could be because 1. i2s_write somehow fails (which imo is unlikely, as the i2s driver is used my many other projects internal and external to Espressif), or 2. your sound generation code loops.

I'm not sure how you 'kick' the watchdog, but if you did that by calling esp_task_wdt_reset(), you're probably mis-understanding the task watchdog system you could try to hack a vTaskDelay(1) into the main loop of the sound generator; that would de-schedule the main code and allow the idle task to run.