ESP32 + STM32 and I2S... Strange strobing sound after a few seconds

[expresspotato]

We're using a combination of ESP32 (wonderful software guys, so much easier to work with than the STM32) and the STM32 for sound processing. The two MCU's are on I2S, 16 kHz, Left & Right channels, no MCLK.

ESP32 as Master
With the ESP32 as master, the BCLK appears to be a bit too slow. On the STM32 I receive an imbalance of about 1-2ms every few seconds, in callbacks of the PDM mics running at 2.048 Mhz an the I2S peripheral. Thus I would guess the I2S BCLK out from the ESP32 is slightly too slow. Here on v.4.4.6, it doesn't spit out the expected clock rates on the console anymore and the I2S code looks completely different so I can't tell for certain right now without the oscilloscope (travelling) or debugging more.

ESP32 as Slave
This works better, and the audio is crystal clear but only for about 50 seconds. Then I get a strange strobing sound (on both L & R), as if it read one too many or one too few bits and now the DMA buffer is off. My voice still comes through, but its distorted. Unplugging the WS or BCLK pins even for a split second resets the audio to being perfect again, but only for a few seconds. Unplugging and re-pluggin the DI input does nothing and I still hear the strobing sound.

apll is set to true... Not much to change on I2S really.

1. Is there a way to reset the internal alignment every WS cycle, for example?
2. Is there a way to take the external MCLK if needed (it seems like in 4.4.6 MCLK is only used for output)?
3. Or any other idea why this all is?

ESP-IDF v.4.4.6.

Edit: Strange, I see the bit clock is set to 512,000 Hz after debugging the i2s.c file... So now I'm wondering two things: what's going on between WS & the bit clock, and why when ESP32 is set as master, it falls behind further and further because it shouldn't...

Edit 2: Also it seems to always happen after around 50 - 60 seconds... Meaning its not a loose wire.

Edit 3: Changing the order of BCLK / WS / DI didn't change anything, I thought it could be the pin getting pulsed for some reason and changing the clock inadvertently.

Edit 4: So this appears to be due to the write to the onboard ES8388 is not fast enough. Setting the port max delay to 0 on the write creates a small click sound every second or two. This is because the DMA buffer is full. The ESP32 is master for that bus (I2S_NUM_0).

This tells me the ESP32 and STM32 don't appear to use the clock timings for I2S. It would still be nice just to reset the ESP32 read somehow on a number of WS cycles. I'll need to debug more.