I2S microphone (RX)

[nic_ru]

Bumping for inactivity

Hi all!
I'm trying to store in a buffer array (with no luck) the voice captured from my I2S mic (https://www.adafruit.com/product/3421) in order to execute a speech to text recognition on my server. I wrote some code (after reading your comments) but I'm probably missing something because the audio file stored on the cloud is basically "empty".

This is my code (I'm a super beginner in this world):

Code: Select all

#include "driver/i2s.h"
#define SAMPLES 128 // make it a power of two for best DMA performance
#define SPEECH_BUFFER_SIZE 40960
uint32_t speech_buffer[SPEECH_BUFFER_SIZE];
int last_position_written = 0;

void i2s_config() {
  // input
  i2s_config_t i2s_in_config = {
mode: (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
    sample_rate: 16000,
bits_per_sample: (i2s_bits_per_sample_t)32,
channel_format: I2S_CHANNEL_FMT_RIGHT_LEFT,
communication_format: (i2s_comm_format_t)(I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
intr_alloc_flags: ESP_INTR_FLAG_LEVEL1,
    dma_buf_count: 14,
    dma_buf_len: 64
  };
  i2s_pin_config_t i2s_in_pin_config = {
    bck_io_num: 16,
    ws_io_num: 4,
data_out_num: -1, //Not used
    data_in_num: 17
  };

  pinMode(17, INPUT);
  pinMode(16, OUTPUT);
  pinMode(4, OUTPUT);

  i2s_driver_install((i2s_port_t)0, &i2s_in_config, 0, NULL);
  i2s_set_pin((i2s_port_t)0, &i2s_in_pin_config);
}

void setup()
{
  Serial.begin(115200);
  delay(4000);

  i2s_config();
}

void loop()
{
//get sample from mic
  for (int i = 0; i < SAMPLES; i++) {
    int sample = 0;
    uint32_t sample_val[2] = {0, 0};
    bytes_read += i2s_pop_sample((i2s_port_t)0, (char *)sample_val, portMAX_DELAY);
    samples_for_buffer[i] = (sample_val[0] << 5);
  }
  merge_buffer(samples_for_buffer);
}

void merge_buffer(uint32_t samples_for_buffer[]) {
//if the buffer reach the max size send it to the cloud
....
//else merge buffer
  for (int j = 0; j < SAMPLES; j++) {
    speech_buffer[last_position_written] = samples_for_buffer[j];
    last_position_written++;
  }
}

Any ideas?

Thank you!

[donny681]

I try to use WM8978 to get I2S data from microphone.I refer your code and find that sample the data in the loop.It will consume CPU resource.I refer the ESP32 camera demo https://github.com/igrr/esp32-cam-demo and it uses the dma interrupt.It means that if the DMA fifo is full,the ESP32 will get into interrupt.It will cost less time .But there is a little datum about I2S and DMA.

[nico_1994]

Hello,

I'm currently working on an I2S readout of the I2S microphone (SPH0645).
My setup is an Arduino MKR1000 + SPH0645. I'm able to read out data from the microphone, see below for a sample:

F9E64000
F9E5C000
F9E50000
F9E38000
F9E2C000
F9E20000
F9E0C000
F9E00000
F9E0C000
F9E04000
F9DF0000
F9DE8000
F9DD0000
F9DC4000
F9DC4000
F9DAC000
F9DA0000
F9D9C000
F9D90000
F9D88000
F9D80000
F9D7C000
F9D70000
F9D64000
F9D60000
F9D58000
F9D50000
F9D3C000

I think this data is correct (right?).

Now my question is how can I replay the data on my serial console in a proper way on my PC?
I put in my WAV-file header the same sampling frequency as used in my Arduino software, also specified only left channel.

When I play the WAV-file I got a very short and silent sound with one click. (I did record people speaking...)

Can anyone point me in a good direction? Do I need the convert the data in another format?

Kind Regards

Nico

[BuddyCasino]

There is no ESP32 on the MKR1000 as far as I can see, right? This is probably not the right forum to help you.

[nico_1994]

There is no ESP32 on the MKR1000 as far as I can see, right? This is probably not the right forum to help you.

That right! The question was also a bit more global.
I wane know how to replay I2S data from a microphone (SPH0645) on PC as a WAV file.
How the I2S data is read isn't depending on the replay of it, maybe other people are also interested that read there data through a ESP32.

[fabrice]

Thanks alot for your code was very helpful in getting things started.

I want to know if you by any chance worked on an implementation to write recorded values into a WAV file format on an SD?

[onehorse]

"I wane know how to replay I2S data from a microphone (SPH0645) on PC as a WAV file."

You can use Audacity as described here: https://www.tindie.com/products/onehors ... icrophone/.

[turkeylegs]

I'm trying to immediately output my i2s input to the dac to i2s, following BuddyCasino's examples but I'm getting heavy distortion. The only thing I can think of is that the DAC is 8 bit and I'm feeding it 32 bits, but nothing I'm doing seems to be working. I also used teh example from here: https://esp-idf.readthedocs.io/en/lates ... s/i2s.html to set up an i2s bus to output to the DAC.

Here's my code:

Code: Select all

#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/i2s.h"
#include "esp_system.h"
#include <math.h>


#define SAMPLE_RATE     (36000)
#define I2S_NUM_RX         (1)
#define I2S_NUM_DAC		   (0)

#define bitDepth        (32)


void app_main()
{
    i2s_config_t i2s_config_rx = {
        .mode = I2S_MODE_MASTER | I2S_MODE_RX,
        .sample_rate = SAMPLE_RATE,
        .bits_per_sample = bitDepth,
        .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
        .communication_format = I2S_COMM_FORMAT_I2S_MSB,
        .dma_buf_count = 32,
        .dma_buf_len = 64,
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1
    };
    i2s_pin_config_t pin_config_rx = {
        .bck_io_num = 16,
        .ws_io_num = 4,
        .data_out_num = -1,
        .data_in_num = 17
    };

    gpio_set_direction(GPIO_NUM_17, GPIO_MODE_INPUT);
    gpio_set_direction(GPIO_NUM_16, GPIO_MODE_OUTPUT);
    gpio_set_direction(GPIO_NUM_4, GPIO_MODE_OUTPUT);

    i2s_driver_install(I2S_NUM_RX, &i2s_config_rx, 0, NULL);
    i2s_set_pin(I2S_NUM_RX, &pin_config_rx);

    // setup dac
    static const i2s_config_t i2s_config_dac = {
         .mode = I2S_MODE_MASTER | I2S_MODE_TX | I2S_MODE_DAC_BUILT_IN,
         .sample_rate = SAMPLE_RATE,
         .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT, /* the DAC module will only take the 8bits from MSB */
         .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
         .communication_format = I2S_COMM_FORMAT_I2S_MSB,
         .intr_alloc_flags = 0, // default interrupt priority
         .dma_buf_count = 32,
         .dma_buf_len = 64,
         .use_apll = false
    };
    i2s_driver_install(I2S_NUM_DAC, &i2s_config_dac, 0, NULL);
    i2s_set_dac_mode(I2S_DAC_CHANNEL_BOTH_EN);
    i2s_set_pin(I2S_NUM_DAC, NULL); // setting null makes it behave as a DAC


    uint16_t buf_len = 1024;
    char *buf = calloc(buf_len, sizeof(char));

    int bytes_written = 0;

    while (true) {

    	size_t bytes_read = 0;
    	while (bytes_read == 0) {
    		i2s_read(I2S_NUM_RX, buf, (size_t) buf_len, &bytes_read, 0);
    	}
    	//                 bytes_read / stereo? / etc.
    	int samples_read = bytes_read / 2 / (bitDepth / 8);

    	bytes_written = samples_read * 2 * (I2S_BITS_PER_SAMPLE_32BIT / 8);
    	size_t bytes_written2 = 0;
    	i2s_write(I2S_NUM_DAC, buf, (size_t) bytes_written, &bytes_written2, portMAX_DELAY);

    }

}

Is there something I'm doing wrong wrt to the DAC?

[BuddyCasino]

I think the DAC takes unsigned samples, while regular I2S is signed. So you'll need to convert the samples from -32768-32767 to 0-65535. Adding 0x8000 per channel should accomplish that.

[felipeduque]

Hi, everyone. First of all, thank you all for this great thread.

I worked on BuddyCasino's code and in case any one is still in need of a working I2S, my code is below. It grabs data from SPH0645 via I2S1 and sends via wifi. Now, I've only managed to get perfect I2S sound for sampling rate 8 kHz. From my experience with other I2S drivers (for H3 boards), it's a matter of proper clock handling. Probably some I2S master clock is too slow - maybe it didn't expect any sampling faster than 8 kHz? Anyway, I did not delve into it.

As for the issue of low dynamic range mentioned earlier in this thread, I'm not sure how I solved it!, but there is no issue in my code - I've checked in Audacity. Maybe it was related to little/big-endian stuff.

The code is not robust to connection loss: you still have to press the reset button sometimes, but the I2S part is finished.

Code: Select all

#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"
#include "esp_system.h"
#include "esp_wifi.h"
#include "esp_event_loop.h"
#include "esp_log.h"
#include "nvs_flash.h"

#include "lwip/err.h"
#include "lwip/sockets.h"
#include "lwip/sys.h"
#include "lwip/netdb.h"
#include "lwip/dns.h"
#include "driver/i2s.h"


#define EXAMPLE_ESP_WIFI_MODE_AP   0 //TRUE:AP FALSE:STA
#define EXAMPLE_ESP_WIFI_SSID      "myssid"
#define EXAMPLE_ESP_WIFI_PASS      "mypass"
#define EXAMPLE_MAX_STA_CONN       4

#define SOCKET_PORT "9030"
#define SERVER_IP "10.11.12.13"


static EventGroupHandle_t wifi_event_group;

const int WIFI_CONNECTED_BIT = BIT0;

static const char *TAG = "simple wifi";

static esp_err_t event_handler(void *ctx, system_event_t *event)
{
    switch(event->event_id) {
    case SYSTEM_EVENT_STA_START:
        esp_wifi_connect();
        break;
    case SYSTEM_EVENT_STA_GOT_IP:
        ESP_LOGI(TAG, "got ip:%s",
                 ip4addr_ntoa(&event->event_info.got_ip.ip_info.ip));
        xEventGroupSetBits(wifi_event_group, WIFI_CONNECTED_BIT);
        break;
    case SYSTEM_EVENT_AP_STACONNECTED:
        ESP_LOGI(TAG, "station:"MACSTR" join, AID=%d",
                 MAC2STR(event->event_info.sta_connected.mac),
                 event->event_info.sta_connected.aid);
        break;
    case SYSTEM_EVENT_AP_STADISCONNECTED:
        ESP_LOGI(TAG, "station:"MACSTR"leave, AID=%d",
                 MAC2STR(event->event_info.sta_disconnected.mac),
                 event->event_info.sta_disconnected.aid);
        break;
    case SYSTEM_EVENT_STA_DISCONNECTED:
        esp_wifi_connect();
        xEventGroupClearBits(wifi_event_group, WIFI_CONNECTED_BIT);
        break;
    default:
        break;
    }
    return ESP_OK;
}

void wifi_init_softap()
{
    wifi_event_group = xEventGroupCreate();

    tcpip_adapter_init();
    ESP_ERROR_CHECK(esp_event_loop_init(event_handler, NULL));

    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    wifi_config_t wifi_config = {
        .ap = {
            .ssid = EXAMPLE_ESP_WIFI_SSID,
            .ssid_len = strlen(EXAMPLE_ESP_WIFI_SSID),
            .password = EXAMPLE_ESP_WIFI_PASS,
            .max_connection = EXAMPLE_MAX_STA_CONN,
            .authmode = WIFI_AUTH_WPA_WPA2_PSK
        },
    };
    if (strlen(EXAMPLE_ESP_WIFI_PASS) == 0) {
        wifi_config.ap.authmode = WIFI_AUTH_OPEN;
    }

    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_AP));
    ESP_ERROR_CHECK(esp_wifi_set_config(ESP_IF_WIFI_AP, &wifi_config));
    ESP_ERROR_CHECK(esp_wifi_start());

    ESP_LOGI(TAG, "wifi_init_softap finished.SSID:%s password:%s",
             EXAMPLE_ESP_WIFI_SSID, EXAMPLE_ESP_WIFI_PASS);
}

void wifi_init_sta()
{
    wifi_event_group = xEventGroupCreate();

    tcpip_adapter_init();
    ESP_ERROR_CHECK(esp_event_loop_init(event_handler, NULL) );

    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    wifi_config_t wifi_config = {
        .sta = {
            .ssid = EXAMPLE_ESP_WIFI_SSID,
            .password = EXAMPLE_ESP_WIFI_PASS
        },
    };

    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA) );
    ESP_ERROR_CHECK(esp_wifi_set_config(ESP_IF_WIFI_STA, &wifi_config) );
    ESP_ERROR_CHECK(esp_wifi_start() );

    ESP_LOGI(TAG, "wifi_init_sta finished.");
    ESP_LOGI(TAG, "connect to ap SSID:%s password:%s",
             EXAMPLE_ESP_WIFI_SSID, EXAMPLE_ESP_WIFI_PASS);

}

static void init_i2s()
{
	const int sample_rate = 8000;

    i2s_config_t i2s_config_rx = {
	.mode = I2S_MODE_MASTER | I2S_MODE_RX,
	.sample_rate = sample_rate,
	.bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT,
	.channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
	.communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
	.dma_buf_count = 32,                            // number of buffers, 128 max.
	.dma_buf_len = 32 * 2,                          // size of each buffer
	.use_apll = 0,
	.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1        // Interrupt level 1
	};

	i2s_pin_config_t pin_config_rx = {
		.bck_io_num = GPIO_NUM_26,
		.ws_io_num = GPIO_NUM_25,
		.data_out_num = I2S_PIN_NO_CHANGE,
		.data_in_num = GPIO_NUM_23
	};

	i2s_driver_install(I2S_NUM_1, &i2s_config_rx, 0, NULL);
	i2s_set_pin(I2S_NUM_1, &pin_config_rx);

	i2s_stop(I2S_NUM_1);
	i2s_start(I2S_NUM_1);

}

void capture_and_send_mic_data()
{
    const struct addrinfo hints = {
        .ai_family = AF_INET,
        .ai_socktype = SOCK_STREAM,
        //.ai_socktype = SOCK_DGRAM,
    };
    struct addrinfo *res = NULL;
	struct addrinfo *rp = NULL;
    int socket_desc = -1;

    // i2s stuff
	uint16_t buf_len = 512;
	char *buf = calloc(buf_len, sizeof(char));

	init_i2s();

    while(1)
    {

        xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT,
                            false, true, portMAX_DELAY);
        ESP_LOGI(TAG, "Connected to AP");

        int e = getaddrinfo(SERVER_IP, SOCKET_PORT, &hints, &res);

        if (e != 0)
        {
            ESP_LOGE(TAG, "... getaddrinfo failed errno=%d", errno);
            freeaddrinfo(res);
            vTaskDelay(5000 / portTICK_PERIOD_MS);
            continue;
        }

        for (rp = res; rp != NULL; rp = rp->ai_next)
        {

        	socket_desc = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);

        	if (socket_desc == -1)
        		continue;

        	// if successfully connected, break the loop and go on
			if(connect(socket_desc, rp->ai_addr, rp->ai_addrlen) != -1) {
				break;
			}

			close(socket_desc);
        }


        if (rp == NULL)
        {
        	ESP_LOGE(TAG, "could not connect to socket\n");
			close(socket_desc);
			continue;
        }

        if(socket_desc < 0)
        {
        	ESP_LOGE(TAG, "... Failed to allocate socket.");
        	vTaskDelay(2000 / portTICK_PERIOD_MS);
			close(socket_desc);
        	continue;
        }

        ESP_LOGI(TAG, "... allocated and connected to socket");

		esp_err_t err = ESP_OK;

		size_t bytes_read = 0;

		while(1)
		{
			char *buf_ptr_read = buf;

			while(bytes_read == 0)
			{
				// the last argument plays an important role when trying to
				// reduce buffer underrun when streaming audio. buffer underrun
				// means that the audio player is trying to play an empty buffer.
				err = i2s_read(I2S_NUM_1, buf, buf_len, &bytes_read, 2048);
			}

			uint32_t samples_read = bytes_read / 2 / (I2S_BITS_PER_SAMPLE_32BIT / 8);
			bytes_read = 0;

			// sending only one of two channels (R, L)
			for(int i = 0; i < samples_read; i++)
			{

				int16_t currRSample = (buf_ptr_read[3] << 8) + buf_ptr_read[2];
				//int16_t currLSample = (buf_ptr_read[1] << 8) + buf_ptr_read[0];
				int16_t data = currRSample;

				buf_ptr_read += 2 * (I2S_BITS_PER_SAMPLE_32BIT / 8);

				if (write(socket_desc, &data, sizeof(data)) < 0)
				{
					close(socket_desc);
					freeaddrinfo(res);
					ESP_LOGE(TAG, "... socket send failed. Must try to reconnect...");
					exit(EXIT_FAILURE);
				}
			}

		}

	}
}


void app_main()
{
    //Initialize NVS
    esp_err_t ret = nvs_flash_init();
    if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
      ESP_ERROR_CHECK(nvs_flash_erase());
      ret = nvs_flash_init();
    }
    ESP_ERROR_CHECK(ret);

#if EXAMPLE_ESP_WIFI_MODE_AP
    ESP_LOGI(TAG, "ESP_WIFI_MODE_AP");
    wifi_init_softap();
#else
    ESP_LOGI(TAG, "ESP_WIFI_MODE_STA");
    wifi_init_sta();
#endif

    xTaskCreate(&capture_and_send_mic_data, "capture_and_send_mic_data", 4096, NULL, 5, NULL);

}