Sending more than 20 bytes with BLE

[Anh-Minh]

Ble has this limit, but you treat the long messages as several messages within the limit, dividing them when sending and joining them when receiving by yourself

What I do to work with larger messages:

- All messages (mobile and esp32) ends with a specific character (in my case \n)

- On Android (Java and now Kotlin) or iOS (swift 4):
- When sending if the message is large,
I split it in pieces and send it (with a small delay forced between them in Android, IOS not need it)
Tip: If I have more than one message to send in moment, I just join it with delimiter \n and send all to one call of routine that split it
to Ble, generating few messages instead call it for each message.

- In Esp32, idem only that does not need the delay

- When receiving messages (app or Esp32), I work with a buffer, and only process the messages upon receiving a \n (since the message can be divided into several pieces)

Now I working to balance use of Esp32 cpu cores. The work of split/join ble messages is now to in task in CPU 1, leaving CPU 0 to system (Ble, timers (tg), etc)

My idea is when I have time, put together a complete solution:
- Android and iOS app and Esp-IDF and put in github.

Hope this helps.

Thank you very much for you respond JoaoLopesF ! Ok, now i understand how it works. Can you show me an example of the function in which you split the data when sending if the message if too long, and how you process the message upon receiving a \n while receiving ?

[JoaoLopesF]

Hi, Anh-Minh

I post now same examples of this (see below).
I translate my code to english for this examples.
*** This only for show how it works
*** I not tested the codes it after translate, I will do it when I made the entire example for put in github
*** After I post code for Ios in Swift 4, for now Android and Esp32 is a good start.

I Hope this helps (please notify if its is not clear to you)

*** Android Code (kotlin)

Code: Select all

fun sendMessage (message: String) {

        val MAX_SIZE = 20

        // Sending?

        if (sending) {

            // If sending, send this after delay

            val handler = Handler ()
            handler.postDelayed ({
	        
	        // Part of my library - commented for this example
                //logI ("Sending after delay!")
                
                sendMessage (message)
            }, 500)

            return
        }

        sending = true

        // Send a message via BLE

        if (bleUartServico == null) {

            // Part of my library - commented for this example
            // if (Utilility.isEmulator == false) {
            //    this.activity.extShowException (stringServiceNotActive)
            //}
            return
        }

        //logD ("send ($ {message.length}) -> $ {message.extExpandStr ()}")

        // In separate Thread

        try {

            object: Thread () {
                override fun run () {

                    // Process message

                    var posBegin = 0
                    val size = message.length
                    var posEnd = if (size> MAX_SIZE) MAX_SIZE else size

                    // Process, sending parts if they are greater than the maximum

                    do {
                        val part = message.substring (posBegin, posEnd)
                        if (part.length> 0) {

                            //if (debugExtra) {
                            //    logD ("send part ($ {part.length}) -> $ {part.extExpandStr ()}")
                            //}
                            var data = ByteArray (0)
                            try {
                                data = part.toByteArray (charset ("UTF-8"))
                            } catch (e: UnsupportedEncodingException) {
                                e.printStackTrace ()
                            }

                            bleUartService!!.writeRXCharacteristic (data)

                            if (posEnd == size) {
                                break
                            }

                            posBegin = posEnd
                            posEnd = posBegin + MAX_SIZE
                            if (posEnd > size) {
                                posEnd = size
                            }
                        } else {
                            break
                        }

                        // Wait a while // Android workaround 
                        try {
                            Thread.sleep (200) 
                        } catch (e: InterruptedException) {
                            e.printStackTrace ()
                        }

                    } while (posEnd <= size)
                }

            } .start ()

        } catch (e: Exception) {
            // Part of my library - commented for this example
            // this.activity.extShowException (e)
        }

        sending = false
    }
    
    
private fun receivedData (data: String) {

    receivedData (data.toByteArray ())
}

 private fun receivedData (data: ByteArray) {

    // Received Bluetooth data

    // logD ("receivedData:" + Utilities.expandStr (new String (data)));

    // Mark the time

    timeLastReceivedData = System.currentTimeMillis ()

     // Process bytes received

     for (character in data) {

        if (character.toInt () == 10) { // New line

            // Save the line

            //logV ("receive line: $ {bufferLine.toString (). extExpandStr ()}")

            // Callback for receiving

            // Part of my library - commented for this example
            //if (btUtilHandler! = null) {
            //     btUtilHandler!!.onReceiveLine (bufferLine.toString ())
            //}

            // Next

            bufferLine = StringBuffer ()

          } else if (character.toInt ()! = 13) {

             // Adds the received character

             buffer.append (character.toChar ())
          }
     }
 }

*** Example of Esp-Idf code (in C++)

*** util/ble_server.cc

Code: Select all

void BleServer::sendData (const char* data) {

    // Send data to APP (via BLE)

    if (! mConnected) {
        LOGE (TAG_BLE, "not connected");
        return;
    }

    // Send data via UART BLE Server

    uint8_t size = strlen (data);

    if (mLogActive) {
        ESP_LOGV (TAG_BLE, "BLE [% d] ->% s", size, data);
    }

    // Send data, respecting the maximum size

    char send [BLE_SEND_MAX_SIZE + 1];

    memset (send, 0, BLE_SEND_MAX_SIZE + 1);

    uint8_t posSend = 0;

    for (uint8_t i = 0; i <size; i ++) {

        send [postSend ++] = data [i];

        if (posSend == BLE_SEND_MAX_SIZE || i == (size -1)) { // Has it reached the maximum or the end?

        // Send the data

        if (mLogActive) {
            ESP_LOGV (TAG_BLE, "BLE sending part [% d] ->% s \ r \ n", postSend, send);
        }

        // BLE routines in C based on pcbreflux example

        ble_uart_server_Send (send, postSend);

        // Next sending

        postSend = 0;

        memset (send, 0, BLE_SEND_MAX_SIZE);

    }
}

static void receive_Task (void * pvParameters) {

    ESP_LOGI (TAG_BLE, "Initializing receives Task - cpu% d", xPortGetCoreID ());

    // Create a queue to send queue data

    xBleQueueReceb = xQueueCreate (BLE_TAM_QUEUE_RECEB, sizeof (BleQueueReceb));

    if (xBleQueueReceb == NULL) {
        ESP_LOGE (TAG_BLE, "Error creating queue");
    }

    /// Data to receive from the Ble Server queue

    BleQueueReceb dataQueue;

    ////// Loop

    for (;;)

        // Expect to receive something from the Ble Server queue
        // Adopted this logic to force processing on CPU 1

        BaseType_t ret = xQueueReceive (xBleQueueReceb, &dataQueue, portMAX_DELAY);

        if (ret == pdPASS) {

            // Check the queue

            if (dataQueue.size> 0) {

                // Data received

                if (mLogActive) {
                    ESP_LOGV (TAG_BLE, "queue -> extracted data (free% d), making the callback", uxQueueSpacesAvailable (xBleQueueReceb));
                }

                // Callback

                if (mBleServerCallbacks) {
                    mBleServerCallbacks-> onReceive (dataQueue.dados, dataQueue.tamanho);
                }

                } else {

                    // Notice

                    ESP_LOGE (TAG_BLE, "queue_task -> queue -> tam 0 !!!");
                }

            } else {

                // Notice

                LOGE (TAG_BLE, "queue_task -> error!");
            }
        }

        ////// End

#ifdef DEBUG
        LOGI (TAG_BLE, "receive task closed");
#endif

    // Delete queue

    if (xBleQueueReceb! = NULL) {
        vQueueDelete (xBleQueueReceb);
    }

    // Delete this task

    vTaskDelete (NULL);
}

*** ble.cc (main)

Code: Select all

void bleReceived (char * data, uint8_t size) {

    // Received data via BLE server (by callback)

    char aux [size + 1];
    sprintf (aux, "%.*s", size, data);

    if (mLogActive) {
        ESP_LOGV (TAG, "*** BLE received (%d):%s", size, Util.strExpand (aux).c_str());
    }

    // Mark the timeSeconds of the last receipt

    mLatTimeReceived = mTimeSeconds; 

    // Process the received data

    for (uint8_t i=0; i <size; i ++) {

        char character = data[i];

        // Process the message upon receiving a new line

        if (caract == '\n') {

            if (mLogActive) {
                ESP_LOGD ("*** BLE line receive:%s", mBleLinha.c_str());
            }

            mBleReceivingLine = false;

            // Process the received line

            if (mBleLine.length() > 0) {

                // Process the message (main.cc)

                processMessage (mBleLine);

            }

            mBleLine = "";

        } else if (caract != '\r') {

            // Concatenates

            mBleLinha.append (1u, character);
            mBleReceivingLine = true;

        }
    }
}

void bleVerifyTimeouts () {

    // Check timeout on receipt of data -- called by timer at each second

    if (mBleReceivingLine
        && (mTimeSeconds - mLastTimeReceived)> BLE_TIMEOUT_REC_LINE) {

        mBleReceivingLine = false;
        mBleLinha = "";

        if (mLogActive) {
            ESP_LOGD (TAG, "receive timeout line");
        }
    }
}

[JoaoLopesF]

*** Update

I found out here in the forum and on the internet, that is possible change the MTU of the BLE, besides the 20 bytes.
Searching the internet, I also saw that iOS already does this, so we can send several messages without delay (necessary for Android)

I worked to change the MTU to Android and Esp32.

The maximum allowed is 517. I'm using 200, which is enough for my application

It is only possible for Android 5.0+

So it looks like this to support new and old Android:

**** Changes on gattCallback

Code: Select all

override fun onConnectionStateChange(gatt: BluetoothGatt, status: Int, newState: Int) {
            val intentAction: String

            if (newState == BluetoothProfile.STATE_CONNECTED) {
                intentAction = ACTION_GATT_CONNECTED
                connectionState = STATE_CONNECTED
                broadcastUpdate(intentAction)
                if (BtUtil.debugExtra) Log.i(TAG, "Connected to GATT server.")

                // If Android >= 5.0 - change MTU  before discover services

                if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {

                    // Request MTU - change it!

                    requestMTU()

                } else { // Request MTU not allowed -> discover services here

                    // Attempts to discover services after successful connection.

                    val ret = bluetoothGatt!!.discoverServices()
                    if (BtUtil.debugExtra) Log.i(TAG, "Attempting to start service discovery: $ret")

                }

            } else if (newState == BluetoothProfile.STATE_DISCONNECTED) {
                intentAction = ACTION_GATT_DISCONNECTED
                connectionState = STATE_DISCONNECTED
                if (BtUtil.debugExtra) Log.i(TAG, "Disconnected from GATT server.")
                   broadcastUpdate(intentAction)
            }
        }

        .....
        
        override fun onMtuChanged(gatt: BluetoothGatt, mtu: Int, status: Int) {
            super.onMtuChanged(gatt, mtu, status)

            if (status == BluetoothGatt.GATT_SUCCESS) {

                mtuChanged = mtu
                Log.w(TAG, "onMtuChanged: mtu changed to: $mtu")

                // Discovery services after mtu changed

                val ret = bluetoothGatt!!.discoverServices()
                if (BtUtil.debugExtra) Log.i(TAG, "After mtu -> Attempting to start service discovery: $ret")
            }
        }

**** Request new MTU

Code: Select all

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
    private fun requestMTU() {

        val mtu = 200 + 3 // Maximum allowed 517 - 3 bytes do BLE

        // Request MTU to max allowed

        bluetoothGatt!!.requestMtu(mtu)

        Log.w(TAG, "requestMTU -> mtu=$mtu")
    }

In Esp32 it is necessary to check this event in the Gatt server:

*** Changed maximum length to Ble Characterics

Code: Select all

//#define GATTS_CHAR_VAL_LEN_MAX	22
#define GATTS_CHAR_VAL_LEN_MAX		200 // Changed maximum to avoid split of messages

*** Changed gatts_profile_event_handler

Code: Select all

	case ESP_GATTS_MTU_EVT:
		DEBUG("ESP_GATTS_MTU_EVT, mtu %d", param->mtu.mtu);
		mMTU = param->mtu.mtu;
		break;

*** Changed the send routine (#define BLE_SIZE_MAX_SEND GATTS_CHAR_VAL_LEN_MAX)

Code: Select all

// Maximum of sending (now it is via MTU)

uint16_t maximum = ble_uart_server_mtu ();

if (maximum> BLE_SIZE_MAX_SEND) {
	maximum = BLE_SIZE_MAX_SEND;
}

****
Before my app on Android was much slower than the app on iOS,
now this is much faster

[viniciusbaesso]

Unless you dont need to send messages longer than about 500-512 bytes you dont have to worry, library will take care of it (have not tested longer messages). Messages longer than mtu are fragmented/defragmented, the issue is with notifications/indications. Those messages are truncated to (mtu - 3) bytes.

When i want to write value on my ESP32 with nRF connect app, only the first 20 bytes are received. I don't really understand how it works

Hi Anh-Minh,


Are you using an android app or ios? If using android, just use the line of code "gatt.requestMtu (size)" in the source of the android app. This code should be entered shortly after a connection to GattServer has been established. If you are using IOS, by default the central OS requires GattServer MTU of 185 bytes.

Hope this helps.

[JoaoLopesF]

Hi, I just finish a set of examples (ESP-IDF, Android and iOS), with large BLE messages working
Please see it in: viewtopic.php?f=18&t=6806

[Boris.neworchad]

My idea is when I have time, put together a complete solution:
- Android and iOS app and Esp-IDF and put in github.
 
Hope this helps.

I am really looking forward for this Github repository!