Struggling with BLE OTA encrypted updates

[netfox]

Hi everyone,

I've been grappling with this issue for several days now, and I'm at a loss. Previously, I had successfully implemented OTA updates using non-encrypted images with ESP-IDF/iOS. I am now transitioning to using encrypted images, which has proven quite complex.

I've integrated the OTA BLE example from the ESP-IDF into my project. The process begins with establishing a connection and completing handshaking through custom characteristics, after which I send encrypted data to my ESP device. The setup involves a callback named ota_recv_fw_cb, which is established with esp_ble_ota_recv_fw_data_callback(ota_recv_fw_cb, decrypt_handle). However, I'm unsure how to correctly route data to this callback.

Here’s how I've configured the GATT characteristic for OTA data:

Code: Select all

{
    // Characteristic: OTA data
    .uuid = &gatt_svr_chr_ota_data_uuid.u,
    .access_cb = gatt_svr_chr_ota_data_cb,
    .flags = BLE_GATT_CHR_F_WRITE,
    .val_handle = &ota_data_val_handle,
}

And here is the callback associated with this characteristic:

Code: Select all

int gatt_svr_chr_ota_data_cb(uint16_t conn_handle, uint16_t attr_handle,
                             struct ble_gatt_access_ctxt *ctxt,
                             void *arg)
{
    uint8_t *data = ctxt->om->om_data; 
    uint16_t len = ctxt->om->om_len;

    ota_recv_fw_cb(data, len);

    return 0;
}

However, after attempting to write the encrypted data to my device, I encounter the following errors:

Code: Select all

E (4378) esp_ota_ops: OTA image has invalid magic byte (expected 0xE9, saw 0xcf)
E (4388) ESP_BLE_OTA: esp_ota_write failed!
E (4388) ESP_BLE_OTA: OTA failed

Why am I seeing this error? Could it be related to the "Firmware package format" as detailed here: https://components.espressif.com/compon ... if/ble_ota. Is there a specific format I need to apply to each data chunk I send? Can someone steer me in the right direction to avoid further wasted hours?

Programming is a hobby of mine, but this part of the project is to frustrating Any guidance or insights would be greatly appreciated.

[ESP_Sprite]

0xcf is the first byte of the *encrypted* image format, but it looks like it's written via function calls that expect the *decrypted* image.

[netfox]

Yes, I see. But shouldn't my code pass the decrypted code, instead of the encrypted code to esp_ota_write? Some pieces of my code to get a better idea of the current code.

Code: Select all

esp_decrypt_cfg_t cfg = {};
    cfg.rsa_priv_key = rsa_private_pem_start;
    cfg.rsa_priv_key_len = rsa_private_pem_end - rsa_private_pem_start;
    decrypt_handle = esp_encrypted_img_decrypt_start(&cfg);
    if (!decrypt_handle) {
        printf("Failed to start decrypt handle\n");
        vTaskDelete(NULL);
    }

  esp_ble_ota_recv_fw_data_callback(ota_recv_fw_cb, decrypt_handle);

Code: Select all

void ota_recv_fw_cb(uint8_t *buf, uint32_t length)
{
  write_to_ringbuf(buf, length);
}

Code: Select all

size_t
write_to_ringbuf(const uint8_t *data, size_t size)
{
  BaseType_t done = xRingbufferSend(s_ringbuf, (void *)data, size, (TickType_t)portMAX_DELAY);
  if (done)
  {
    return size;
  }
  else
  {
    return 0;
  }
}

Code: Select all

for (;;)
  {
    data = (uint8_t *)xRingbufferReceive(s_ringbuf, &item_size, (TickType_t)portMAX_DELAY);

    ESP_LOGI(TAG, "recv: %u, recv_total:%" PRIu32 "\n", item_size, recv_len + item_size);
    if (item_size != 0)
    {
      if (esp_ota_write(out_handle, (const void *)data, item_size) != ESP_OK)
      {
        ESP_LOGE(TAG, "esp_ota_write failed!\r\n");
        goto OTA_ERROR;
      }
      recv_len += item_size;
      vRingbufferReturnItem(s_ringbuf, (void *)data);

      if (recv_len >= esp_ble_ota_get_fw_length())
      {
        break;
      }
    }
  }

I think this is the same as the example. Or am I missing something?

[netfox]

Also spend a few hours by looking into the android app (https://github.com/EspressifApps/esp-bl ... ses/tag/rc) and adding Sector_Index numbers and Packet_Seq numbers in my packages, by updating my iOS code. Like: https://components.espressif.com/compon ... if/ble_ota tells me. But still:

Code: Select all

E (8959) esp_ota_ops: OTA image has invalid magic byte (expected 0xE9, saw 0x00)
E (8969) ESP_BLE_OTA: esp_ota_write failed!
E (8979) ESP_BLE_OTA: OTA failed

Different first byte of course, because of how the packages are set up.

Still no solution in sight. I really hope to meet someone on this forum, who already tried and succeeded this procedure.

[netfox]

It's driving me nuts. I have completely redone the code, and used the example in nimble_ota.c

After succesfully transmitting data, I get error:

E (72038) esp_encrypted_img: Magic Verification failed
E (72038) NimBLE_BLE_OTA: Error in decryption: -1

This is my current swift code:

Code: Select all

func initPackets() {
        let totalSectors = (firmwareData!.count + 4095) / 4096
        var packets = [Data]()

        for sectorIndex in 0..<totalSectors {
            let sectorStart = sectorIndex * 4096
            let sectorEnd = min((sectorIndex + 1) * 4096, firmwareData!.count)
            let sectorData = firmwareData!.subdata(in: sectorStart..<sectorEnd)
            
            var packetSeq: UInt16 = 0
            var start = 0
            while start < sectorData.count {
                let end = min(start + EXPECTED_PACKET_SIZE, sectorData.count)
                var packet = Data()
                packet.append(contentsOf: withUnsafeBytes(of: UInt16(sectorIndex).bigEndian) { Data($0) })
                packet.append(contentsOf: withUnsafeBytes(of: packetSeq.bigEndian) { Data($0) })
                packet.append(sectorData[start..<end])

                if start + EXPECTED_PACKET_SIZE >= sectorData.count {
                    packetSeq = 0xFF // Last packet of the sector
                    let crc16 = calculateCRC16(sectorData)
                    packet.append(contentsOf: withUnsafeBytes(of: crc16) { Data($0) })
                    packet.append(contentsOf: [UInt8](repeating: 0, count: EXPECTED_PACKET_SIZE - packet.count))
                }
                
                start = end
                packetSeq += 1
                packets.append(packet)
            }
        }
        self.packets = packets
    }
    
    func sendNextPacket(for peripheral: CBPeripheral) {
        guard !packets.isEmpty else {
            sendEndCommand(for: peripheral)
            return
        }
        
        let packet = packets.removeFirst()
        writePacketToOTACharacteristic(packet, for: peripheral)
    }

    func sendStartCommand(for peripheral: CBPeripheral) {
        var packet = Data()
        packet.append(contentsOf: withUnsafeBytes(of: UInt16(OTACommandID.start).bigEndian) { Data($0) })
        packet.append(contentsOf: Array(repeating: UInt8(0), count: 18 - packet.count))
        writePacketToOTACharacteristic(packet, for: peripheral)
    }

    func sendEndCommand(for peripheral: CBPeripheral) {
        var packet = Data()
        packet.append(contentsOf: withUnsafeBytes(of: UInt16(OTACommandID.end).bigEndian) { Data($0) })
        packet.append(contentsOf: Array(repeating: UInt8(0), count: 18 - packet.count))
        writePacketToOTACharacteristic(packet, for: peripheral)
    }

It is designed around this documentation: https://github.com/espressif/esp-iot-so ... ansmission

The relevant code for decryption on the esp32 is:

Code: Select all

static void
esp_ble_ota_write_chr(struct os_mbuf *om)
{

  printf("write char");
  
#ifndef CONFIG_OTA_WITH_PROTOCOMM
  esp_ble_ota_char_t ota_char = find_ota_char_and_desr_by_handle(attribute_handle);
#endif

#ifdef CONFIG_PRE_ENC_OTA
  esp_err_t err;
  pre_enc_decrypt_arg_t pargs = {};

  pargs.data_in_len = om->om_len - 3;

  if (SLIST_NEXT(om, om_next) != NULL)
  {
    struct os_mbuf *temp2 = SLIST_NEXT(om, om_next);
    pargs.data_in_len += temp2->om_len;
  }

  pargs.data_in = (const char *)malloc(pargs.data_in_len * sizeof(char *));
  err = os_mbuf_copydata(om, 3, pargs.data_in_len, pargs.data_in);

  if (om->om_data[2] == 0xff)
  {
    pargs.data_in_len -= SECTOR_END_ID;
  }

  err = esp_encrypted_img_decrypt_data(decrypt_handle_cmp, &pargs);
  if (err != ESP_OK && err != ESP_ERR_NOT_FINISHED)
  {
    ESP_LOGE(TAG, "Error in decryption: %d", err);
    return;
  }
#endif