ESP32: Partition Table Update through OTA

[Maxzillian]

Just to add to this discussion, we've successfully written some code that allows us to expand the partition sizes. Our current partition table has 3 partitions: Factory, OTA 0 and OTA 1. We needed to expand the individual partition sizes from 1 MB to 2 MB. This works out well in our case as the new OTA 1 location exists in flash that is not utilized by the pre-existing partition table (this is a very important detail!) To accomplish this we do the following steps:

-Include the new partition binary file by adding `set(COMPONENT_EMBED_FILES assets/partition-table.bin)` to the CMakeList definition.

-Parse the binary to find the new location and size of the OTA 1 partition

Code: Select all

{
extern const uint8_t partStart[] asm("_binary_partition_table_bin_start");
extern const size_t partSize asm("partition_table_bin_length");
size_t index = 0;
parsePartitionTableBinary("ota_1", partStart, partSize, &index);

// find the position, end and length of OTA_1
uint32_t otaPosition = 0;
uint32_t otaLength = 0;
index -= 8;
memcpy(&otaPosition, partStart + index, sizeof(otaPosition));
index += 4;
memcpy(&otaLength, partStart + index, sizeof(otaLength));
}

bool parsePartitionTableBinary(char* label, const uint8_t start[], const size_t size, size_t *index)
{
    size_t length = strlen(label);
    for(; *index < size - length; (*index)++) {
        if(memcmp(start + *index, label, length) == 0) {
            return true;
        }
    }

    // didn't find the partition definition
    return false;
}

-Verify that this region does not overlap with the currently operating app

Code: Select all

// find the location of the currently active app and check for size conflicts
const esp_partition_t * currentApp = esp_ota_get_boot_partition();

if(currentApp->address + currentApp->size >= otaPosition) {
    // end of current app overlaps with beginning of new partition
    return false;
}
if(currentApp->size > otaLength) {
    // current app is too large for the new partition
    return false;
}

-erase flash in the region of the new OTA 1 partition

Code: Select all

if(spi_flash_erase_range(otaPosition, otaLength) != ESP_OK) {
    return false;
}

-copy the currently running app into the OTA 1 partition

Code: Select all

// map flash location to memory address
const void *memPtr;
spi_flash_mmap_handle_t handle;
if(esp_partition_mmap(currentApp, 0, currentApp->size, SPI_FLASH_MMAP_DATA, &memPtr, &handle) != ESP_OK) {
    return false;
}
// copy current app data to new partition
if(spi_flash_write(otaPosition, memPtr, currentApp->size) != ESP_OK) {
    return false;
}
spi_flash_munmap(handle);

That's effectively stage one. So far everything that has been done (as long as the safety checks are observed) won't result in a bricked controller. From here we get into the risky stuff that can brick the controller if interrupted:

-Erase the old partition table and flash in the new one

Code: Select all

// erase old partition table
spi_flash_erase_range(0x8000, 0x1000);
// flash new partition table
spi_flash_write(0x8000, partStart, partSize);

-Set the next boot partition. I haven't dug into why, but you would think this would refer to the OTA 1 location of the old partition as the new table hasn't been parsed yet. However, this method appears to work just fine.

Code: Select all

esp_partition_iterator_t esp_it = esp_partition_find(ESP_PARTITION_TYPE_APP, ESP_PARTITION_SUBTYPE_APP_OTA_1, NULL);
esp_ota_set_boot_partition(esp_partition_get(esp_it));

At this point the system must be hard reset. Calling esp_restart() will not result in the expected behavior. However, once powered down and back on again, the controller should parse the new partition table and launch the app found at OTA 1.

Should there be a problem with the OTA 1 partition, it should still be able to fall back to the factory app as the starting location of that has not changed and the old app size still fits within the new partition definition.

[DrMickeyLauer]

I'm in need of a similar, but hopefully more simple, problem.

I have the following partition table:

Code: Select all

nvs,      data, nvs,     0x9000,  0x4000,
otadata,  data, ota,     0xd000,  0x2000,
phy_init, data, phy,     0xf000,  0x1000,
ota_0,    app,  ota_0,   0x10000, 0x1A0000
ota_1,    app,  ota_1,          , 0x1A0000
storage,  data, fat,            , 0x80000

which leaves about 12MB of unused flash on our devices.

I want to append two partitions at runtime, so that it looks like:

Code: Select all

nvs,      data, nvs,     0x9000,  0x4000,
otadata,  data, ota,     0xd000,  0x2000,
phy_init, data, phy,     0xf000,  0x1000,
ota_0,    app,  ota_0,   0x10000, 0x1A0000
ota_1,    app,  ota_1,          , 0x1A0000
storage,  data, fat,            , 0x80000
coredump, data, coredump,,        64K
littlefs,	data,	spiffs,, 10M

Can I do that by copying the new partition table over the old one and
erasing the new regions or is it more involved? How would
I "format" the littlefs partition for the first-time?

[boarchuz]

You risk bricking the device if the erase+write is interrupted for any reason, so imo it's worth avoiding this where possible.

Unless these changes need to be backwards compatible, there's no need to literally alter the partition table.

You can specify the partition in your esp_vfs_littlefs_conf_t, for example.
You could also use linker wrapping to provide a custom esp_partition_find_first that captures any requests for these new partitions, else passes them through to the real esp_partition_find_first. That will work with coredumps as here: https://github.com/espressif/esp-idf/bl ... lash.c#L80

[DrMickeyLauer]

Oh, this is very interesting. So the partition table is only a convenience for a bunch of offsets and sizes and meta-data and we can just "fake" it (as risky as this may be, when the software supports multiple hardware revisions). Thanks.