Page 1 of 1

16MB Flash max app size with OTA

Posted: Wed Apr 28, 2021 8:08 pm
by buster
Hello,

I am using a 16MB ESP32 WROVER to use in projects. I wanted the 16MB version so i have the most space available for my program. In the Arduino IDE i can select the 16MB Flash, but in the partition, the biggest i can set is (3MB APP / 9MB FATFS). That is only 12MB, not 16.

I also use over the air updates via internet (HTTPUpdate). What is the max safe partition scheme (and how do i select / set it up) to use on a 16MB ESP32, i think i read that the scheme cannot be bigger than 50% of the flash size because of the download / extract?

But can i set 8MB for app, 8MB for firmware download, or doesn't it work like that?

Thanks a lot

Re: 16MB Flash max app size with OTA

Posted: Thu Apr 29, 2021 4:13 am
by lbernstone

Re: 16MB Flash max app size with OTA

Posted: Thu Apr 29, 2021 6:30 am
by buster
Oh okay, i understand now, but is 3MB max? Why can't it be 8MB * 2, or 6MB *2 + 3MB?
Where and how can the FAT be used?

Re: 16MB Flash max app size with OTA

Posted: Thu Apr 29, 2021 1:11 pm
by lbernstone

Re: 16MB Flash max app size with OTA

Posted: Thu Apr 29, 2021 5:43 pm
by ullixesp
I use ESP32 with 4, 8, and 16 MB flash. In all of them I configure the partitions the same way, except that FFat gets a different share.

Code: Select all

# used with 4MB,8MB, 16 MB chips
#       Espressif ESP32 Partition Table
#       Name,       Type,   SubType,    Offset,     Size,      
        nvs,        data,   nvs,        0x9000,     0x5000,
        otadata,    data,   ota,        0xe000,     0x2000,
        app0,       app,    ota_0,      0x10000,    0x140000,
        app1,       app,    ota_1,      0x150000,   0x140000,

        #ffat,      data,   fat,        0x290000,   0x170000,   # for  4 MB
        #ffat,      data,   fat,        0x290000,   0x570000,   # for  8 MB
        ffat,       data,   fat,        0x290000,   0xD70000,   # for 16 MB
So, the app gets a max size of 0x140000 (=1.25MB). Presently my sketch is ~1MB (about 81% of max), which I upload (mostly) via OTA by HTML, using the async server. Goes well most of the time, but there are hiccups, like once every dozen times. The speed is slightly less than a serial upload with 921000 baud.

OTA updates of HTML and JS files - typically < 100k - into FFat are fast and convenient.

Re: 16MB Flash max app size with OTA

Posted: Thu Apr 29, 2021 8:27 pm
by buster
Thank you for the reply. I am planning on using the ESP32 in a production environment. Dozens, maybe hundreds of devices are going to be sold and when we have an update, they all download the BIN file from our server and start updating. If the update fails, it should restart, but if failing means the device will not boot anymore i am in big trouble.

Is there a way to prevent this, is there a way to have a backup feature so if something goes corrupt it can be restored (via a button or something else).

Re: 16MB Flash max app size with OTA

Posted: Fri Apr 30, 2021 7:47 am
by ullixesp
These are exactly my plans, too, and here is some good news: during the last about 2 years I used OTA on ESP32 and I never ever had a corruption from OTA with it! The most, what was needed to continue, was a reboot.

The reason for this very benign behaviour is that you use 2 partitions of 'app0' and 'app1' of type 'app' and subtype 'ota_0' and 'ota_1', resp., both of the same size. See above for a list of my partitions as example. My interpretation: One of the partitions is the working one, while the other is waiting and will receive the next OTA update. When the OTA is done successfully, and only then, their role is switched.

At least this is what I believe how this works. Perhaps someone else can confirm or correct.

Yet, bottom line, no corruption of the working partition.

When I said it fails then I meant it fails on the first attempt. But when it fails, then most of the time you need more than one repeat, sometimes up to 4. Why? No idea. In some cases the WiFi had failed during the OTA; others may be due to some bad programming on my side. I am trying to switch off all activities not essential for OTA during OTA, yet it still happens.

OTA is not a huge problem. Corruption of FFat is a much bigger one!

Re: 16MB Flash max app size with OTA

Posted: Mon May 03, 2021 6:59 am
by buster
What do you mean with "Corruption of FFat is a much bigger one". When did you have corrupted FFat storage?

Re: 16MB Flash max app size with OTA

Posted: Mon May 03, 2021 8:27 am
by ullixesp
When did you have corrupted FFat storage?
Depending on what you do: like hourly?

I am building a data logger with savings every minute. There is always a chance for corrupting the FFat file system in situations where power loss, reboots, uploads, crashes happen at the moment of writing to FFat. When I can control the situation - and not forget - I shut down all such FFat writing activity before reboot, upload etc.

Outside of development I had it runnning for 2 weeks straight (and then stopped it intentionally). So, long runs are certainly possible. But you can never control power outages.

FFat is quite reliable and resonably consistent in reading, writing, and, important for me, overwriting. But it is awfully easy to corrupt.