ESP32 remote firmware update options

[zazas321]

Hello. I am working with ESP32 device which will be used as a ultra low power tracking device. This device will be used somewhere far away from me, hence I am concerned about remote firmware update.

I have tested the OTAwebupdater example as such:
https://lastminuteengineers.com/esp32-o ... duino-ide/

and I can confirm that it works great, however, in my application, I cannot be sure that the device will have access to wifi therefore this may not work. My device also has GSM and GPS. Is it possible to do a firmware update through GPS?

Please any device is appreciated!

[gbarry]

GPS is a broadcast only service. You can't put messages on it. It's also a very slow data rate, but that's a detail.
GSM could possibly be used, but coverage isn't a sure thing, much as WiFi would not be.

[gfreund]

GPS is a navigation system and no, you can't use it for any other communication. GSM (really GPRS) is very slow and therefore might not be suitable. Also basic GSM/GPRS is currently end-of-lived by the two US carriers that support it (AT&T and T-Mobile) so it might not work much longer. Want you need is a modem that supports LTE CAT-M and/or NB-IoT depending on the carrier you're planning to use and your region. For example any module based on the Qualcomm 9205 LTE chip such as Simcom 7080G or Quectel BG95-M3 would do just fine. Many of these modules already contain a GPS receiver for position etc. data.

As far as the update process is concerned you might want to look at https://docs.espressif.com/projects/esp ... m/ota.html. Works fine under Arduino. Essentially you need two OTA partitions, one to run your current image and one to update. Then you download your image onto the "spare" one and tell the ESP32 bootloader to "switch". There are ways to roll back the update should the new image run into problems.

Happy to answer more specific questions.

~G

[zazas321]

The standart Arduino OTA requires the ESP32 to be connected to the internet using WIFI which is not convenient for some IoT applications. Some devices operate in places where simply there is no available WiFi to connect to. The solution that I have found for now is GSM OTA . I have tested it and im quite happy with the result. The only thing you need is to setup a server where you can store your binary file. The GSM module will do the rest and it did it quite fast.. I have uploaded only very small sketch and it took like 15 seconds so not so bad


Could you explain a little further what you meant about the the problems of standard GSM/GPRS and what will be advantages of other modules that you have suggested?


The full code:

Code: Select all

#include <Update.h>
#define TINY_GSM_MODEM_SIM800
// Increase RX buffer
#define TINY_GSM_RX_BUFFER 1030

const char apn[] = "omnitel";
const char user[] = "omni";
const char pass[] = "omni";



#define MODEM_RST             5
#define MODEM_PWRKEY          4
#define MODEM_POWER_ON       23
#define MODEM_TX             27
#define MODEM_RX             26
#define LED_GPIO             13
#define LED_ON               HIGH
#define LED_OFF              LOW


#define SerialAT  Serial1

#include <TinyGsmClient.h>
#include <CRC32.h>

#include "FS.h"
#include "SPIFFS.h"

#ifdef DUMP_AT_COMMANDS
#include <StreamDebugger.h>
StreamDebugger debugger(SerialAT, Serial);
TinyGsm modem(debugger);
#else
TinyGsm modem(SerialAT);
#endif


TinyGsmClient client(modem);

const char server[] = "myesptestserver.ddns.net";
const int port = 80;
const char resource[] = "/esp.bin"; //here de bin file

uint32_t knownCRC32 = 0x6f50d767;
uint32_t knownFileSize = 1024; // In case server does not send it

void setup()
{
    SerialAT.begin(115200, SERIAL_8N1, MODEM_RX, MODEM_TX);
    // Set console baud rate
    Serial.begin(115200);

    setupModem();
    delay(10);

    if (!SPIFFS.begin(true))
    {
        Serial.println("SPIFFS Mount Failed");
        return;
    }
    SPIFFS.format();
    listDir(SPIFFS, "/", 0);

    // Set GSM module baud rate
    
    delay(3000);

    // Restart takes quite some time
    // To skip it, call init() instead of restart()
    Serial.println("Initializing modem...");
    modem.restart();

    String modemInfo = modem.getModemInfo();
    Serial.print("Modem: ");
    Serial.println(modemInfo);

    // Unlock your SIM card with a PIN
    //modem.simUnlock("1234");
}



void loop()
{
    Serial.print("Waiting for network...");
    if (!modem.waitForNetwork())
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(apn);
    if (!modem.gprsConnect(apn, user, pass))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(server);

    // if you get a connection, report back via serial:
    if (!client.connect(server, port))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");
    
    // Make a HTTP request:
    client.print(String("GET ") + resource + " HTTP/1.0\r\n");
    client.print(String("Host: ") + server + "\r\n");
    client.print("Connection: close\r\n\r\n");

    long timeout = millis();
    while (client.available() == 0)
    {
        if (millis() - timeout > 5000L)
        {
            Serial.println(">>> Client Timeout !");
            client.stop();
            delay(10000L);
            return;
        }
    }

    Serial.println("Reading header");
    uint32_t contentLength = knownFileSize;

    File file = SPIFFS.open("/update.bin", FILE_APPEND);


    while (client.available())
    {
        String line = client.readStringUntil('\n');
        line.trim();
        //Serial.println(line);    // Uncomment this to show response header
        line.toLowerCase();
        if (line.startsWith("content-length:"))
        {
            contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
        }
        else if (line.length() == 0)
        {
            break;
        }
    }


    timeout = millis();
    uint32_t readLength = 0;
    CRC32 crc;

    unsigned long timeElapsed = millis();
    printPercent(readLength, contentLength);
    

    while (readLength < contentLength && client.connected() && millis() - timeout < 10000L)
    {
        int i = 0;
        while (client.available())
        {
                // read file data to spiffs
            if (!file.print(char(client.read())))
            {
                Serial.println("Appending file");
            }
            //Serial.print((char)c);       // Uncomment this to show data
            //crc.update(c);
            readLength++;

            if (readLength % (contentLength / 13) == 0)
            {
                printPercent(readLength, contentLength);
            }
            timeout = millis();
        }
    }

    file.close();

    printPercent(readLength, contentLength);
    timeElapsed = millis() - timeElapsed;
    Serial.println();

    client.stop();
    Serial.println("stop client");

    modem.gprsDisconnect();
    Serial.println("gprs disconnect");
    Serial.println();

    float duration = float(timeElapsed) / 1000;
  /*
    Serial.print("Tamaño de Archivo: ");
    Serial.println(contentLength);
    Serial.print("Leido:  ");
    Serial.println(readLength);
    Serial.print("Calculado. CRC32:    0x");
    Serial.println(crc.finalize(), HEX);
    Serial.print("Conocido CRC32:    0x");
    Serial.println(knownCRC32, HEX);
    Serial.print("Bajado en:       ");
    Serial.print(duration);
    Serial.println("s");

    Serial.println("Se genera una espera de 3 segundos");
    for (int i = 0; i < 3; i++)
    {
        Serial.print(String(i) + "...");
        delay(1000);
    }
  */
    //readFile(SPIFFS, "/update.bin");

    updateFromFS();

    // Do nothing forevermore
    while (true)
    {
        delay(1000);
    }
}







void appendFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if (!file)
    {
        Serial.println("Failed to open file for appending");
        return;
    }
    if (file.print(message))
    {
        Serial.println("APOK");
    }
    else
    {
        Serial.println("APX");
    }
}

void readFile(fs::FS &fs, const char *path)
{
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if (!file || file.isDirectory())
    {
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while (file.available())
    {
        Serial.write(file.read());
        delayMicroseconds(100);
    }
}

void writeFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if (!file)
    {
        Serial.println("Failed to open file for writing");
        return;
    }
    if (file.print(message))
    {
        Serial.println("File written");
    }
    else
    {
        Serial.println("Write failed");
    }
}

void listDir(fs::FS &fs, const char *dirname, uint8_t levels)
{
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if (!root)
    {
        Serial.println("Failed to open directory");
        return;
    }
    if (!root.isDirectory())
    {
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while (file)
    {
        if (file.isDirectory())
        {
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if (levels)
            {
                listDir(fs, file.name(), levels - 1);
            }
        }
        else
        {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void deleteFile(fs::FS &fs, const char *path)
{
    Serial.printf("Deleting file: %s\n", path);
    if (fs.remove(path))
    {
        Serial.println("File deleted");
    }
    else
    {
        Serial.println("Delete failed");
    }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin");
    if (updateBin)
    {
        if (updateBin.isDirectory())
        {
            Serial.println("Directory error");
            updateBin.close();
            return;
        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("Starting update");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, archivo vacío");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("no such binary");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("Writes : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("Written only : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA finished!");
            if (Update.isFinished())
            {
                Serial.println("Restart ESP device!");
                ESP.restart();
            }
            else
            {
                Serial.println("OTA not fiished");
            }
        }
        else
        {
            Serial.println("Error occured #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("Cannot beggin update");
    }
}





void printPercent(uint32_t readLength, uint32_t contentLength)
{
    // If we know the total length
    if (contentLength != -1)
    {
        Serial.print("\r ");
        Serial.print((100.0 * readLength) / contentLength);
        Serial.print('%');
    }
    else
    {
        Serial.println(readLength);
    }
}



void setupModem()
{
#ifdef MODEM_RST
    // Keep reset high
    pinMode(MODEM_RST, OUTPUT);
    digitalWrite(MODEM_RST, HIGH);
#endif

    pinMode(MODEM_PWRKEY, OUTPUT);
    pinMode(MODEM_POWER_ON, OUTPUT);

    // Turn on the Modem power first
    digitalWrite(MODEM_POWER_ON, HIGH);

    // Pull down PWRKEY for more than 1 second according to manual requirements
    digitalWrite(MODEM_PWRKEY, HIGH);
    delay(100);
    digitalWrite(MODEM_PWRKEY, LOW);
    delay(1000);
    digitalWrite(MODEM_PWRKEY, HIGH);

    // Initialize the indicator as an output
    pinMode(LED_GPIO, OUTPUT);
    digitalWrite(LED_GPIO, LED_OFF);
}

[gfreund]

There's nothing wrong per se with your code/solution from what I can tell. However two considerations:

GSP/GPRS/3G services will stop working in the US. Your solution might stop working within a couple of years depending on the carrier you choose. The reason behind this is that the carriers are recycling the 3G frequencies for 5G use. At one point your Sim800 modem won't be able to connect at all. I've also seen carriers selectively switching 3G support off on some cells or refusing to activate new 3G devices. I would recommend a modem that supports LTE-Cat-M1 and NB-IoT. For some background see https://www.lightreading.com/5g/goodbye ... -id/763362.

The ESP_OTA_... solution I mentioned in the previous post has significant advantages over stock Arduino. By using 2 or more OTA partitions you can implement a roll-back solution. Should something go wrong with the upgrade (for example the upgrade crashes before the system finishes all self-tests) the bootloader can switch back to the previous version. This also includes an anti-rollback scheme to prevent a rollback to a version with known security vulnerabilities.

However none of this might be important to you particularly if you don't plan a commercial rollout.

~G