Problem uploading file with WiFiClientSecure

[grasvezel]

Hi,

I'm working on a datalogger. The thing is supposed to pass measurements on to a webserver. That part works. Over SSL using WiFiClientSecure. If the network or server is unavailable, I want to store data on an SD card. Works too. But, when I'm trying to upload this (text) file to the webserver over SSL, it fails. I receive only the first 5 bytes of the file (excluding the HTTP headers) and that's it. If I use WiFiClient and change the port from 443 to 80 it works fine, but I want to use SSL (core debug level set to ERROR) I get:

[E][ssl_client.cpp:28] handle_error(): UNKNOWN ERROR CODE (004E)
[E][ssl_client.cpp:30] handle_error(): MbedTLS message code: -78

If I compile with core debug level set to VERBOSE, the exact same code works fine and I am able to upload a file using SSL. Terribly slow, as there is a lot of debug info to be written. It only works with VERBOSE, if I lower the debug level to DEBUG I get the same error as pasted above.

This seems like a bug to me. I would really appreciate it if someone would be willing to look at this. Let me know if you need more input.

[grasvezel]

No one? Wrong category? No one interested in using the security features of the ESP32 (that would be consistent with most IoT devices out there), or just no one who has a solution?

[ESP_Sprite]

To be fair, you're not giving us much to go on... it would help if you could e.g. post the source code of the bit failing. If you really do think this is a SDK bug, I suggest you create an issue in the ESP32 Arduino Github, as the forum is meant as a more casual way of conversing.

[grasvezel]

Thanks for pointing that out. I'm quite sure it's a bug, but I don't know if it's a bug in the SDK or in WiFiClientSecure.

The code is below. This works, but if I change WiFiClient to WiFiClientSecure (and change the port from 80 to 443) it works only if I compile the sketch with verbose debugging enabled. Without verbose debugging, I only receive the first 5 bytes of the file.

The 'file received' thingy is sent as a reply by the server to indicate the succesfull reception of the file, just to make sure not to delete it from the SD card if something went wrong.

Code: Select all

// file upload from SD
void uploadSDfile(String filename, bool delAfterUpload) {
  if(SD.begin()) {
    File file = SD.open(filename);
    if(!file) {
      addLog(LOG_LEVEL_ERROR, "SD   : Unable to open " + filename);
      file.close();
      return;
    }
    SD.end();
    String url = "/upload/?file=log&id=" + String(chipMAC);
    char* server = DEFAULT_LOG_HOST;
    WiFiClient uploadclient;
    int port = 80;
    if (!uploadclient.connect(server, port)) {
      // Don't write to log. File is opened!
      Serial.println("                    WEBCL: Connection to " + String(server) + " failed!");
    } else {
      Serial.println("                    WEBCL: Connected to " + String(server));
      uploadclient.println("POST " + url + " HTTP/1.1");
      uploadclient.println("Host: " + String(server));
      uploadclient.println("Connection: close");
      uploadclient.println("Content-type: text/plain");
      uploadclient.println("Content-length: " + String(file.size()));
      uploadclient.println();
      // send file contents to webserver
      while (file.available() && uploadclient.connected()) {
        uploadclient.print((char)file.read());
      }
      file.close();
      while (uploadclient.connected()) {
        String line = uploadclient.readStringUntil('\n');
        if(line == "*file received*") {
           Serial.println("                    UPLD : File received!");
           if(delAfterUpload) {
             Serial.println("                    UPLD : Deleting file " + filename);
             if(SD.remove(filename)) {
               addLog(LOG_LEVEL_INFO, "UPLD : File " + filename + " deleted after upload");
             }
           }
        }
      }
      uploadclient.stop();
    }
  } else {
    addLog(LOG_LEVEL_ERROR, "SD   : Unable to mount SD card");
  }
}

[tele_player]

Probably unrelated, but seems to me all access to the SD card should be between SD.begin() and SD.end().

[grasvezel]

It is. I had some trouble pasting the code and I messed it up.

[grasvezel]

I've solved this issue. Well, I didn't really solve it, but I found a workaround. After talking to copercini (the author of the lib) and doing some experimenting with adding delays at various points in the code (wich helped a bit but didn't really solve the issue), I modified my code to send larger packets. It turns out the library does not buffer bytes written until the IP packet to be sent over WiFi is full, but it sends out a packet for every .write(). So I decided to use a String as a buffer. After that modification to the code, I can now upload larger files (I tested up to about 1.5MB) over HTTPS. This is the relevant part of the code:

Code: Select all

    // upload the file in 1350 byte chunks
    while(file.available()) {
      int nextPacketSize = file.available();
      if (nextPacketSize > 1350) {
        nextPacketSize = 1350;
      }
      String buffer = "";
      for(int i=0;i<nextPacketSize;i++) {
        buffer += (char)file.read();
      }
      uploadclient.print(buffer);
    }

This now works with both WiFiClient and WiFiClientSecure.