loading AWS certs from SPIFFS

[Trialblazer47]

@ESP_Angus so what do you think this issue could be?

is it with the spiffs.bin file I create with mkspiffs or spiffs component having trouble reading a file?

[ESP_Angus]

I'm not sure. I've asked some colleagues for input as well.

If you can post a full minimal project which demonstrates the problem, as well as your spiffs.bin file and flashing info, then we can try to reproduce it here.

[Trialblazer47]

Hi sure, here I attached modified spiffs example for testing this. also including my spiffs.bin

command to flash that I used:

Code: Select all

 python esptool.py --chip esp32 --port /dev/cu.SLAB_USBtoUART --baud 115200 write_flash -z 0x158000 spiffs.bin

attached is the code.

within spiffs folder, there is a file named spiffs.bin that is the spiffs image containing 2 files aws-root-ca.pem and test.txt
Please check if it works fine for you.

[lil_spli]

Hi

We had a similar problem loading files from SPIFFS. We found that setting file metadata to 0 bytes worked.

CONFIG_SPIFFS_META_LENGTH=0 (make menuconfig > SPIFFS Configuration > Size of per-file metadata field)

Cheers


Si

[Trialblazer47]

Oh wow thanks for that it worked now.

Can I know what is that meta data field?

[Trialblazer47]

Hey it(modifying meta length) worked with the code I had attached but not with AWS IoT subscribe publish example it still gives question marks.
here is output from attached code of spiffs:

Code: Select all

I (276) example: Partition size: total: 52961, used: 6275
I (276) example: FILE EXIST
I (276) example: st->st_size: 1220
I (276) example: Reading file
I (286) example: using fgets()
I (286) example: Read from file: '-----BEGIN CERTIFICATE-----
'
I (296) example: read 63 bytes: MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADC

I (306) example: SPIFFS unmounted

and this is my AWS IoT modified sub_pub code:

Code: Select all

   struct stat st;
    if (stat(ROOT_CA_PATH, &st) == 0) {
      ESP_LOGI(TAG,"FILE EXIST");
      //  unlink("/spiffs/foo.txt");
    }
    ESP_LOGI(TAG,"st->st_size: %d",(int)st.st_size);
    if (stat(DEVICE_CERTIFICATE_PATH, &st) == 0) {
      ESP_LOGI(TAG,"FILE EXIST");
      //  unlink("/spiffs/foo.txt");
    }
    ESP_LOGI(TAG,"st->st_size: %d",(int)st.st_size);
    if (stat(DEVICE_PRIVATE_KEY_PATH, &st) == 0) {
      ESP_LOGI(TAG,"FILE EXIST");
      //  unlink("/spiffs/foo.txt");
    }
    ESP_LOGI(TAG,"st->st_size: %d",(int)st.st_size);
    
   ESP_LOGI(TAG, "Opening file %s",ROOT_CA_PATH);
   FILE* f = fopen(ROOT_CA_PATH, "r");
    char line[1759];
    fgets(line, sizeof(line), f);
    if(f!=NULL){
        line[1758]=0;
        ESP_LOGI(TAG, "Read from file: '%s'", line);
    }else{
        ESP_LOGI(TAG,"Failed to open file");
    }
   if (f == NULL) {
       ESP_LOGE(TAG, "Failed to open file for writing");
       return;
   }
   fclose(f);

    ESP_LOGI(TAG, "Opening file %s",DEVICE_CERTIFICATE_PATH);
    f = fopen(DEVICE_CERTIFICATE_PATH, "r");
    fgets(line, sizeof(line), f);
    if(f!=NULL){
        line[1758]=0;
        ESP_LOGI(TAG, "Read from file: '%s'", line);
    }else{
        ESP_LOGI(TAG,"Failed to open file");
    }
    if (f == NULL) {
        ESP_LOGE(TAG, "Failed to open file for writing");
        return;
    }
    fclose(f);

    ESP_LOGI(TAG, "Opening file %s",DEVICE_PRIVATE_KEY_PATH);
    f = fopen(DEVICE_PRIVATE_KEY_PATH, "r");
    fgets(line, sizeof(line), f);
    if(f!=NULL){
        line[1758]=0;
        ESP_LOGI(TAG, "Read from file: '%s'", line);
    }else{
        ESP_LOGI(TAG,"Failed to open file");
    }
    if (f == NULL) {
        ESP_LOGE(TAG, "Failed to open file for writing");
        return;
    }
    fclose(f);

and out put of it:

Code: Select all

W (286) subpub: spiffs register Status: 0
W (286) subpub: Partition size: total: 52961, used: 6275
W (296) subpub: SPIFFS Mounted.......
I (296) subpub: FILE EXIST
I (306) subpub: st->st_size: 1760
I (306) subpub: FILE EXIST
I (306) subpub: st->st_size: 1220
I (316) subpub: FILE EXIST
I (316) subpub: st->st_size: 1679
I (316) subpub: Opening file /spiffs/aws-root-ca.pem
I (326) subpub: Read from file: '????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????@?@0'
I (396) wifi: n:1 0, o:1 0, ap:255 255, sta:1 0, prof:1
I (476) subpub: Opening file /spiffs/certificate.pem.crt
I (486) subpub: Read from file: '????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????@?@0'
I (626) subpub: Opening file /spiffs/private.pem.key
I (636) subpub: Read from file: '????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????@?@0'
I (1456) wifi: state: init -> auth (b0)
I (1466) wifi: state: auth -> assoc (0)
I (1476) wifi: state: assoc -> run (10)
I (1496) wifi: connected with UNICORN, channel 1
I (2116) event: sta ip: 192.168.43.197, mask: 255.255.255.0, gw: 192.168.43.1
I (2116) subpub: Connecting to AWS...
E (2116) aws_iot: failed!  mbedtls_x509_crt_parse returned -0x3e00 while parsing root cert
E (2126) subpub: Error(-19) connecting to :8883

[Trialblazer47]

ok So I check spiffs examples sdkconfig and compared with AWS config spiffs part and change object name length from 64 to 32 and now one file is read completely(one where I used fread() and other two use fgets()) but other two files not completely

Code: Select all

I (316) subpub: Opening file /spiffs/aws-root-ca.pem
I (326) subpub: Read from file: '-----BEGIN CERTIFICATE-----
'
I (336) subpub: Opening file /spiffs/certificate.pem.crt
I (336) subpub: Read from file: '-----BEGIN CERTIFICATE-----
MIIDWTCCAkGgAwIBAgIUMTzxTRNO0gfZU4eGIjHN77VOaoYwDQYJKoZIhvcNAQEL
BQAwTTFLMEkGA1UECwxCQW1hem9uIFdlYiBTZXJ2aWNlcyBPPUFtYXpvbi5jb20g
SW5jLiBMPVNlYXR0bGUgU1Q9V2FzaGluZ3RvbiBDPVVTMB4XDTE4MDEyOTEyMTgw
MFoXDTQ5MTIzMTIzNTk1OVowHjEcMBoGA1UEAwwTQVdTIElvVCBDZXJ0aWZpY2F0
ZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJ9AE5PFOzAEby4ZV+Cj
7pkXeydp0fwKIwB1UoO/X3JoQkKizOdXtl+PL+sPYFcjdZGHl4cgbwXUEPt5sxGe
C/0w4gZyo2AmsskHPtunzeE2zLrbhwafYPKgLMQ3F7ZK1hJ0qt6UC5lBsZHheoYf
fCQEhwxTvByQOVPrAnL+BwskrxBP1fzniuVtBrr91MIOLmxi41yx0orLKZqMp34N
EbUPwdmK691eYREZT3n3EhWH+co6J9mizaLpXVcbScsBW0WCKTdhoCGFfF1M0Var
iFmmdQPNSjw1SJ5gnXFLDMHR1h+qwVdy8zkRPgnevgmlMwqqbx29mRmENkTWwQ03
/AMCAwEAAaNgMF4wHwYDVR0jBBgwFoAUcLWC9JYd29Uv6U/bvyy+0cDTkHYwHQYD
VR0OBBYEFDtRGU+GVKP65Ak8znci5BJSsQoaMAwGA1UdEwEB/wQCMAAwDgYDVR0P
AQH/BAQDAgeAMA0GCSqGSIb3DQEBCwUAA4IBAQAUF7OjkgF8r63nfHkBijQUQPuL
4M/6U0jVMLlVt/Yw14HppItnh4pVQE/tsCRYiuddkEnCQ175zJskrWjN4PN2kZyW
u1K4/PsylI57X0lNzaWZocNzVJyIM++qKadM4sLY44xMwhr2+UFGDfmc+C+ppfbQ
cZ1t6XIm4Y72B21GUzk0W8ZsrisLgTNVzOCy4Lh8/ac5F/F/yvectG+ae5JX8xgO
ZABaX+9B1fJf5hYUffai0p8AyI6F/EvlL54AXKQgGzRDqif25iFGDIFh4gGmPv1N
sRrUnbAFuLEgINQj6KMoY1oWCL8RjsBpvOjxwpqD4LGgBZaG82/wwQbgxVF9
-----END CERTIFICATE-----
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????@0'????????????????????????@?(
I (396) wifi: n:1 0, o:1 0, ap:255 255, sta:1 0, prof:1
I (486) subpub: Opening file /spiffs/private.pem.key
I (496) subpub: Read from file: '-----BEGIN RSA PRIVATE KEY-----
'
I (1476) wifi: state: init -> auth (b0)

so I think that fread() reads all data but fgets() does not.. also still I get

Code: Select all

E (208826) aws_iot: failed! mbedtls_net_connect returned -0x52
E (208826) subpub: Error(-23) connecting to :8883

I think this I could resolve . (wrong endpoint)

And it worked thanks to every ones support ...without it I would not find the fault.

[WiFive]

fgets will stop reading at line break