ESP32 Forum

Posted: **Fri Aug 23, 2024 8:02 am**

I am currently using the fatfs library to allocate a contiguous file for my project. The problem is that I think I found a bug in the fatfs library, for example I want to first try opening a file with FA_OPEN_EXISTING, and if the file doesn't exist and the function return fail, I'd like to fall back to creating it with FA_CREATE_NEW.

The problem is that, and I have no idea why, when I put FA_CREATE_NEW as a parameter in the second call to f_open for when OPEN_EXISTING fails, it creates the file directly in the first call to f_open and completely ignore the FA_OPEN_EXISTING.

please, can someone try this out to see what I mean? I think I'm going mad it's been 3 hours debugging this and I can't understand why it is made this way.

All I want is open a file if it exists(FA_OPEN_EXISTING), if so execute a conditional block, and if it doesn't execute another conditional block where I try to create it(FA_CREATE_NEW).

if I use FA_CREATE_NEW only once(anywhere in my code) FA_OPEN_EXISTING doesn't work and acts as FA_CREATE_NEW in the first call

I am currently using the esp idf 5.0.2, this looks like a bug to me, if someone could confirm what I think..

Code: Select all

// Attempt to open the file
    fr = f_open(&file, FILE_PATH, FA_OPEN_EXISTING | FA_READ | FA_WRITE);
    if (fr == FR_OK) {
        printf("File opened with success\n");
        printf("Impossible to bypass this path even when the file doesn't exist yet when the second call contains FA_CREATE_NEW \n");
    }
    else if (fr == FR_NO_FILE) {
        printf("Can't open file, creating it\n");
        fr = f_open(&file, FILE_PATH, FA_CREATE_NEW | FA_READ | FA_WRITE);
        if (fr == FR_OK) {
            printf("File created successfully.\n");
            f_expand(&file, 1000000, 1);
        } else {
            printf("Failed creating file\n");
        }
    } else {
        printf("Failed to open file with error: %d\n", fr);
    }

Posted: **Fri Aug 23, 2024 2:47 pm**

After a successful FA_CREATE_NEW, you do delete the file before the next test with FA_OPEN_EXISTING, right?

Posted: **Fri Aug 23, 2024 2:55 pm**

well, for example I definitely tried to change the filename, like if it was "filetext.txt" I rename it "filetex2.txt".
normally I would expect that opening the same file twice execute the 1st code block(FA_CREATE_NEW creates a new file), then the second open would execute the 2nd code block(FA_OPEN_EXISTING creates a new file)

I do not delete the file.

By the way I also tried with the esp idf 5.3 and I get the exact same behavior

Posted: **Fri Aug 23, 2024 3:06 pm**

Hello. I am not sure why is it happening, but can you try something like this? I am using `access` to see if the file exists.

Code: [Select all] [Expand/Collapse]

#include <unistd.h>
 
//...
 
FIL* file = (FIL*) ff_memalloc(sizeof(FIL));
if (file == NULL) { ;/* handle NULL */ }
memset(file, 0, sizeof(FIL));
FRESULT fr = 0;
 
if (access(FILE_PATH, FR_OK) == 0) {
    // file exists
    fr = f_open(file, FILE_PATH, FA_WRITE | FA_READ);
} else {
    // file doesn't exist
    fr = f_open(file, FILE_PATH, FA_CREATE_NEW | FA_WRITE);
    if (fr == FR_OK) {
        fr = f_expand(file, 1000000, 1);
        if (fr != FR_OK) { ;/* handle it */}
    } else { ;/* handle it */}
    f_close(file);
    fr = f_open(file, FILE_PATH, FA_WRITE | FA_READ);
    //...
}
 
//...
 
free(file);
 
 

GeSHi © Codebox Plus Extension

Posted: **Fri Aug 23, 2024 3:09 pm**

kernic wrote: ↑
Fri Aug 23, 2024 2:55 pm
well, for example I definitely tried to change the filename, like if it was "filetext.txt" I rename it "filetex2.txt".
normally I would expect that opening the same file twice execute the 1st code block(FA_CREATE_NEW creates a new file), then the second open would execute the 2nd code block(FA_OPEN_EXISTING creates a new file)

I do not delete the file.

By the way I also tried with the esp idf 5.3 and I get the exact same behavior

Oh, have you enabled long file namew (LFN)? The default for FATFS is 8.3 format (short) file names (SFN), whcih means maximum 8 characters in file name, then a period . and then maximum 3 characters in the extension. If you haven't enabled LFN, then the filename gets truncated and mangled automatically by FATFS. http://elm-chan.org/docs/fat_e.html#name_conversion

Posted: **Fri Aug 23, 2024 7:03 pm**

hahaha, I love your solution where you check if the file exists, I came to a similar conclusion, except my solution doesn't work unless I do a weird conditional check with the file and then do a wait call for at least 3 seconds, sorry if its a bit convoluted with the function:

Code: Select all

FRESULT checkFileExists(const char* path) {
    FILINFO fno;
    FRESULT fr;

    fr = f_stat(path, &fno);

    if (fr == FR_OK) {
        // File exists
        return FR_OK;
    } else if (fr == FR_NO_FILE) {
        // File does not exist
        return FR_NO_FILE;
    } else {
        // An error occurred
        return fr;
    }
}

    FRESULT fileExists = checkFileExists(FILE_PATH);

    if(fileExists == FR_OK) {
    }
    else if(fileExists == FR_NO_FILE) {
    }
    else {
    }

    vTaskDelay(3000 / portTICK_PERIOD_MS);

    f_open(&file, FILE_PATH, FA_OPEN_ALWAYS | FA_READ | FA_WRITE);

    if(fileExists == FR_OK) {
        printf("File opened with success\n");
    }
    else if(fileExists == FR_NO_FILE) {
        printf("init file for the first time\n");

        f_expand(&file, 10000, 1);
    }
    else {
        printf("something happened with the file\n");
    }

as for the names, indeed I can only use 8 characters by default, which I thought was weird but I didn't give much attention to that.. And when I try to put FF_USE_LFN to 2 or 3 it doesn't compile and gives a bunch of error, and option 1 says it can't run in a safe environment when I try to compile. here's some of the generated error logs when FF_USE_LFN=2 :

Code: Select all

C:/Users/Nicolas/esp/v5.3/esp-idf/components/mbedtls/mbedtls/library/x509_crt.c:1612:21: error: implicit declaration of function 'readdir' [-Werror=implicit-function-declaration]
 1612 |     while ((entry = readdir(dir)) != NULL) {
      |                     ^~~~~~~             ^
C:/Users/Nicolas/esp/v5.3/esp-idf/components/mbedtls/mbedtls/library/x509_crt.c:1614:56: error: invalid use of undefined type 'struct dirent'
 1614 |                                    "%s/%s", path, entry->d_name);
      |                                                        ^~
C:/Users/Nicolas/esp/v5.3/esp-idf/components/mbedtls/mbedtls/library/x509_crt.c:1650:5: error: implicit declaration of function 'closedir' [-Werror=implicit-function-declaration]
 1650 |     closedir(dir);

Thanks a lot for all your help!

Posted: **Sat Aug 24, 2024 1:40 am**

ESP_adokitkat wrote: ↑
Fri Aug 23, 2024 3:06 pm
Hello. I am not sure why is it happening, but can you try something like this? I am using `access` to see if the file exists.

Code: [Select all] [Expand/Collapse]
#include <unistd.h>

//...

FIL* file = (FIL*) ff_memalloc(sizeof(FIL));
if (file == NULL) { ;/* handle NULL */ }
memset(file, 0, sizeof(FIL));
FRESULT fr = 0;

if (access(FILE_PATH, FR_OK) == 0) {
// file exists
fr = f_open(file, FILE_PATH, FA_WRITE | FA_READ);
} else {
// file doesn't exist
fr = f_open(file, FILE_PATH, FA_CREATE_NEW | FA_WRITE);
if (fr == FR_OK) {
fr = f_expand(file, 1000000, 1);
if (fr != FR_OK) { ;/* handle it */}
} else { ;/* handle it */}
f_close(file);
fr = f_open(file, FILE_PATH, FA_WRITE | FA_READ);
//...
}

//...

free(file);

GeSHi © Codebox Plus Extension

I just tried your given code with the esp idf 5.3 (I was hoping yours wouldn't need a 3 second delay like mine to work) and unfortunately your solution doesn't seem to work, I remplaced the comments with printf and it always says "file doesn't exist" each time, even the first time.

I'd really like to find a fix for this, as I really need a contiguous file for my small database project. (and also I need to know when a file is newly created)
Thanks again for reading

Posted: **Sat Aug 24, 2024 2:06 am**

Ok, I just noticed a very strange behavior. Even if my code is behaving as 'expected' (provided you delay 3 seconds and execute the weird conditional logic), the file doesn't exist when I insert my sd card in my laptop.. Even if the esp32 is able to read it back again when I mount back the sd card in it..

I hope this will help diagnosing my issue.. Thanks

Posted: **Sat Aug 24, 2024 3:36 am**

So this is my last consecutive post(since I don't want to overwhelm the chat with consecutive posts) but I found this very simple snippet:

Code: Select all

    f_open(&file, FILE_PATH, FA_OPEN_ALWAYS | FA_READ | FA_WRITE);

    if(f_size(&file) != 0) {
        printf("File opened with success\n");
    }
    else {
        printf("File opened for the first time\n");
        f_expand(&file, 10000, 1);
    }

    f_close(&file);

This snippet will actually work, but only once (if after creating the 1st file you change the FILE_PATH, afterward if you directly create a 2nd file with the esp32 it won't appear on windows, also when you create the 2nd file for the first time it erroneously thinks that f_size(&file) != 0.. and it produces the same bug again)

Also I implemented the whole thing with the Standard C Library I/O that has a FILE instead of a FIL and makes use of fopen() instead of f_open(), same behavior. I can provide the code for the FILE version if you want. I think at least the Standard C Library I/O version should be working, normally, even if the fatfs version isn't because of some misconfiguration on the fatfs part.

Thanks again..

Posted: **Sun Aug 25, 2024 5:19 pm**

kernic wrote: ↑
Sat Aug 24, 2024 2:06 am
Ok, I just noticed a very strange behavior. Even if my code is behaving as 'expected' (provided you delay 3 seconds and execute the weird conditional logic), the file doesn't exist when I insert my sd card in my laptop.. Even if the esp32 is able to read it back again when I mount back the sd card in it..

I hope this will help diagnosing my issue.. Thanks

I would try to reformat the sd card, maybe the filesystem got a bit corrupted?

Or this may be related to cached information not being flushed, as in FATFS it is only done when f_close() or f_sync() is called (f_sync doesn't close the file descriptor). You can try it instead of waiting 3s. http://elm-chan.org/fsw/ff/doc/sync.html

If you updated to v5.3, can you try `esp_vfs_fat_create_contiguous_file` and `esp_vfs_fat_test_contiguous_file` functions? Here you can see the implementation https://github.com/espressif/esp-idf/bl ... at.c#L1309 and here tests for it https://github.com/espressif/esp-idf/bl ... on.c#L1162

Are you familiar with `idf.py` script and/or its `menuconfig` option? I am sorry but I don't use the VS code extension so I am not sure how to set it there. In menuconfig you can find `(Top) → Component config → FAT Filesystem support` and there are settings for LFN and also "Enable automatic f_sync" (but I would not enable it in your case if you need performance, just call f_sync yourself when you need it). For performance uplift I would also set " Default block size" there to 4092 as well as enable "Enable fast seek algorithm when using lseek function through VFS FAT" (this only works on files opened in read-only mode).

If you don't know/can't use idf.py menuconfig then you can set all of these in Kconfig file named "sdkconfig.defaults" you can put in the root directory of your project (then then do `idf.py fullclean` or just remove the `build` folder and `sdkconfig` and `sdkconfig.old` files before recompilation).

Code: [Select all] [Expand/Collapse]

CONFIG_FATFS_LFN_HEAP=y
CONFIG_FATFS_SECTOR_4096=y
CONFIG_FATFS_USE_FASTSEEK=y
CONFIG_FATFS_FAST_SEEK_BUFFER_SIZE=64
CONFIG_FATFS_VFS_FSTAT_BLKSIZE=4092

ESP32 Forum

can't fail to open file

can't fail to open file

Re: can't fail to open file

Re: can't fail to open file

Re: can't fail to open file

Re: can't fail to open file

Re: can't fail to open file

Re: can't fail to open file

Re: can't fail to open file

Re: can't fail to open file

Re: can't fail to open file