【求助】ESP32通过spi外接flash的fat格式与PC不能兼容？

[jame113]

求助， 如题， 参照官方demo:ext_flash_fatfs， 外接flash, 问题如下：
1、如果通过esp32格式化flash,读写都正常， 但使用外接设备将flash通过u盘模式连接到windows后， 显示磁盘没有格式化。
2、如果在windows将flash格式化成fat格式后， esp32连接flash就没法挂在， 显示日志如下。 但如果开启格式化参数后， 还是可以重新格式化的。 esp32自动格式化后， 问题同1。
3、如果使用的是苹果系统，将esp32格式化后，在苹果里面能够读取，但写入的数据， 通过esp32读取，能够找到文件， 但读取到的文件内容却是错误的
4、menuconfig中， CONFIG_FATFS_CODEPAGE选项已修改为936,如果用默认的437，那么苹果系统可以看到flash中的文件， 但读取到的内容为空。

Flash型号为华邦的W25Q32, IDF版本为4.1，连接方式与官网demo：ext_flash_fatfs一致，编译环境为mac os

Flash在windows中格式化为fat后，ESP32不能mount的日志如下：

Code: Select all

I (6231) example: Initializing external SPI Flash
I (6231) example: Pin assignments:
I (6231) example: MOSI: 23   MISO: 19   SCLK: 18   CS:  5
D (6235) spi: SPI3 use iomux pins.
V (6238) intr_alloc: esp_intr_alloc_intrstatus (cpu 0): checking args
V (6245) intr_alloc: esp_intr_alloc_intrstatus (cpu 0): Args okay. Resulting flags 0x80E
D (6253) intr_alloc: Connected src 31 to int 13 (cpu 0)
D (6258) FLASH_HAL: extra_dummy: 0
V (6262) memspi: raw_chip_id: 1640EF

V (6266) memspi: chip_id: EF4016

V (6269) memspi: raw_chip_id: 1640EF

V (6273) memspi: chip_id: EF4016

D (6276) spi_flash: trying chip: issi
D (6280) spi_flash: trying chip: gd
D (6284) spi_flash: trying chip: generic
I (6288) spi_flash: detected chip: generic
I (6293) spi_flash: flash io: dio
V (6297) memspi: raw_chip_id: 1640EF

V (6300) memspi: chip_id: EF4016

I (6304) example: Initialized external Flash, size=4096 KB, ID=0xef4016
I (6311) example: Adding external Flash as a partition, label="storage", size=4096 KB
D (6320) partition: Loading the partition table
V (6325) calculated md5: 0x3ffb4828   bf 25 82 2c 0f a6 d8 64  1b d9 30 25 1e 06 b4 c4  |.%.,...d..0%....|
V (6334) stored md5: 0x3f4180d0   bf 25 82 2c 0f a6 d8 64  1b d9 30 25 1e 06 b4 c4  |.%.,...d..0%....|
D (6343) partition: Partition table MD5 verified
I (6348) example: Listing data partitions:
I (6353) example: - partition 'nvs', subtype 2, offset 0x9000, size 16 kB
I (6360) example: - partition 'otadata', subtype 0, offset 0xd000, size 8 kB
I (6368) example: - partition 'phy_init', subtype 1, offset 0xf000, size 4 kB
I (6376) example: - partition 'storage', subtype 129, offset 0x0, size 4096 kB
I (6384) example: Mounting FAT filesystem
V (6388) wl_flash: config start_addr=0x00000000, full_mem_size=0x00400000, page_size=0x00001000, sector_size=0x00001000, updaterate=0x00000010, wr_size=0x00000010, version=0x00000002, temp_buff_size=0x00000020
D (6407) wl_flash: config - config result: state_size=0x00005000, cfg_size=0x00001000, addr_cfg=0x003ff000, addr_state1=0x003f5000, addr_state2=0x003fa000, flash_size=0x003f4000
D (6423) wl_flash: init - config ID=2, stored ID=2, access_count=0, block_size=4096, max_count=16, pos=0, move_count=0x00000000
D (6434) wl_flash: init starts: crc1= 0x86e1885b, crc2 = 0x86e1885b, this->state.crc= 0x86e1885b, state_copy->crc= 0x86e1885b, version=2, read_version=2
D (6448) wl_flash: init: crc1=0x86e1885b, crc2 = 0x86e1885b, result= 0x00000000
V (6456) wl_flash: recoverPos start
V (6459) wl_flash: recoverPos - check pos: result=0x00000000, position= 0, pos_bits= 0x00000000
D (6468) wl_flash: recoverPos - this->state.pos= 0x00000000, position= 0x00000000, result= 0x00000000, max_pos= 0x000003f5
V (6479) wl_flash: recoverPos done
D (6482) wl_flash: init - move_count= 0x00000000
D (6487) vfs_fat_spiflash: using pdrv=0
V (6491) ff_diskio_spiflash: ff_wl_ioctl: cmd=2

V (6496) ff_diskio_spiflash: ff_wl_read - pdrv=0, sector=0, count=1

D (6502) wl_flash: read - src_addr= 0x00000000, size= 0x00001000
V (6508) wl_flash: calcAddr - addr= 0x00000000 -> result= 0x00001000, dummy_addr= 0x00000000
W (6518) vfs_fat_spiflash: f_mount failed (13)
E (6522) example: Failed to mount FATFS (ESP_FAIL)

[ESP_igrr]

Hi jame113, sorry for replying in English.
ext_flash_fatfs example uses ESP-IDF wear_levelling library as an intermediate layer between FATFS and flash. The purpose of wear_levelling library is to reorder the sectors and distribute erase operations across the flash partition, effectively extending the lifetime of SPI Flash device. If you use an external device to make the same SPI Flash accessible over USB, then the external device will see the sectors in the wrong order, as wear_levelling reorders the sectors. This likely explains why ESP-IDF cannot read the data written by the host, and vice versa.

[jame113]

Hi jame113, sorry for replying in English.
ext_flash_fatfs example uses ESP-IDF wear_levelling library as an intermediate layer between FATFS and flash. The purpose of wear_levelling library is to reorder the sectors and distribute erase operations across the flash partition, effectively extending the lifetime of SPI Flash device. If you use an external device to make the same SPI Flash accessible over USB, then the external device will see the sectors in the wrong order, as wear_levelling reorders the sectors. This likely explains why ESP-IDF cannot read the data written by the host, and vice versa.

非常感谢， 我找下没有使用war_leveling liberay的例子试试，如果您有类似示例，麻烦回复我， 多谢。

[jame113]

尝试修改官方的例子， 在mount时， 使用：esp_vfs_fat_rawflash_mount替换原来的esp_vfs_fat_spiflash_mount，
可以mount成功，也可以写入成功， 写入成功后立即读取， 读取到的内容与写入内容不符，
重启esp32后重新读取写入的文件，显示fresult=4, 查了下，好像是文件不存在的错误码。

Code: Select all

I (6723) example: Mounting FAT filesystem 2
D (6728) vfs_fat_spiflash: using pdrv=0
V (6732) diskio_rawflash: ff_raw_ioctl: cmd=2n
V (6736) diskio_rawflash: ff_raw_read - pdrv=0, sector=0, count=1n
E (6744) example: Mount Success
V (6746) diskio_rawflash: ff_raw_read - pdrv=0, sector=1, count=1n
I (6755) example: FAT FS: 4024 kB total, 4020 kB free
I (6758) example: ===========================重启后第一次读取错误=============================
I (6766) example: Reading file: 4.mp3
V (6770) vfs_fat: vfs_fat_open: path="/4.mp3", flags=0, mode=1b6
V (6776) diskio_rawflash: ff_raw_read - pdrv=0, sector=2, count=1n
D (6784) vfs_fat: vfs_fat_open: fresult=4
E (6786) example: Failed to open file for reading
I (6792) example: ============================写入文件============================
I (6800) example: Opening file
V (6804) vfs_fat: vfs_fat_open: path="/4.mp3", flags=601, mode=1b6
E (6810) example: Written rand:-641189875
D (6815) vfs_fat: vfs_fat_write: fresult=1
D (6819) vfs_fat: vfs_fat_close: fresult=1
I (6823) example: File written
I (6827) example: ===========================写入后立即读取=============================
I (6835) example: Reading file: 4.mp3
V (6839) vfs_fat: vfs_fat_open: path="/4.mp3", flags=0, mode=1b6
E (6845) example: Read from file hex: 8048fb3f0c //读取到的内容与写入内容不符

[jame113]

如上，补充说明：
使用esp_vfs_fat_rawflash_mount替换原来的esp_vfs_fat_spiflash_mount后，
如果是使用windows格式化的fat32，则不能mount成功；
如果先用esp_vfs_fat_spiflash_mount，进行格式化，然后在用esp_vfs_fat_rawflash_mount就可以挂载成功。

[ESP_igrr]

Hi Jame113,
esp_vfs_fat_rawflash_mount intentionally mounts the filesystem in read-only mode. This is because creating a FAT filesystem in SPI NOR Flash without wear levelling is not a very good idea. FAT filesystem very often writes to a few sectors at the beginning of the partition, and it is quite easy to exceed 100k erase cycles for these sectors even after short time. If you think you won't exceed 100k erase cycles in your specific use case, you can hack the "rawflash" driver to add write support. The function which should implement write operations is ff_raw_write. You can implement it similar to ff_raw_read, calling esp_partition_erase_range and esp_partition_write. Note that FAT sector size has to be 4096 bytes in this case, since flash can only be erased in 4096 byte units.

[jame113]

收到， 非常感谢，我试试您说的办法。
我们因为有其他的第三方mcu需要与esp32一起读写flash， 所以官方的wear levelling中间层虽然很好， 但我们用不了；并且我们的写次数是非常有限的。

[jame113]

还是mount不了Windows下格式化的fat格式， 能否帮看看问题出在哪？谢谢。
使用IDF版本： v4.1.1-351-g935deb4082
编译环境：macOS

mount时日志如下：

Code: Select all

I (8225) example: Initializing external SPI Flash
I (8225) example: Pin assignments:
I (8225) example: MOSI: 23   MISO: 19   SCLK: 18   CS:  5
D (8229) spi: SPI3 use iomux pins.
V (8232) intr_alloc: esp_intr_alloc_intrstatus (cpu 0): checking args
V (8239) intr_alloc: esp_intr_alloc_intrstatus (cpu 0): Args okay. Resulting flags 0x80E
D (8247) intr_alloc: Connected src 31 to int 13 (cpu 0)
D (8252) FLASH_HAL: extra_dummy: 0
V (8256) memspi: raw_chip_id: 1640EF

V (8260) memspi: chip_id: EF4016

V (8263) memspi: raw_chip_id: 1640EF

V (8267) memspi: chip_id: EF4016

D (8270) spi_flash: trying chip: issi
D (8274) spi_flash: trying chip: gd
D (8278) spi_flash: trying chip: generic
I (8282) spi_flash: detected chip: generic
I (8287) spi_flash: flash io: dio
V (8291) memspi: raw_chip_id: 1640EF

V (8294) memspi: chip_id: EF4016

I (8298) example: Initialized external Flash, size=4096 KB, ID=0xef4016
I (8305) example: Adding external Flash as a partition, label="storage", size=4096 KB
D (8314) partition: Loading the partition table
V (8319) calculated md5: 0x3ffb4788   bf 25 82 2c 0f a6 d8 64  1b d9 30 25 1e 06 b4 c4  |.%.,...d..0%....|
V (8328) stored md5: 0x3f4180d0   bf 25 82 2c 0f a6 d8 64  1b d9 30 25 1e 06 b4 c4  |.%.,...d..0%....|
D (8337) partition: Partition table MD5 verified
I (8342) example: Listing data partitions:
I (8347) example: - partition 'nvs', subtype 2, offset 0x9000, size 16 kB
I (8354) example: - partition 'otadata', subtype 0, offset 0xd000, size 8 kB
I (8362) example: - partition 'phy_init', subtype 1, offset 0xf000, size 4 kB
I (8370) example: - partition 'storage', subtype 129, offset 0x0, size 4096 kB
I (8378) example: Mounting FAT filesystem
D (8382) vfs_fat_spiflash: using pdrv=0
V (8386) diskio_rawflash: ff_raw_ioctl: cmd=2n
V (8391) diskio_rawflash: ff_raw_read - pdrv=0, sector=0, count=1n
W (8401) vfs_fat_spiflash: f_mount failed (13)
E (8406) example: Failed to mount FATFS (ESP_FAIL)

1、windows格式化时，选择的是4096byte。
2、相关的配置项如下：

Code: Select all

# SPI Flash driver
#
# CONFIG_SPI_FLASH_VERIFY_WRITE is not set
# CONFIG_SPI_FLASH_ENABLE_COUNTERS is not set
CONFIG_SPI_FLASH_ROM_DRIVER_PATCH=y
CONFIG_SPI_FLASH_DANGEROUS_WRITE_ABORTS=y
# CONFIG_SPI_FLASH_DANGEROUS_WRITE_FAILS is not set
# CONFIG_SPI_FLASH_DANGEROUS_WRITE_ALLOWED is not set
# CONFIG_SPI_FLASH_BYPASS_BLOCK_ERASE is not set
CONFIG_SPI_FLASH_YIELD_DURING_ERASE=y
CONFIG_SPI_FLASH_ERASE_YIELD_DURATION_MS=60
CONFIG_SPI_FLASH_ERASE_YIELD_TICKS=1
# CONFIG_SPI_FLASH_SIZE_OVERRIDE is not set

#
# Auto-detect flash chips
#
CONFIG_SPI_FLASH_SUPPORT_ISSI_CHIP=y
CONFIG_SPI_FLASH_SUPPORT_GD_CHIP=y
# end of Auto-detect flash chips

# CONFIG_SPI_FLASH_ENABLE_ENCRYPTED_READ_WRITE is not set
# end of SPI Flash driver


#
# FAT Filesystem support
#
CONFIG_FATFS_CODEPAGE_936=y
# CONFIG_FATFS_CODEPAGE_949 is not set
# CONFIG_FATFS_CODEPAGE_950 is not set
CONFIG_FATFS_CODEPAGE=936
CONFIG_FATFS_LFN_NONE=y
# CONFIG_FATFS_LFN_HEAP is not set
# CONFIG_FATFS_LFN_STACK is not set
CONFIG_FATFS_FS_LOCK=0
CONFIG_FATFS_TIMEOUT_MS=10000
CONFIG_FATFS_PER_FILE_CACHE=y
# end of FAT Filesystem support

3、mout的代码

Code: Select all

	ESP_LOGI(TAG, "Mounting FAT filesystem");
	const esp_vfs_fat_mount_config_t mount_config = {
		.max_files = 4,
		.format_if_mount_failed = true,//无效
		.allocation_unit_size = CONFIG_WL_SECTOR_SIZE//无效
		};

	esp_err_t err = esp_vfs_fat_rawflash_mount(base_path, partition_label, &mount_config);
	if (err != ESP_OK)
	{
		ESP_LOGE(TAG, "Failed to mount FATFS (%s)", esp_err_to_name(err));
		return false;
	}
	return true;

4、ff_raw_write函数

Code: Select all

DRESULT ff_raw_write(BYTE pdrv, const BYTE *buff, DWORD sector, UINT count)
{
	//esp_partition_erase_range and esp_partition_write

	ESP_LOGV(TAG, "ff_raw_write - pdrv=%i, sector=%i, count=%in", (unsigned int)pdrv, (unsigned int)sector, (unsigned int)count);
	const esp_partition_t *part = ff_raw_handles[pdrv];
	assert(part);
	esp_err_t err = esp_partition_erase_range(part, sector * SPI_FLASH_SEC_SIZE, count * SPI_FLASH_SEC_SIZE);
	if (unlikely(err != ESP_OK))
	{
		ESP_LOGE(TAG, "esp_partition_write erase failed (0x%x)", err);
		return RES_ERROR;
	}
	ESP_LOGV(TAG, "esp_partition_write erase success, offset: %d, size:%d", sector * SPI_FLASH_SEC_SIZE, count * SPI_FLASH_SEC_SIZE);

	err = esp_partition_write(part, sector * SPI_FLASH_SEC_SIZE, buff, count * SPI_FLASH_SEC_SIZE);
	if (unlikely(err != ESP_OK))
	{
		ESP_LOGE(TAG, "esp_partition_write failed (0x%x)", err);
		return RES_ERROR;
	}

	ESP_LOGV(TAG, "ff_raw_write - success");
	return RES_OK;
}

[ESP_igrr]

I think the reason why you can't mount the filesystem created by Windows is that the firmware code uses 4096 bytes as the sector size.
When you format the partition in Windows, it allows you to choose the cluster size, not sector size. Windows sets sector size to 512 bytes.

You can try to modify the code in diskio_rawflash as follows:
1. In ff_raw_ioctl, return 512 bytes from GET_SECTOR_SIZE.
2. In the same function, change GET_SECTOR_COUNT calculation to divide the size by 512.
3. In ff_raw_read use 512 bytes instead of SPI_FLASH_SEC_SIZE as the unit.
4. In ff_raw_write, the logic becomes more complex. Allocate a temporary 4kB buffer, read the 4kb aligned flash sector containing the block you need into the buffer. Replace the part that needs to be overwritten. Erase the 4kB flash sector. Write the 4kb sector to flash.

Note that if your device can be suddenly powered off, this read-modify-write approach in ff_raw_write may result in a loss of data. I don't think it makes things a lot worse, though, as FATFS has pretty poor tolerance to such power-off scenarios anyway.

[jame113]

多谢， 我先试试mac或linux下格式化， 确认扇区大小为4096byte, 如果还是不行， 就按您的意见尝试修改代码。

I think the reason why you can't mount the filesystem created by Windows is that the firmware code uses 4096 bytes as the sector size.
When you format the partition in Windows, it allows you to choose the cluster size, not sector size. Windows sets sector size to 512 bytes.

You can try to modify the code in diskio_rawflash as follows:
1. In ff_raw_ioctl, return 512 bytes from GET_SECTOR_SIZE.
2. In the same function, change GET_SECTOR_COUNT calculation to divide the size by 512.
3. In ff_raw_read use 512 bytes instead of SPI_FLASH_SEC_SIZE as the unit.
4. In ff_raw_write, the logic becomes more complex. Allocate a temporary 4kB buffer, read the 4kb aligned flash sector containing the block you need into the buffer. Replace the part that needs to be overwritten. Erase the 4kB flash sector. Write the 4kb sector to flash.

Note that if your device can be suddenly powered off, this read-modify-write approach in ff_raw_write may result in a loss of data. I don't think it makes things a lot worse, though, as FATFS has pretty poor tolerance to such power-off scenarios anyway.