【已解决】使用esp-idf-tool离线安装器遇到的子模块更新问题

[山竹君_kk]

电脑环境：win7
安装工具：esp-idf-tools-setup-offline-2.10.exe（官网下载）
备注：在此之前已经安装过esp-idf-v4.2-dev版本的工具链，但因为4.2版本中blufi例程有问题，发帖询问后告知需更新到4.2.2.于是做过以下操作。
1.直接更新过工具链，然后checkout到v4.2.2版本（此时安装也成功 能编译blufi例程，但是之前4.2版本的工程无法编译（工程带LVGL），也不能正常打开menuconfig）。
2.使用过esp-idf-tools在线安装器安装过，不成功。（子模块更新失败）
3.删掉过C:\Users\lw\AppData\Local\Temp路径下的文件（后又通过回收站还原了一部分）

电脑使用的Vscode。之前安装4.2是使用的别人已经安装好了的工具链压缩包 直接解压改路径。

现在的问题：
新建了一个文件夹，使用esp-idf-tools-setup-offline-2.10离线安装器安装，到最后一步出现以下问题
Installation process succeeded.
2021-11-04 14:48:42.376 Extracting temporary file: C:\Users\lw\AppData\Local\Temp\is-BL8J7.tmp\7za.exe
2021-11-04 14:48:42.440 Executing: C:\Users\lw\AppData\Local\Temp\is-BL8J7.tmp\idf-env.exe driver install --ftdi --silabs --espressif
2021-11-04 14:49:28.304 Result: Downloading https://www.silabs.com/documents/public ... Driver.zip to cp210x.zip
Downloading https://www.ftdichip.com/Drivers/CDM/CD ... tified.zip to ftdi.zip
Downloading https://dl.espressif.com/dl/idf-driver/ ... -07-15.zip to idf-driver-esp32-usb-jtag-2021-07-15.zip
Installing driver with INF D:\ESP32\.espressif\tools\idf-driver\silabs-2021-05-03\silabser.inf -> C:\Windows\INF\oem44.inf ... Ok
Installing driver with INF D:\ESP32\.espressif\tools\idf-driver\ftdi-2021-05-03\ftdiport.inf -> C:\Windows\INF\oem45.inf ... Ok
Installing driver with INF D:\ESP32\.espressif\tools\idf-driver\idf-driver-esp32-usb-jtag-2021-07-15\usb_jtag_debug_unit.inf -> C:\Windows\INF\oem46.inf ... Ok

2021-11-04 14:49:28.304 Setting PATH for this process: D:\ESP32\.espressif\tools\idf-python\3.8.7\;D:\ESP32\.espressif\tools\idf-git\2.30.1\cmd\;C:\Program Files\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Common Files\NSEC\;E:\Git\Git\cmd;C:\Program Files\TortoiseGit\bin;E:\InstallSoftware\GitServer\nssm-2.24\win64;;E:\VSCode\Microsoft VS Code\bin;C:\Users\lw\AppData\Local\Programs\Python\Python37;C:\Program Files (x86)\MyDrivers\DriverGenius;C:\Program Files (x86)\MyDrivers\DriverGenius\ksoft
2021-11-04 14:49:28.304 PYTHONNOUSERSITE=True
2021-11-04 14:49:28.304 PYTHONPATH=
2021-11-04 14:49:28.304 PYTHONHOME=
2021-11-04 14:49:28.304 IDFTempPath - location of bundle: D:\ESP32\.espressif\releases\esp-idf-bundle
2021-11-04 14:49:28.304 Updating submodules: D:\ESP32\.espressif\tools\idf-git\2.30.1\cmd\git.exe -C D:\ESP32\esp-idf-v4.2.2 checkout v4.2.2
2021-11-04 14:49:36.360 Done, exit code=0
2021-11-04 14:49:36.360 --------
2021-11-04 14:49:36.360 Running command: D:\ESP32\.espressif\tools\idf-git\2.30.1\cmd\git.exe -C D:\ESP32\esp-idf-v4.2.2 checkout v4.2.2
warning: unable to rmdir 'components/bt/controller/lib_esp32': Directory not empt
warning: unable to rmdir 'components/bt/controller/lib_esp32c3_family': Directory not empt
warning: unable to rmdir 'components/cmock/CMock': Directory not empt
Previous HEAD position was c9646ff0b versions: Update version to 4.3.
HEAD is now at 7ab8f793c versions: Update version to 4.2.

2021-11-04 14:49:36.360 --------
2021-11-04 14:49:36.368 Updating submodules: D:\ESP32\.espressif\tools\idf-git\2.30.1\cmd\git.exe -C D:\ESP32\esp-idf-v4.2.2 submodule update --init --recursive
2021-11-04 14:50:27.321 Done, exit code=1
2021-11-04 14:50:27.321 --------
2021-11-04 14:50:27.321 Running command: D:\ESP32\.espressif\tools\idf-git\2.30.1\cmd\git.exe -C D:\ESP32\esp-idf-v4.2.2 submodule update --init --recursive
Submodule path 'components/asio/asio': checked out '3b66e5b051381fb70de9c2791df70a06181c64e3
Submodule path 'components/bootloader/subproject/components/micro-ecc/micro-ecc': checked out 'd037ec89546fad14b5c4d5456c2e23a71e554966
Submodule path 'components/bt/controller/lib': checked out '1f7e45e2f645f115fc4ca7fc35ecee2d894ff2ea
From https://github.com/espressif/esp-nimbl
94afe271..e4f18624 nimble-1.3.0-idf -> origin/nimble-1.3.0-id
fatal: unable to access 'https://github.com/espressif/esp-nimble.git/': OpenSSL SSL_read: Connection was reset, errno 1005
Fetched in submodule path 'components/bt/host/nimble/nimble', but it did not contain 57d751b93a5532e650ec3787d376876ab0ce367e. Direct fetching of that commit failed

2021-11-04 14:50:27.321 --------
2021-11-04 14:50:27.321 Message box (OK):
Installation has failed with exit code 1
2021-11-04 14:50:43.791 User chose OK.
2021-11-04 14:50:43.798 Message box (Yes/No):
Installation log has been created, it may contain more information about the problem.
Display the installation log now?
2021-11-04 14:50:45.250 User chose Yes.
2021-11-04 14:50:45.325 CurStepChanged raised an exception.


此时安装路径下已经出现了刚才安装的4.2.2版本的工具链。但是其中D:\ESP32\esp-idf-v4.2.2\components路径下有很多子模块没有更新成功文件图标有红色感叹号或红叉（下面有截图附件）。
然后拷贝工具链例程中blufi例程出来，使用VScode打开：打开终端，进入到esp-idf-v4.2.2路径下，运行install.bat ，成功后又运行export.bat也成功。此时已出现
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

idf.py build

然后编译 idf.py build，能编译一些：[337/1198] Building C object esp-idf/lwip/CMakeFiles/__idf_lwip.dir/lwip/src/core/pbuf.c.obj
就这样一直编译，直到出现以下错误：

D:\ESP32\blufi>idf.py build
Executing action: all (aliases: build)
Running ninja in directory d:\esp32\blufi\build
Executing "ninja all"...
[1/586] Performing build step for 'bootloader'
ninja: no work to do.
[2/584] Generating x509_crt_bundle
FAILED: esp-idf/mbedtls/x509_crt_bundle
cmd.exe /C "cd /D D:\ESP32\blufi\build\esp-idf\mbedtls && D:\ESP32\.espressif\python_env\idf4.2_py3.7_env\Scripts\python.exe D:/ESP32/esp-idf-v4.2.2/components/mbedtls/esp_crt_bundle/gen_crt_bundle.py --input D:/ESP32/esp-idf-v4.2.2/components/mbedtls/esp_crt_bundle/cacrt_all.pem -q"
gen_crt_bundle.py: Invalid certificate in D:/ESP32/esp-idf-v4.2.2/components/mbedtls/esp_crt_bundle/cacrt_all.pem
Invalid certificate
ninja: build stopped: subcommand failed.
ninja failed with exit code 1



至此编译失败


然后尝试了以下方法：
进入到esp-idf-v4.2.2路径下打开git bash，运行以下命令
git submodule update --init --recursive
在等待一段时间后出现以下错误：
fatal: unable to access 'https://github.com/espressif/esp-nimble.git/': HTTP/2 stream 1 was not closed cleanly before end of the underlying stream
fatal: Fetched in submodule path 'components/bt/host/nimble/nimble', but it did not contain 57d751b93a5532e650ec3787d376876ab0ce367e. Direct fetching of that commit failed.
再次重复运行改命令，又报错：



fatal: unable to access 'https://github.com/espressif/esp-nimble.git/': OpenSSL SSL_read: Connection was reset, errno 10054
Unable to fetch in submodule path 'components/bt/host/nimble/nimble'; trying to directly fetch 57d751b93a5532e650ec3787d376876ab0ce367e:
From https://github.com/espressif/esp-nimble
* branch 57d751b93a5532e650ec3787d376876ab0ce367e -> FETCH_HEAD
error: The following untracked working tree files would be overwritten by checkout:
.gitignore
.rat-excludes
.travis.yml
CODING_STANDARDS.md
LICENSE
NOTICE
README.md
RELEASE_NOTES.md
apps/blecent/pkg.yml
apps/blecent/src/blecent.h
apps/blecent/src/main.c
apps/blecent/src/misc.c
apps/blecent/src/peer.c
apps/blecent/syscfg.yml
apps/blecsc/README.md
apps/blecsc/pkg.yml
apps/blecsc/src/blecsc_sens.h
apps/blecsc/src/gatt_svr.c
apps/blecsc/src/main.c
apps/blecsc/syscfg.yml
apps/blehci/pkg.yml
apps/blehci/src/main.c
apps/blehci/syscfg.yml
apps/blehr/README.md
apps/blehr/pkg.yml
apps/blehr/src/blehr_sens.h
apps/blehr/src/gatt_svr.c
apps/blehr/src/main.c
apps/blehr/syscfg.yml
apps/blemesh/pkg.yml
apps/blemesh/src/main.c
apps/blemesh/syscfg.yml
apps/blemesh_light/pkg.yml
apps/blemesh_light/src/light_model.c
apps/blemesh_light/src/light_model.h
apps/blemesh_light/src/main.c
apps/blemesh_light/src/ws2812.c
apps/blemesh_light/src/ws2812.h
apps/blemesh_light/syscfg.yml
apps/blemesh_models_example_1/README.md
apps/blemesh_models_example_1/pkg.yml
apps/blemesh_models_example_1/src/main.c
apps/blemesh_models_example_1/syscfg.yml
apps/blemesh_models_example_2/README.md
apps/blemesh_models_example_2/pkg.yml
apps/blemesh_models_example_2/src/app_gpio.c
apps/blemesh_models_example_2/src/app_gpio.h
apps/blemesh_models_example_2/src/ble_mesh.c
apps/blemesh_models_example_2/src/ble_mesh.h
apps/blemesh_models_example_2/src/common.h
apps/blemesh_models_example_2/src/device_composition.c
apps/blemesh_models_example_2/src/device_composition.h
apps/blemesh_models_example_2/src/main.c
apps/blemesh_models_example_2/src/no_transition_work_handler.c
apps/blemesh_models_example_2/src/no_transition_work_handler.h
apps/blemesh_models_example_2/src/publisher.c
apps/blemesh_models_example_2/src/publisher.h
apps/blemesh_models_example_2/src/state_binding.c
apps/blemesh_models_example_2/src/state_binding.h
apps/blemesh_models_example_2/src/storage.c
apps/blemesh_models_example_2/src/storage.h
apps/blemesh_models_example_2/src/transition.c
apps/blemesh_models_example_2/src/transition.h
apps/blemesh_models_example_2/syscfg.yml
apps/blemesh_shell/pkg.yml
apps/blemesh_shell/src/main.c
apps/blemesh_shell/syscfg.yml
apps/bleprph/pkg.yml
apps/bleprph/src/bleprph.h
apps/bleprph/src/gatt_svr.c
apps/bleprph/src/main.c
apps/bleprph/src/misc.c
apps/bleprph/src/phy.c
apps/bleprph/syscfg.yml
apps/btshell/pkg.yml
apps/btshell/src/btshell.h
apps/btshell/src/cmd.c
apps/btshell/src/cmd.h
apps/btshell/src/cmd_gatt.c
apps/btshell/src/cmd_gatt.h
apps/btshell/src/cmd_l2cap.c
apps/btshell/src/cmd_l2cap.h
apps/btshell/src/gatt_svr.c
apps/btshell/src/main.c
apps/btshell/src/misc.c
apps/btshell/src/parse.c
apps/btshell/syscfg.yml
apps/bttester/README
apps/bttester/pkg.yml
apps/bttester/src/atomic.h
apps/bttester/src/bttester.c
apps/bttester/src/bttester.h
apps/bttester/src/bttester_pipe.h
apps/bttester/src/gap.c
apps/bttester/src/gatt.c
apps/bttester/src/glue.c
apps/bttester/src/glue.h
apps/bttester/src/l2cap.c
apps/bttester/src/main.c
apps/bttester/src/mesh.c
apps/bttester/src/rtt_pipe.c
apps/bttester/src/uart_pipe.c
apps/bttester/syscfg.yml
apps/ext_advertiser/pkg.yml
apps/ext_advertiser/src/main.c
apps/ext_advertiser/src/patterns.h
apps/ext_advertiser/syscfg.yml
docs/.gitignore
docs/Makefile
docs/README.rst
docs/ble_hs/ble_att.rst
docs/ble_hs/ble_gap.rst
docs/ble_hs/ble_gattc.rst
docs/ble_hs/ble_gatts.rst
docs/ble_hs/ble_hs.rst
docs/ble_hs/ble_hs_id.rst
docs/ble_hs/ble_hs_return_codes.rst
docs/ble_sec.rst
docs/ble_setup/ble_addr.rst
docs/ble_setup/ble_lp_clock.rst
docs/ble_setup/ble_setup_intro.rst
docs/ble_setup/ble_sync_cb.rst
docs/btshell/btshell_GAP.rst
docs/btshell/btshell_GATT.rst
docs/btshell/btshell_advdata.rst
docs/btshell/btshell_api.rst
docs/conf.py
docs/doxygen.xml
docs/index.rst
docs/mesh/index.rst
docs/mesh/mesh_lightning_model.jpg
docs/mesh/mesh_topology.jpg
docs/mesh/sample.rst
ext/tinycrypt/AUTHORS
ext/tinycrypt/LICENSE
ext/tinycry
error: The following untracked working tree files would be removed by checkout:
.style_ignored_dirs
apps/advertiser/pkg.yml
apps/advertiser/src/main.c
apps/blestress/README.md
apps/blestress/pkg.yml
apps/blestress/src/main.c
apps/blestress/src/misc.c
apps/blestress/src/misc.h
apps/blestress/src/rx_stress.c
apps/blestress/src/rx_stress.h
apps/blestress/src/stress.c
apps/blestress/src/stress.h
apps/blestress/src/stress_gatt.c
apps/blestress/src/stress_gatt.h
apps/blestress/src/tx_stress.c
apps/blestress/src/tx_stress.h
apps/blestress/syscfg.yml
nimble/controller/include/controller/ble_ll_rfmgmt.h
nimble/controller/src/ble_ll_priv.h
nimble/controller/src/ble_ll_rfmgmt.c
nimble/host/mesh/src/nodes.c
nimble/host/mesh/src/nodes.h
nimble/transport/da1469x/cmac_driver/include/cmac_driver/cmac_host.h
nimble/transport/da1469x/cmac_driver/pkg.yml
nimble/transport/da1469x/cmac_driver/scripts/build_libcmac.sh
nimble/transport/da1469x/cmac_driver/src/cmac_host.c
nimble/transport/da1469x/cmac_driver/syscfg.yml
porting/examples/linux/include/logcfg/logcfg.h
porting/examples/linux/include/sysflash/sysflash.h
porting/examples/linux_blemesh/include/logcfg/logcfg.h
porting/examples/linux_blemesh/include/sysflash/sysflash.h
porting/nimble/include/log_common/ignore.h
porting/nimble/include/log_common/log_common.h
porting/nimble/include/logcfg/logcfg.h
porting/nimble/include/os/util.h
porting/nimble/include/sysflash/sysflash.h
porting/npl/riot/include/logcfg/logcfg.h
porting/npl/riot/include/sysflash/sysflash.h
porting/targets/dummy_app/pkg.yml
porting/targets/dummy_app/src/dummy.c
porting/targets/dummy_bsp/bsp.yml
porting/targets/dummy_bsp/include/bsp/bsp.h
porting/targets/dummy_bsp/pkg.yml
porting/targets/linux/pkg.yml
porting/targets/linux/syscfg.yml
porting/targets/linux/target.yml
porting/targets/linux_blemesh/pkg.yml
porting/targets/linux_blemesh/syscfg.yml
porting/targets/linux_blemesh/target.yml
porting/targets/porting_default/pkg.yml
porting/targets/porting_default/target.yml
porting/targets/riot/pkg.yml
porting/targets/riot/syscfg.yml
porting/targets/riot/target.yml
porting/update_generated_files.sh
targets/unittest/pkg.yml
targets/unittest/target.yml
uncrustify.cfg
Please move or remove them before you switch branches.
Aborting
error: The following untracked working tree files would be overwritten by checkout:
.appveyor.yml
.gitattributes
.gitignore
.tag
.travis.yml
Doxyfile
LICENSE
Makefile
README
TODO
VERSION
examples/examples.pro
examples/simplereader.c
examples/simplereader.pro
scripts/maketag.pl
scripts/update-docs.sh
src/cbor.dox
src/cbor.h
src/cborencoder.c
src/cborencoder_close_container_checked.c
src/cborerrorstrings.c
src/cborinternal_p.h
src/cborjson.h
src/cborparser.c
src/cborparser_dup_string.c
src/cborpretty.c
src/cborpretty_stdio.c
src/cbortojson.c
src/cborvalidation.c
src/compilersupport_p.h
src/open_memstream.c
src/parsetags.pl
src/src.pri
src/tags.txt
src/tinycbor-version.h
src/tinycbor.pro
src/utf8_p.h
tests/.gitignore
tests/c90/c90.pro
tests/c90/tst_c90.c
tests/cpp/cpp.pro
tests/cpp/tst_cpp.cpp
tests/encoder/encoder.pro
tests/encoder/tst_encoder.cpp
tests/parser/parser.pro
tests/parser/tst_parser.cpp
tests/tests.pro
tests/tojson/tojson.pro
tests/tojson/tst_tojson.cpp
tinycbor.pc.in
tools/cbordump/cbordump.c
tools/cbordump/cbordump.pro
tools/json2cbor/json2cbor.c
tools/json2cbor/json2cbor.pro
Please move or remove them before you switch branches.
Aborting
fatal: unable to access 'https://github.com/espressif/esp32-wifi-lib.git/': OpenSSL SSL_read: Connection was reset, errno 10054
Unable to fetch in submodule path 'components/esp_wifi/lib'; trying to directly fetch 188461a1825a02367557c5b2bb1c6137fa07a0d5:
fatal: unable to access 'https://github.com/espressif/esp32-wifi-lib.git/': Failed to connect to github.com port 443 after 21005 ms: Timed out
fatal: Fetched in submodule path 'components/esp_wifi/lib', but it did not contain 188461a1825a02367557c5b2bb1c6137fa07a0d5. Direct fetching of that commit failed.








到此原本打算从4.2版本更新到4.2.2就彻底失败了。这个问题花了我两天时间。期间试过各种问题，网络有问题的也试过从gitee上克隆，但发现gitee上子模块已经很久没更新了。也试过直接从github上下载压缩包后解压替换。但最终都回到子模块更新失败上面。
在尝试的过程中发现idf.py build编译有时候会编译完成，但到了链接的时候就说少文件，提示的路径也在子模块里。

至此已完全没了头绪，遂来论坛找各位帮忙，希望有解决思路的能提供一些详细帮助。

[山竹君_kk]

在更新子模块的时候也碰到过这种错误：

Aborting
fatal: Unable to checkout '57d751b93a5532e650ec3787d376876ab0ce367e' in submodule path 'components/bt/host/nimble/nimble'
fatal: Unable to checkout 'd2dd95cb8841d88d5a801e3ef9c328fd6200e7bd' in submodule path 'components/cbor/tinycbor'
fatal: Unable to checkout '188461a1825a02367557c5b2bb1c6137fa07a0d5' in submodule path 'components/esp_wifi/lib'
fatal: Unable to checkout '3c8935676a97c7c97bf006db8312875b4f292f6c' in submodule path 'components/json/cJSON'
fatal: Unable to checkout 'da850b0add1e71b3659bfac5d797cc834dc3e89b' in submodule path 'components/mqtt/esp-mqtt'
fatal: Unable to checkout '3bcc416e13cc790e2fb45fcfe9111d38609c5032' in submodule path 'components/nghttp/nghttp2'
fatal: Unable to checkout '28f89e13473d40637574bcbfe4142633b39899fd' in submodule path 'components/tinyusb/tinyusb'

[山竹君_kk]

上面说的idf.py build出错的那个错误解决了，是什么受信任证书的锅，在menuconfig中去掉就好了。但出现了新的错误


D:/ESP32/esp-idf-v4.2.2/components/cbor/tinycbor/src/open_memstream.c:45:4: error: #error "Cannot implement open_memstream!"
# error "Cannot implement open_memstream!"
^~~~~
D:/ESP32/esp-idf-v4.2.2/components/cbor/tinycbor/src/open_memstream.c:57:8: error: unknown type name 'RetType'
static RetType write_to_buffer(void *cookie, const char *data, LenType len)
^~~~~~~
D:/ESP32/esp-idf-v4.2.2/components/cbor/tinycbor/src/open_memstream.c:57:64: error: unknown type name 'LenType'; did you mean 'CborType'?
static RetType write_to_buffer(void *cookie, const char *data, LenType len)
^~~~~~~
CborType
d:\esp32\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\xtensa-esp32-elf\sys-include\stdio.h: In function 'open_memstream':
D:/ESP32/esp-idf-v4.2.2/components/cbor/tinycbor/src/open_memstream.c:113:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
At top level:
D:/ESP32/esp-idf-v4.2.2/components/cbor/tinycbor/src/open_memstream.c:82:12: warning: 'close_buffer' defined but not used [-Wunused-function]
static int close_buffer(void *cookie)
^~~~~~~~~~~~
cc1.exe: some warnings being treated as errors
[812/1171] Building C object esp-idf/coap/CMakeFiles/__idf_coap.dir/libcoap/src/net.c.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

[山竹君_kk]

其实原因归根到底是我离线安装时idf是安装完成了但是子模块没有拉取成功。
解决方法就是去

• https://github.com/espressif/esp-idf/releases?page=1

里面找到对应版本，然后在Download an archive with submodules included下面点那个链接下载已经配置完所有子模块的压缩包，这个下载就很快。网上那些杂七杂八的对github上下载优化的对esp-idf拉子模块都是扯淡。直接下载官方配置好的压缩包解压覆盖他不香吗？