[IDF-4348] ESP-IDF Fails to Install

[Xylopyrographer]

Trying to install the ESP-IDF on macOS Big Sur Version 11.6.1 (20G224)

Following the instructions at: https://docs.espressif.com/projects/esp ... index.html

At Step 4, seems to be a path error that is not resolved by the export.sh script.

The install.sh script runs fine, but export.sh can't find 9 packages that are installed by install.sh

Thoughts on a magic incantation to fix?

Terminal output:

Code: Select all

🌮  ~/esp/esp-idf % ./install.sh esp32
Detecting the Python interpreter
Checking "python" ...
Checking "python3" ...
Python 3.9.8
"python3" has been detected
Installing ESP-IDF tools
Installing tools: xtensa-esp32-elf, xtensa-esp32s2-elf, xtensa-esp32s3-elf, riscv32-esp-elf, esp32ulp-elf, esp32s2ulp-elf, openocd-esp32
Skipping xtensa-esp32-elf@esp-2021r1-8.4.0 (already installed)
Skipping xtensa-esp32s2-elf@esp-2021r1-8.4.0 (already installed)
Skipping xtensa-esp32s3-elf@esp-2021r1-8.4.0 (already installed)
Skipping riscv32-esp-elf@esp-2021r1-8.4.0 (already installed)
Skipping esp32ulp-elf@2.28.51-esp-20191205 (already installed)
Skipping esp32s2ulp-elf@2.28.51-esp-20191205 (already installed)
Skipping openocd-esp32@v0.10.0-esp32-20210401 (already installed)
Installing Python environment and packages
Python 3.9.8
pip 21.3.1 from /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages/pip (python 3.9)
Installing Python packages from /Users/xylo/esp/esp-idf/requirements.txt
Looking in indexes: https://pypi.org/simple, https://dl.espressif.com/pypi
Ignoring None: markers 'sys_platform == "win32"' don't match your environment
Requirement already satisfied: setuptools>=21 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 4)) (58.3.0)
Requirement already satisfied: click>=7.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 8)) (8.0.3)
Requirement already satisfied: pyserial>=3.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 9)) (3.5)
Requirement already satisfied: future>=0.15.2 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 10)) (0.18.2)
Requirement already satisfied: cryptography>=2.1.4 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 11)) (35.0.0)
Requirement already satisfied: pyparsing<2.4.0,>=2.0.3 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 12)) (2.3.1)
Requirement already satisfied: pyelftools>=0.22 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 13)) (0.27)
Requirement already satisfied: gdbgui==0.13.2.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (0.13.2.0)
Requirement already satisfied: pygdbmi<=0.9.0.2 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 18)) (0.9.0.2)
Requirement already satisfied: python-socketio<5 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 21)) (4.6.1)
Requirement already satisfied: kconfiglib==13.7.1 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 23)) (13.7.1)
Requirement already satisfied: reedsolo<=1.5.4,>=1.5.3 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 26)) (1.5.4)
Requirement already satisfied: bitstring>=3.1.6 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 27)) (3.1.9)
Requirement already satisfied: ecdsa>=0.16.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 28)) (0.17.0)
Requirement already satisfied: construct==2.10.54 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from -r /Users/xylo/esp/esp-idf/requirements.txt (line 32)) (2.10.54)
Requirement already satisfied: Flask-SocketIO<3.0,>=2.9 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (2.9.6)
Requirement already satisfied: Pygments<3.0,>=2.2.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (2.10.0)
Requirement already satisfied: Flask<1.0,>=0.12.2 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (0.12.5)
Requirement already satisfied: Flask-Compress<2.0,>=1.4.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (1.10.1)
Requirement already satisfied: gevent<2.0,>=1.2.2 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (1.5.0)
Requirement already satisfied: cffi>=1.12 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from cryptography>=2.1.4->-r /Users/xylo/esp/esp-idf/requirements.txt (line 11)) (1.15.0)
Requirement already satisfied: six>=1.9.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from python-socketio<5->-r /Users/xylo/esp/esp-idf/requirements.txt (line 21)) (1.16.0)
Requirement already satisfied: python-engineio<4,>=3.13.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from python-socketio<5->-r /Users/xylo/esp/esp-idf/requirements.txt (line 21)) (3.14.2)
Requirement already satisfied: pycparser in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=2.1.4->-r /Users/xylo/esp/esp-idf/requirements.txt (line 11)) (2.21)
Requirement already satisfied: itsdangerous>=0.21 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from Flask<1.0,>=0.12.2->gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (2.0.1)
Requirement already satisfied: Werkzeug<1.0,>=0.7 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from Flask<1.0,>=0.12.2->gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (0.16.1)
Requirement already satisfied: Jinja2>=2.4 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from Flask<1.0,>=0.12.2->gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (3.0.3)
Requirement already satisfied: brotli in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from Flask-Compress<2.0,>=1.4.0->gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (1.0.9)
Requirement already satisfied: greenlet>=0.4.14 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from gevent<2.0,>=1.2.2->gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (1.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages (from Jinja2>=2.4->Flask<1.0,>=0.12.2->gdbgui==0.13.2.0->-r /Users/xylo/esp/esp-idf/requirements.txt (line 15)) (2.0.1)
All done! You can now run:

  . ./export.sh

======================================
🌮  ~/esp/esp-idf % . ./export.sh                 
Setting IDF_PATH to '/Users/xylo/esp/esp-idf'
Detecting the Python interpreter
Checking "python" ...
Checking "python3" ...
Python 3.9.8
"python3" has been detected
Adding ESP-IDF tools to PATH...
Using Python interpreter in python: aliased to /usr/local/bin/python3
Checking if Python packages are up to date...
The following Python requirements are not satisfied:
click>=7.0
future>=0.15.2
pyparsing>=2.0.3,<2.4.0
pyelftools>=0.22
gdbgui==0.13.2.0
pygdbmi<=0.9.0.2
python-socketio<5
kconfiglib==13.7.1
construct==2.10.54
To install the missing packages, please run "/Users/xylo/esp/esp-idf/install.sh"
Diagnostic information:
    IDF_PYTHON_ENV_PATH: /Users/xylo/.espressif/python_env/idf4.3_py3.9_env
    Python interpreter used: /usr/local/opt/python@3.9/bin/python3.9
    Warning: python interpreter not running from IDF_PYTHON_ENV_PATH
    PATH: /Users/xylo/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin:/Users/xylo/.espressif/tools/xtensa-esp32s2-elf/esp-2021r1-8.4.0/xtensa-esp32s2-elf/bin:/Users/xylo/.espressif/tools/xtensa-esp32s3-elf/esp-2021r1-8.4.0/xtensa-esp32s3-elf/bin:/Users/xylo/.espressif/tools/riscv32-esp-elf/esp-2021r1-8.4.0/riscv32-esp-elf/bin:/Users/xylo/.espressif/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/Users/xylo/.espressif/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/Users/xylo/.espressif/tools/openocd-esp32/v0.10.0-esp32-20210401/openocd-esp32/bin:/Users/xylo/.espressif/python_env/idf4.3_py3.9_env/bin:/Users/xylo/esp/esp-idf/tools:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/lib/node_modules:/Applications/Little Snitch.app/Contents/Components:/Library/Apple/usr/bin
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'click'
🌮  ~/esp/esp-idf %

[ESP_Roland]

Hi Xylopyrographer,

please try to unset your "python" alias in the terminal before running install & export.

The script needs to set "python" and it seems that is not able because of that.

[Xylopyrographer]

Many thanks Roland. The reply is appreciated.

unset python didn't work, had to unalias python to make it go.

As background, before installing ESP-IDF, python3 was installed using Homebrew -- a fairly common method. Homebrew sets a few alias' as part of that install.

Wondering -- since the export.sh script detects the python it needs isn't the one being used -- if that script could be modified to use it's IDF required python instance in the ~/.espressif directory. I mention that as there are a few applications that install local versions of python and that do so without conflict with the Homebrew installed python. Or if the export.sh script could prompt the user to, temporarily, re-path as necessary to complete its work.

If it matters, all this is being done using zsh, the now default on macOS.

Food for thought.

[ESP_Roland]

Thanks. I'm aware of the fact that this has to be fixed. I just gave you a workaround so you don't have to wait for the proper fix.

[Xylopyrographer]

Appreciate the help. Thanks.

Running through the "Get Started" steps at:
https://docs.espressif.com/projects/esp ... -variables

At Step 8 Build the Project, after running:

Code: Select all

 ~/esp/hello_world % idf.py build

things proceeded until the following error...

Code: Select all

[86/86] Generating binary image from built executable
esptool.py v3.1-dev
Merged 1 ELF section
Generated /Users/xylo/esp/hello_world/build/bootloader/bootloader.bin
[663/966] Generating x509_crt_bundle
FAILED: esp-idf/mbedtls/x509_crt_bundle /Users/xylo/esp/hello_world/build/esp-idf/mbedtls/x509_crt_bundle 
cd /Users/xylo/esp/hello_world/build/esp-idf/mbedtls && /Users/xylo/.espressif/python_env/idf4.3_py3.9_env/bin/python /Users/xylo/esp/esp-idf/components/mbedtls/esp_crt_bundle/gen_crt_bundle.py --input /Users/xylo/esp/esp-idf/components/mbedtls/esp_crt_bundle/cacrt_all.pem -q
gen_crt_bundle.py: Invalid certificate in /Users/xylo/esp/esp-idf/components/mbedtls/esp_crt_bundle/cacrt_all.pem
Invalid certificate
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

How can I update the required certificate(s)?