ESP-IDF used as Submodule like ESP-ADF

[mikemoy]

In another thread @chegewara had what I thought was a great idea. But I am not sure how to implement it.
He said:

Lets look at esp-adf project for example. Esp-idf is used as submodule with particular commit:
https://github.com/espressif/esp-adf

I believe you can build any project you want the same way and instead of having global esp-idf, you can have esp-idf per project.

I like this idea. The reason being that looking in the future I can see a potential problem arising where projects that are built today and released using say v3.3.1. But later ESP-IDF will be on another version say v4.x.x. Should my customer want to make changes a year later I need to build that project built on v3.3.1 but my current setup is on v4.x.x which wont work.

Did all that makes sense How can I copy v3.3.1 into the each project so I know its all self contained much like what ESP-ADF does

[chegewara]

1) create project and git init
2) ad esp-idf as submodule https://stackoverflow.com/questions/420 ... repository
3) go to esp-idf and git commit to version you want
4) git add . & git push

https://github.com/chegewara/commit-test

Remember to configure CMakeLists.txt to use esp-idf from subfolder.

[mikemoy]

Thank you for the example. But this did not go smooth sailing for me on the PC.
My default Windows IDF_PATH is C:\Users\Mike\ESP32\esp-idf

Its not clear to me in your example how its pointing to the new submodule.
The CMakeLists.txt file contains:

Code: Select all

# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(hello-world)

The way i read this is that it would be still using the defaul path and not pointing to the included ESP-IDF folder.
Am i just not seeing the light here or what ?

[mrrosen]

For one of my projects, I did this exact thing however, this was right during the shift to cmake so I did it in the old Makefile system; like so (where I created the esp-idf as a submodule in the idf directory):

Code: Select all

export IDF_PATH = $(PWD)/idf

PROJECT_NAME := my_project

include $(IDF_PATH)/make/project.mk

Pretty sure you could do that same thing with cmake by setting the IDF_PATH environment variable at the start of the file:

Code: Select all

cmake_minimum_required(VERSION 3.5)

set(ENV{IDF_PATH} ${CMAKE_CURRENT_LIST_DIR}/idf)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(my_project)

[chegewara]

https://github.com/espressif/esp-adf/bl ... .txt#L6-L8

I didnt try it yet, but i believe you need to set IDF_PATH without if, just this line:

Code: Select all

set(ENV{IDF_PATH} "${PROJECT_PATH}/esp-idf/")
set(EXTRA_COMPONENT_DIRS ${PROJECT_PATH}/components)

[mikemoy]

Thanks guys for the help. Didn't work for me. I got much closer with

Code: Select all

set(ENV{IDF_PATH} "${PROJECT_PATH}/esp-idf/")
set(EXTRA_COMPONENT_DIRS ${PROJECT_PATH}/components)

At the moment I just don't have the time to dig into this.

[chegewara]

Ok, ive been overthinking it, but it is very simple. I am using PIO and from PIO terminal i had to do this:

Code: Select all

. ./esp-idf/export.sh
idf.py build

It was with esp-idf v3.3 as submodule described earlier.

PS no changes needed in CMakeLists.txt i mentioned earlier, just forget it

[phatpaul]

I had been doing this since IDF 3 with make. I had a custom build script.
When I ported to IDF 4 with CMake, I created a new init scripts to set the environment.
(My CMakeLists.txt doesn't need anything special for this b/c my init script sets IDF_PATH in the environment. It still uses include("$ENV{IDF_PATH}/tools/cmake/project.cmake"))

I made a windows version and a linux of the script. It expects the esp-idf to be a git submodule in the project repo/tools/esp-idf/.
You can just double-click the _START_IDF_ENV.cmd file to start a cmd window.
On linux you have to type: source sourceme_IDF_ENV.sh

Code: [Select all] [Expand/Collapse]

1. @REM Script for launching ESP-IDF environment on Windows
2. @REM Author: Paul Abbott, 2022
3. @ECHO OFF
4. TITLE ESP-IDF
5. SETLOCAL EnableDelayedExpansion
6.  
7. : %CD% gives current dir
8. SET THISDIR=%CD%
9. : %~dp0 gives the directory of this batch file (with a trailing \)
10. SET SCRIPTDIR=%~dp0
11.  
12. :: if using IDF installed globally ::
13. : SET "IDF_INIT_SCRIPT=C:\Espressif\idf_cmd_init.bat"
14. :: See C:\Espressif\esp_idf.json for installed IDF version strings
15. :: Version 4.4
16. : SET "IDF_VERS_STRING=esp-idf-8de2bd0d9cffd2eca3d3f8442939a034"
17. :: Version 4.3
18. : SET "IDF_VERS_STRING=esp-idf-7a3cc3545977e8fdecfa73521e1f3180"
19.  
20. :: if using IDF inside repo as submodule::
21. SET "IDF_INIT_SCRIPT=%SCRIPTDIR%..\tools\esp-idf\install.bat"
22. SET "IDF_INIT_SCRIPT2=%SCRIPTDIR%..\tools\esp-idf\export.bat"
23.  
24. IF NOT EXIST %IDF_INIT_SCRIPT% (
25.     MORE %SCRIPTDIR%readme.md
26.     ECHO Script %IDF_INIT_SCRIPT% does not exist!.
27.     GOTO :DIE
28. )
29. CALL %IDF_INIT_SCRIPT% %IDF_VERS_STRING%
30. IF !ERRORLEVEL! NEQ 0 goto :DIE
31. IF NOT EXIST %IDF_INIT_SCRIPT% GOTO :DONE
32.  
33. CALL %IDF_INIT_SCRIPT2%
34. IF !ERRORLEVEL! NEQ 0 goto :DIE
35.  
36. :DONE
37. GOTO :SHELL
38.  
39. :DIE
40. : Color the screen red
41. COLOR 4f
42. ECHO ERROR!
43. PAUSE
44. GOTO :QUIT
45.  
46. :QUIT
47. : Restore the Color of the screen
48. COLOR 07
49. GOTO :eof
50.  
51. :SHELL
52. : Restore the Color of the screen
53. COLOR 07
54. ECHO/
55. ECHO   idf build flash monitor -p COM15
56. ECHO/
57. cmd /k

GeSHi © Codebox Plus Extension

Code: [Select all] [Expand/Collapse]

1. #!/bin/bash
2.  
3. # Script for setting up ESP_IDF environment on Ubuntu
4. # use it like this: `source sourceme_IDF_ENV.sh`
5. # Author: Paul Abbott, 2022
6. SCRIPT_VER="1.0"
7. SCRIPT_DIR=$(realpath "$(dirname "${BASH_SOURCE[0]}")")
8. PWD=$(pwd)
9.  
10. # this only works when sourced (not run as a script), because it is adding variables into the local shell
11. [[ "${BASH_SOURCE[0]}" == "${0}" ]] && echo "Error, must source this script!  Try: \"source ${BASH_SOURCE[0]}\"" && exit 1
12.  
13. source $SCRIPT_DIR/../tools/esp-idf/export.sh
14. alias idf="idf.py"
15.  
16. # These commands will not execute, but will be in the history for convenience.
17. history -s idf build
18. history -s idf build flash monitor -p /dev/ttyUSB0
19.  

GeSHi © Codebox Plus Extension

[phatpaul]

The scripts I posted above work OK, as long as you have the correct version of esp-idf checked-out as git submodule working copy. But I find it is common that a dev forgot to do git submodule update --init --recursive after pulling other changes that expect a different esp-idf.

So I added another script that checks that all submodules are at their expected versions. This one is called by CMake.
(I adapted this code from https://github.com/iree-org/iree/pull/6568)

Code: [Select all] [Expand/Collapse]

1. cmake_minimum_required(VERSION 3.5)
2.  
3. #-------------------------------------------------------------------------------
4. # Check if git submodules have been initialized.
5. # This will only run if python3 is available.
6. #-------------------------------------------------------------------------------
7. find_package(Python3 COMPONENTS Interpreter QUIET)
8. if(Python3_FOUND)
9.   message("check_submodules")
10.   execute_process(
11.     COMMAND ${Python3_EXECUTABLE} ../tools/check_submodules.py
12.     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
13.     RESULT_VARIABLE ret
14.   )
15.   if(NOT ret EQUAL "0")
16.     message(FATAL_ERROR "check_submodule_init.py failed, see the logs above")
17.   endif()
18. endif()
19.  
20. include("$ENV{IDF_PATH}/tools/cmake/project.cmake")
21. project(poco-fw-ota)

GeSHi © Codebox Plus Extension

Code: [Select all] [Expand/Collapse]

1. # Copyright 2021 The IREE Authors
2. #
3. # Licensed under the Apache License v2.0 with LLVM Exceptions.
4. # See https://llvm.org/LICENSE.txt for license information.
5. # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6.  
7. import os
8. import subprocess
9. import sys
10.  
11.  
12. def run():
13.   # No-op if we're not in a git repository.
14.   try:
15.     subprocess.check_call(['git', 'rev-parse', '--is-inside-work-tree'],
16.                           stdout=subprocess.DEVNULL,
17.                           stderr=subprocess.DEVNULL)
18.   except:
19.     return
20.  
21.   output = os.popen("git submodule status --recursive")
22.   submodules = output.readlines()
23.  
24.   for submodule in submodules:
25.     if (submodule.strip()[0] == "-"):
26.       print(
27.           "Error: The git submodule '%s' is not initialized. Please run `git submodule update --init --recursive`"
28.           % (submodule.split()[1]))
29.       sys.exit(1)
30.     if (submodule.strip()[0] == "+"):
31.       print(
32.           "Warning: The git submodule '%s' is at a different version than expected. Please run `git submodule update --init --recursive`"
33.           % (submodule.split()[1]))
34.  
35. if __name__ == "__main__":
36.   run()

GeSHi © Codebox Plus Extension

BTW:
I'm curious if the new idf-component-manager is a better tool for this though. It looks like it can set the required IDF version in your manifest file.
https://docs.espressif.com/projects/esp ... nager.html