Need help updating from a v4.1-dirty release to the current clean version of 4.1

[dritter]

Hello,

I'm using the standard ESP32 programming environment in Windows and I'm trying to get rid of the "v4.1-dirty" version. I just want a standard and clean "v4.1" install.

I've repeatedly installed new versions of esp-idf on my computer and they keep coming up as dirty.

So I've also repeatedly tried to use git to update esp-idf to the current release version. But it just won't do it. Instead, I keep getting errors like this:

"Please commit your changes or stash them before you switch branches."

(and other similar things). But I don't want to commit anything. Nor do I want to switch branches. I have changed nothing and this is a totally fresh install directly from the esp32 api download page. All I want is to get rid of the "-dirty" version.

And I do understand that changes may have occurred on the git release that are not in the .zip I downloaded and installed my local version from. That is fine and so now I just want to apply the latest changes so that my local copy of the release will go from "-dirty" to clean.

So, how do I force the issue here and just make the version on my computer match the current release?

I have tried numerous things to get past this. But every single time I try to do anything in git, it basically fails and refuses to updated. For example, a commit will fail. So will the "update submodules" command. Checkout to get the latest version does not seem to work either.

In fact, every time I try to do anything at all in git, it just fails. Or it pretends to work but nothing changes and my version still says dirty. And it keeps showing that now ultra-annoying error that I still have uncommitted changes. (These are not even my changes and i just want to overwrite them, but anyway...)

So, as it stands right now, my computer never gets updated to the current version and I seem to be stuck with a dirty version no matter what I do.

pls help...
and
Why is this even happening?

[dastoned]

Did you actually do a git clone? In such case:

Code: Select all

git reset --hard
git submodule update --init
git submodule foreach git reset --hard

Proceeded by checking out whatever version you want, presumably the branch "release/v4.1"

Code: Select all

git checkout -b release/v4.1 origin/release/v4.1

If you downloaded a packaged snapshot ("Downlaod ZIP" download option in github), then this is not a git repository and there is nothing for git to work with.

[dritter]

Thanks for the tips.

I did all the steps and git did update my local esp-idf copy. But now the esp version reports as:

v4.1-rc-15-gbd72a9ab2-dirty

So it's still considered dirty for some reason. I don't know why for sure this is, but it seems to be due to some original ESP files still being marked as changed.

I ran "git status" and it came back with this:

Code: Select all

On branch release/v4.1
Your branch is up to date with 'origin/release/v4.1'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)

        modified:   components/app_update/otatool.py
        modified:   components/asio/asio (modified content)
        modified:   components/bootloader/subproject/components/micro-ecc/micro-ecc (modified content)
        modified:   components/bt/controller/lib (modified content)
        modified:   components/bt/host/nimble/nimble (modified content)
        modified:   components/cbor/tinycbor (modified content)
        modified:   components/coap/libcoap (modified content)
        modified:   components/efuse/efuse_table_gen.py
        modified:   components/efuse/test_efuse_host/efuse_tests.py
        modified:   components/esp_wifi/lib (modified content)
        modified:   components/esp_wifi/test_md5/test_md5.sh
        modified:   components/espcoredump/espcoredump.py
        modified:   components/espcoredump/test/test_espcoredump.py
        modified:   components/espcoredump/test/test_espcoredump.sh
        modified:   components/esptool_py/esptool (modified content)
        modified:   components/expat/expat (modified content)
        modified:   components/heap/test_multi_heap_host/test_all_configs.sh
        modified:   components/json/cJSON (modified content)
        modified:   components/libsodium/libsodium (modified content)
        modified:   components/lwip/lwip (modified content)
        modified:   components/mbedtls/mbedtls (modified content)
        modified:   components/mqtt/esp-mqtt (modified content)
        modified:   components/nghttp/nghttp2 (modified content)
        modified:   components/nvs_flash/nvs_partition_generator/nvs_partition_gen.py
        modified:   components/partition_table/gen_esp32part.py
        modified:   components/partition_table/parttool.py
        modified:   components/partition_table/test_gen_esp32part_host/gen_esp32part_tests.py
        modified:   components/protobuf-c/protobuf-c (modified content)
        modified:   components/spiffs/spiffsgen.py
        modified:   components/ulp/esp32ulp_mapgen.py
        modified:   components/unity/unity (modified content)
        modified:   docs/check_doc_warnings.sh
        modified:   docs/check_lang_folder_sync.sh
        modified:   docs/gen-dxd.py
        modified:   docs/gen-version-specific-includes.py
        modified:   examples/build_system/cmake/idf_as_lib/build-esp32.sh
        modified:   examples/build_system/cmake/idf_as_lib/build.sh
        modified:   examples/build_system/cmake/idf_as_lib/run-esp32.sh
        modified:   examples/build_system/cmake/idf_as_lib/run.sh
        modified:   examples/build_system/cmake/import_lib/main/lib/tinyxml2 (modified content)
        modified:   examples/storage/parttool/parttool_example.py
        modified:   examples/system/ota/otatool/otatool_example.py
        modified:   install.sh
        modified:   tools/build_apps.py
        modified:   tools/check_kconfigs.py
        modified:   tools/check_python_dependencies.py
        modified:   tools/ci/apply_bot_filter.py
        modified:   tools/ci/build_examples.sh
        modified:   tools/ci/build_examples_cmake.sh
        modified:   tools/ci/check-executable.sh
        modified:   tools/ci/check-line-endings.sh
        modified:   tools/ci/check_build_warnings.py
        modified:   tools/ci/check_deprecated_kconfigs.py
        modified:   tools/ci/check_examples_cmake_make.sh
        modified:   tools/ci/check_examples_rom_header.sh
        modified:   tools/ci/check_idf_version.sh
        modified:   tools/ci/check_ut_cmake_make.sh
        modified:   tools/ci/checkout_project_ref.py
        modified:   tools/ci/envsubst.py
        modified:   tools/ci/fix_empty_prototypes.sh
        modified:   tools/ci/get-full-sources.sh
        modified:   tools/ci/get_supported_examples.sh
        modified:   tools/ci/mirror-submodule-update.sh
        modified:   tools/ci/multirun_with_pyenv.sh
        modified:   tools/ci/push_to_github.sh
        modified:   tools/ci/test_build_system.sh
        modified:   tools/ci/test_build_system_cmake.sh
        modified:   tools/ci/test_configure_ci_environment.sh
        modified:   tools/cmake/convert_to_cmake.py
        modified:   tools/cmake/run_cmake_lint.sh
        modified:   tools/docker/entrypoint.sh
        modified:   tools/docker/hooks/build
        modified:   tools/elf_to_ld.sh
        modified:   tools/esp_app_trace/logtrace_proc.py
        modified:   tools/esp_app_trace/sysviewtrace_proc.py
        modified:   tools/esp_app_trace/test/logtrace/test.sh
        modified:   tools/esp_app_trace/test/sysview/compare_json.py
        modified:   tools/esp_app_trace/test/sysview/test.sh
        modified:   tools/find_apps.py
        modified:   tools/format.sh
        modified:   tools/gen_esp_err_to_name.py
        modified:   tools/idf.py
        modified:   tools/idf_monitor.py
        modified:   tools/idf_size.py
        modified:   tools/idf_tools.py
        modified:   tools/kconfig/check.sh
        modified:   tools/kconfig/lxdialog/check-lxdialog.sh
        modified:   tools/kconfig/merge_config.sh
        modified:   tools/kconfig/streamline_config.pl
        modified:   tools/kconfig_new/confgen.py
        modified:   tools/kconfig_new/confserver.py
        modified:   tools/kconfig_new/test/test_confserver.py
        modified:   tools/ldgen/ldgen.py
        modified:   tools/ldgen/test/test_fragments.py
        modified:   tools/ldgen/test/test_generation.py
        modified:   tools/mass_mfg/mfg_gen.py
        modified:   tools/set-submodules-to-github.sh
        modified:   tools/test_check_kconfigs.py
        modified:   tools/test_idf_monitor/run_test_idf_monitor.py
        modified:   tools/test_idf_py/test_idf_py.py
        modified:   tools/test_idf_size/test.sh
        modified:   tools/test_idf_tools/test_idf_tools.py
        modified:   tools/unit-test-app/tools/get_available_configs.sh
        modified:   tools/unit-test-app/unit_test.py
        modified:   tools/windows/eclipse_make.sh
        modified:   tools/windows/tool_setup/build_installer.sh
        modified:   tools/windows/tool_setup/sign_installer.sh

no changes added to commit (use "git add" and/or "git commit -a")

This is exactly what was I was seeing before the update. Note that I did not change these files either. They were installed automatically during the windows platform installation process.

And these apparently changed files seem to persistently stay changed, even when I try things like updating to esp-idf v4.2 for example, or doing hard resets that are supposed to make git ignore changes and overwrite the files.

Whatever is going on in git, It is my guess is that these persistent changes are the reason that ESP reports my executable builds as being dirty.

It also occurred to me that these files might only be used locally and are not even part of the clean distribution. On that idea, I did once try to use git to add them and do a commit. But that failed too.

So I'm still stuck in dirt...

[dastoned]

As the output from git suggests, discarding changes to a file is done with command "git checkout -- <target>". To discard everything in current directory, do "git checkout -- ."

Unless you have made changes to ESP IDF source code which you want to keep, perhaps it would be easiest to delete the git respository and clone a fresh one?

[chegewara]

If its on windows its most likely issue with EOL, which is different on linux/unix and on windows.

[dritter]

dastoned: That makes sense. If something is changing these files during installation, or they are not getting overwritten during the upgrades for some reason, it's probably best to just delete the entire thing and fully replace it. I did not try that specific step yet so it's worth a try.

chegewara: if I understand correctly, you are are saying it might be because the CR-LF end-of-line marker in windows is a different character sequence than the single LF character end-of-line marker is in linux?

That does sort of make sense to me but I don't see an obvious way of fixing it. Is there a cleanup script or maybe some way to tell git to ignore those differences?

[dastoned]

Yeah, might be the line endings. This comes into play when you open a file in some zombie editor which "helpfully" converts Unix line endings into Windows ones and then saves the changed files. Some editors also clean up trailing whitespace etc. Any change you or your editor make is flagged by git as - gasp - a change . If you wish to dig into the details, "git diff" tells the truth.

If line endings are the case, you you two options: either use a normal editor (all the nice modern IDEs and editors - VS Code, Sublime, Notepad++... should leave Unix line endings as they are) or configure git to convert the line endings on checkin/checkout: https://docs.github.com/en/free-pro-tea ... ne-endings (I can't really recommend this option over using a normal editor).

[dritter]

Indeed, it does look kind of messy and it may not be something I can easily and instantly fix. On the other hand, the project code all runs fine even with these diff's, and so other than the "-dirty" suffix showing up in the reported version, it does not actually affect my project.

If I find an easy and quick solution, I'll post it here. Thanks for the help, at least now I know it's not a major thing.

[chegewara]

Another possibly reason is you didnt update submodules.

[dritter]

Yes, that could be occurring as well. I did run the submodule update commands as mentioned above, and git did respond by apparently downloading a number of files. But I didn't log the activity so I cannot now say for sure what exactly happened.

Either way, I did just run "git diff" and it shows that there are still a large number of diff's between what's on my hard disk and what's in the public git repository. And the files seem to match the ones listed above in this thread.

And since I didn't change any of those files myself, it seems most likely that git is not happy with the lines ending that get written to my disk when these files are copied over from the repository. Which to me means that the ideal fix is to fix the original files on the repository so they come across clean for windows users. I don't know if doing that is practical or not, since it may affect things on the linux side.

For now, there are more pressing issues on my project anyway so I probably won't be able to look at this again for a while. Mainly, I was just wanting to remove the "-dirty" text in the version string. Plus, there was also the possibility that source code diff's might affect the project I am working on. But if it's just line endings, that nothing serious so I can just live with it for now.

Also, it sounds like I can build a Linux image and compile a clean production version off that if I have to when the time comes. That's maybe not the best solution but it is a viable plan B if I can't get the code to appear clean on the Windows computer.