Hi Greg,
This should work, we do it in a number of IDF's internal components (take a look at "components/lwip" or "components/libsodium" for some directory structures that go pretty deep.)
.d is a generated dependency file. If you've been moving files/directories around, then you may need to do a "make clean" before things build nicely. This is a limitation of the build system that's probably not going away soon, unfortunately (the good news, it's about the only time you should need a "make clean")
If "make clean" doesn't work, try renaming the build directory out of the way and then re-run make. If that works, it's a bug in the build system (please let us know if that's the case.)
If none of this works, can you please give some more details about exactly what the contents of your component.mk is and exactly what files are in which directories?
PS The "components" concept is designed to make this kind of organisation straightforward, as well. You can keep all your files in subdirectories under "main" and this will work fine, but you can also make a structure where your project has its own components. Something like the structure shown here:
http://esp-idf.readthedocs.io/en/latest ... le-project
(Whether you choose to do this is up to you, entirely.)