I have noticed the problem is about align from his code and then start scratch the dirty hard, and found if the alignment is correct it won't fire up InstrFetchProhibited error in general. And the rest of problems are from reloc and literal address fixes, somehow is much different with ELF header implementation.
I didn't ask for the license question yet, I'm currently okay with the dirtyhack way handling
One function is quite interesting here. Maybe some answers live in it with D/IRAM. In heap_caps.c there's a fcuntion called dram_alloc_to_iram_addr which seems like in charge of dram and iram management which is a static function didn't exposed out. I tried to use that to handle DIRAM, but it did return a pointer of address which is outside of DIRAM, maybe I need to dig more into the code to find how to handle this part lol.