If you are using design with optocoupled connection to relay module, grounding of all power sources is not a good option. The goal of opto-coupling is exactly that, to achieve two separate circuits and protection of each of other. So no.1.I have not grounded 12V (DC adapter) and from 5 V (Buck converter). Do I need to ground both of them ?
Well, optocoupled design should not produce any kind of notable heating on ESP32. Do you use WIFI module? Are not used pads of esp32 free? (not connected) What max current your 5V buck converter can provide, is it enough for both ESP32 and relay(s)? What max current your 12V source can provide? Is it enough for additional buck, both relay(s), ESP32 and your 12V valve? Consider also an inverted logic on ESP32 pad if your relay has third pin at output which close circuits when relay is not "energized". In general, your valve can be in one of two states, ON or OFF. If it is known that it is more often in let's say ON state, then use relay to produce that state when it is NOT energized (using that third output) and vice verse. Sure, you have to modify your source code to invert the states of pads.2. I have put everything into a box. May be there is no enough ventilation and it get overheated . Probably I shall change the enclosure if there are no problems with the circuit.
That depends on does ESP32 run stable or not.3.Do I need to add any capacitor after the 5V ?
For example, if your ESP32 has a mysterious resets when relay turns ON, it could be caused by brownout protection, suggesting any of your voltage sources can not provide enough current for both ESP32 and relay when relay starts sucking current. In that case capacitor can help only if turn on state of your relay is short. However, if your relay has to be in ON state for a long period, capacitor wont help but you have to provide adapters capable to deliver higher current.
There also could be possible mysterious resets at any relay(s) transition ON or OFF, suggesting the current from self inductance of solenoid in relay affects the rest of circuit. (in general, even sparking on live contactors on relay, can produce large enough EMF to induce some current in solenoid .... but that is not your case with 12V valve) . That's why opto-coupling is usually used, to prevent that, but it has sense ONLY if both power lines, Vcc and GND are separated for both sides of optocoupled circuit. In your case both +5V and GND from that 5V source are common to both powering the esp32 part of circuit and powering relays circuit, so reverse currents from relay module can affect ESP32 through it even in design with optocouplers, but not through controlling pad, but through power lines. Mostly of that I suggested a simpler design by mosfet(s) because of single power source, but anyway, if you stick to design with optocoupled relay module, the only proper way to "isolate" both parts of circuits not affecting each other is to use separate power lines. In your case it would be two 12->5V bucks. One for powering ESP board and another to powering relay(s) module. That second +5V would then be connected to that JD-VCC and jumper removed. The first buck 5V would be connected to USB +5V on your dev board, and 3.3V from your dev board to that VCC from upper schematic. If U use just a single power source and you leave that jumper, it is a bad idea. That way you actually connected VCC 3.3V of ESP32 with your 5V power source and that possibly can cause the fry. ( It is true, on upper schematic, that there would be a voltage drop on two diodes in series approx -1.4V so even in case of 5V input to optocoupler led, ESP pad would "sense" no more than 3.6V however you mentioned that your relay module board is similar to that which not necessary means both designs are identical. Also it is not clear from your schematic how those VCC lines are connected to your dev board).
You can always make a simple test. There are so many cheap those battery power banks for cell phones. And most of dev boards have USB connector. So you can use such a power bank to power your dev board with ESP32 only. It will produce 5V to USB side of your board. However, due to 5->3.3V conversion on board, VCC on ESP32 will be 3.3V. That VCC 3.3V you have to provide to VCC at relay board and DISCONNECT that jumper to separate JD-VCC 5V from esp32's VCC 3.3V. Now you can connect your 5V buck output to that JD-VCC. It would be nice if the same can be possible with GND but according to that schematic both are connected which narrows range of designs where it is applicable.
