Control multiple small robots from a single pc, how to connect

[Reetus]

Hi all,

new here and not sure if this is the correct forum to ask this so feel free to guide me to a better place if needed.

Anyway, picture attached is somewhat what I would like to build, multiple small 3D-printed robots/cars that I could control from a single pc with a controller one at a time and then change the unit I'm controlling with a D-pad on the controller or choose the unit from a software that's running on the pc. Or have multiple "players" with controllers that could shuffle the free units and each control one at a time.

I know how to write the software for the pc and I know how to write the code for the robots moving and stuff.

The question:
What is unclear to me however is how could I handle the communication between the robot unit being controlled and the pc. I guess it should be some kind of an ssh connection but not an ssh? Some form of stream of inputs from the pc to the esp. Basically rc-control but over the wifi.

The connection should be somehow terminated when a user switches from controlled unit to another so that the pc software can see that a unit is "free" to use and the robot unit itself knows to halt all motion. And then the connection should be formed again when a user decides to control that unit.

I don't think an usecase like this needs to be very secure so any kind of a raw data stream should be fine.

Most of the tutorials I find online seem to all be about making single commands to turn lights etc on/off but how would I have a constant input with joysticks and buttons. So if you know any documentation about this kind of project please leave a link.

Bonus question:
Are there some limitations which boards would be suitable for this kind of application?

Thanks in advance for any tips or help you are able to provide

[MicroController]

Probably the easiest way would be to just send UDP packets over WiFi to each robot, and potentially also from robots to the PC.
You can build arbitrarily complex protocols using just UDP packets if you like. E.g. each robot could send a "hello" packet to the PC when it is turned on/joins the WiFi, so that the PC knows the IP address of every robot. If a robot loses network or doesn't receive a UDP packet for some time it stops for safety, &c.
UDP is usually prefered over TCP for near real-time communication because TCP's congestion control and packet retransmission can cause uncontrollable delays.

[Reetus]

Thanks for your suggestion, you got pretty good idea of what I was gonna do and this might have been just the what I was looking for.

You think pretty much any board would be capable for this kind of a project?

[MicroController]

You think pretty much any board would be capable for this kind of a project?

Yes. Any WiFi-capable controller would easily handle the networking, controlling a bunch of servos, and taking readings from a couple of sensors.
I recon that in addition a lot of "complex" autonomous behaviour would also fit into RAM, flash and CPU limitations.
A dual-core SoC is probably not necessary, because I believe that some (ISR) jitter of even 1ms or so wouldn't matter much in this case.