Developing with ESP-IDF under WSL2

[ESP_michael]

# Background

The support for MSYS2 has been removed (since IDFv5.0?). Now the offically supported ways of developing are:

- Command prompt
- Powershell

But those Windows users used to to work on with MSYS2 (shell commands) may feel bad with the two approaches above. This article describe one approach to develop under WSL2.

Now WSL have two versions, WSL1 and WSL2.

WSL1: An adapter layer built above Windows-NT kernel, doesn't have its own file system. All operations are translated to Windows syscalls.

WSL2: Linux kernel directly built upon hypervisor (Hyper-V, hardware contorller). Have its own filesystem. Faster compilationon on Linux filesystem, but very slow IO to Windows filesystem.

See more difference between WSL1 and WSL2: https://docs.microsoft.com/en-us/window ... e-versions

One of the important considerations is the compiling time, so I'd like to try WSL2. However, among all the disadvantages of WSL2, the most significant one is the support of USB serial ports. This page is mainly to solve that problem, though also covers some other parts during WSL2 installation.

Note that, with the method in this page, you will be able to have access to the USB device on WSL2. But when the device is connected, you will have to manually run a command to redirect it to WSL2. This is inconvenient for someone whose USB connection is not permanent (may power off or using USB serial JTAG).

# Requirements

1. Good enough hardware. WSL2 usually cost (typical value for my laptop)
- about 500 MB to 3 GB RAM on my laptop.
- And also the CPU usage
- 10GB for each distro (virtual disk image). I guess this is resizable.

# Installation WSL2

Follow these gudies:
1. https://docs.microsoft.com/en-us/windows/wsl/install
2. https://docs.microsoft.com/en-us/window ... all-manual

## Trouble shooting

### Needs to enable both WSL and HW virtualization features in Windows. (See Step 1 and 3 in the guide 2 above.)

https://github.com/microsoft/WSL/issues/4766

If you see "Please enable the Virtual Machine Platform Windows feature and ensure virtualization is enabled in the BIOS." when starting or setting default version to WSL2, you may need to reboot your PC, or enable the feature in your BIOS. (https://jingyan.baidu.com/article/4b52d ... 74b5a.html)

# How to add support of USB to WSL2

See https://docs.microsoft.com/en-us/window ... onnect-usb

## Installation

This includes 3 steps:

- Install usbipd service and tools on your Windows system
- Install usbip on your Linux distribution
- Recompile and replace the WSL2 kernel with USB support (Windows 10 only)

For Windows 10, the default WSL kernel doesn't support USB, so we need to re-compile and replace the kernel: https://github.com/dorssel/usbipd-win/w ... l-2-kernel.

Basically this step is to:

1. Download the WSL2 kernel repo, checkout the tag of your installed Kernel version
2. Configure to enable USB support (I suggest to also enable converter drivers of CP2102 and CH340, when you are enabling `Device Drivers -> USB Serial Converter Support -> USB FTDI Single port Serial Driver`, not sure if this is necessary though)
3. Build the kernel
4. Do a few other stuff (I don't know if they are necessary... Are they integrated into the kernel we have just compiled?)
5. Copy the compiled kernel `arch/x86/boot/bzImage` to the Windows system, let WSL2 use it by setting up the `.wslconfig` file. (You may need to reboot your PC to make the settings take effect)

Notes: I think the new distro (exported and imported) in the process can be discarded and will not be used any more. At least after I unregister it from WSL, USB devices can still be used in WSL2.

## Steps to attach your USB device to the WSL2

Every time an USB device is connected to your PC, you need to do the following steps to make it available for WSL2. You need to share the device on Windows with usbipd first, and attach the shared device to WSL2 with usbip.

### On Windows

1. Make sure the usbipd server is running on Windows. You can do this by either:

1. Make sure the Windows service `USBIP Device Host` is still running (Right Click `This PC` -> Management -> Service). Usually the service that is always running, but sometimes it crashes and cannot recover quickly. I recommend to execute the server in Powershell command instead. (see below)

2. Run `usbipd server` in Powershell can also run the server, but the console needs to be kept open. I think this is more convenient than the Windows service: you can see if the server is running in command line, and disconnect the devices by terminating the server at any time (CTRL+C), or recover the server quickly.

2. Run `usbipd list` in Windows powershell to see all your USB devices. Remember the BUSID of the desired USB device.

Note: here you will see all the USB devices of this PC. Devices with `Shared` state will be available for WSL2 to attach. Devices with `Attach` state is already attached by WSL2, and cannot be attached by other users.

3. Share your USB device on Windows. Run `usbipd bind -b [busid]` to share the desired device.

Note: step 2 and 3 only needs to be performed once. After this step, the same device connected to the same physical port will be automatically binded (shared). If WSL2 cannot attach any device, check if the usbipd server is running on Windows (step 1).

### On WSL2

1. Get the ip of Windows system by running `ip route` in WSL2. You will see:
```
$ ip route
default via 172.22.80.1 dev eth0
172.22.80.0/20 dev eth0 proto kernel scope link src 172.22.88.162
```
The ip shown in the default line is the IP of the Windows. You can also get this IP easily by alias a command (`alias win_ip='ip route | grep default | awk '\''{print $3}'\'''`).

2. Check the binded (shared) devices on Windows by running `sudo usbip list -r [windows_ip]`. You will see:
```
$ sudo usbip list -r 172.22.80.1
Exportable USB devices
======================
- 172.22.80.1
1-1: Silicon Labs : CP210x UART Bridge (10c4:ea60)
: USB\VID_10C4&PID_EA60\88E1A435661CEC11AA7FA08DF01C6278
: (Defined at Interface level) (00/00/00)
: 0 - Vendor Specific Class / unknown subclass / unknown protocol (ff/00/00)
```
Here 1-1 is the busid of the device. I guess it will also keep the same value, for the same devices connected to the same physical port.

3. Attach the device to WSL2 by `sudo usbip attach -r [windows_ip] -b [bus_id]`. You can do this easily by alias command (`alias usbc='sudo usbip attach -r $(win_ip) -b '`), and run `usbc 1-1`.

4. You will see your USB device disappear from the Windows device manager. Now you can see the USB devices in WSL2 by calling `dmesg | grep tty`, `lsusb` or `ls /dev | grep ttyUSB*`.

## Troubleshooting

1. If you see `usbipd: error: Server is currently not running.`, or steps in `On WSL2` is blocked without any log, it means the usbipd server is not running. Go back to `On Windows` step 1, run usbipd server and try again.

2. (Windows 10) If you see information like: `USBIPD is not supported on WSL2`, or `Using wrong WSL version`, it may mean that, the kernel you are using doesn't have the USB support. Go back to `Recompile and replace the WSL2 kernel with USB support` and try it again.

[Melbourne]

Note: WSL2 also is unable to connect to PCI cards (pcie, pci-x and pci express). Including serial-port cards. The solution is the same: install a network serial port virtualization program on client and host.

[demooe]

Since the title doubtless brings others to this page as well, here are my (2024) updated instructions -> https://github.com/lure23/ESP32-WSL

Note: Only tried it with RISC V based development board (C3). Contributions are welcome!

[ivobrett]

Hi,
Thanks for this guide. I want to develop matter applications. For matter commissioning / discovery, the commissioner needs to do service discovery using mDNS. When I have an external matter device (e.g. running on a ESP32 SOC) then service discovery to WSL2 doesnt work.

I tried the following on WSL2 ubuntu22.04:

# Ensured that dbus and avahi-daemon has started

Code: Select all

sudo service dbus start
sudo service avahi-daemon start

# Next we will check that the services are running

Code: Select all

/etc/init.d/dbus status
/etc/init.d/avahi-daemon status

# Next we will check that we can discover both commissioned and commissioning devices

Code: Select all

avahi-browse -rt _matterc._udp
avahi-browse -rt _matter._tcp

!!! This fails i.e. times out with no devices found.

Note: people have reported issues with mDNS from WSL2 and have suggested some routing at the hyper-v level but this is unclear to me.

Anyone found a solution so that matter development for esp32 can be done on WSL2?