Wierdly encoded characters when receiving data.

[Feelfree]

Hi! I'm writing IRC bouncer on esp32. Implemented basic functionality, but when I receive data, I'm randomly getting characters replaced like ToggleSwitchWid�Ž�Žû?�gû?�, cherryh.freenode.net @�û?Í352 - Hmm, maybe its something with this buffer to string operation? Thanks!
https://gist.github.com/feelfreelinux/8 ... cbda0f463e

[ESP_Angus]

Hi Feelfree,

You don't seem to have tried fixing the buffer overflow issue that I mentioned on github (receiving the full size of the buffer doesn't leave space for a terminating null byte, so you can't treat it as a string). Are you sure that's not the problem?

Angus

[Feelfree]

Hi Feelfree,

You don't seem to have tried fixing the buffer overflow issue that I mentioned on github (receiving the full size of the buffer doesn't leave space for a terminating null byte, so you can't treat it as a string). Are you sure that's not the problem?

Angus

Forgot to update gist - resized buffer to 500, problem still persist. Looks like it's only seen when there's a lot of messages in short peroid of time(Like IRC motd).

[ESP_igrr]

Can you update the gist now? It still has the problem Angus has described, that is you aren't zero-terminating your string.
Second thing: if you increase buffer size, make sure you increase the stack size as well.

[Feelfree]

Ok, updated gist.

[Feelfree]

And I ported this to PC, linux. This code works correctly here. Looks like its something with lwip/esp32. https://gist.github.com/feelfreelinux/9 ... 44a9ae6186

[ESP_Angus]

The Linux version doesn't (yet) buffer overflow because it's receiving less than 500 bytes at a time (a combination of more CPU power and faster standard output as it's not speed limited by the serial port). The potential buffer overflow is still there on Linux, it just isn't triggered as easily (some combination of CPU load and network load would eventually trigger it).

EDIT: actually a better explanation of why the Linux version doesn't crash is that the first byte of data after the 500 byte buffer happens to be zero. You can't rely on this , though!

Here's the ESP32 main.cpp file modified to avoid the buffer overflow. This works for me, whereas the previous version triggers a fatal exception or produces garbage output: https://gist.github.com/projectgus/8596 ... a6f0b5fbfa

Some tips for things to read about in order to gain more skills with this kind of network code development:

• Learn what a null-terminated string is in C, and what the difference between a string and a char buffer is in C. Knowing this difference will help you understand why the code above fixes the problem, whereas increasing the buffer size to 500 did not.
• Read the 'man' pages for recv, connect, socket. Update your code to check results from these.
• You'll probably want to adapt your code to use 'select' at some point.
• In your code, remember that a single call to recv() can return any number of bytes (TCP is a byte stream not a packet stream) - so you can receive anything between 0 whole lines of output (ie a partial incomplete line) and multiple lines of output from the server in one hit. You'll need to accumulate that raw data into a buffer, and then parse that buffer for lines of output.

Enjoy!