socket connect error 104

[davdav]

Hi everybody,
In our project ESP32 connects to a TCP server and stay connected sending some data and doing periodic keepalive.If connection is lost, ESP32 reconnects after 10 second. So far so good.

I have tested with http-request example (modified for my scope) and it works. I then copied the code in my project and the first connection is good with an allocated socket integer to 8192. When I simulate a lost of connection and ESP32 reconnects, the assigned socket integer number is 8193 (or above) and then it fails to connect reporting errno=104.

I have checked that the allocated socket number in http-request is around 4096-4104 and it works..could it be a problem of the integer number allocated to the socket?

If I delete the task with vTaskDelete and then create again with

Code: Select all

xTaskCreate(smm_task, "SmmTask", 8192, NULL, 10, &WIFI_SmmTask_Hndl);

the first connection is good. Could it be a problem of resource not released? However, when the connection is lost I use

Code: Select all

shutdown(socket, SHUT_RDWR);
close(socket);

to close the socket before trying reconnection.

I would like also to report a couple of crash I had during test:

1. Allocated socket integer was 8194

Code: Select all

Guru Meditation Error: Core  1 panic'ed (LoadProhibited)
. Exception was unhandled.
Core 1 register dump:
PC      : 0x401750c9  PS      : 0x00060230  A0      : 0x8016232d  A1      : 0x3ffe5060  
A2      : 0x0001364a  A3      : 0x00000008  A4      : 0x3ffee05c  A5      : 0x00000000  
A6      : 0x00000000  A7      : 0x00060023  A8      : 0x80175315  A9      : 0x3ffe5040  
A10     : 0x3ffc8d44  A11     : 0x00000000  A12     : 0x00000000  A13     : 0x00000000  
A14     : 0x00060023  A15     : 0xff000000  SAR     : 0x00000019  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x0001364a  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  

Backtrace: 0x401750c9:0x3ffe5060 0x4016232a:0x3ffe5080 0x401641e5:0x3ffe50a0 0x4016ac0d:0x3ffe50c0 0x4016de02:0x3ffe50e0 0x40174366:0x3ffe5100 0x40164c19:0x3ffe5120

Entering gdb stub now.
$T0b#e6
xtensa-esp32-elf-gdb ./build/Avior_ESP32_WiFi.elf -b 115200 -ex 'target remote COM6'
GNU gdb (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a5) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-host_pc-mingw32 --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/Avior_ESP32_WiFi.elf...done.
Remote debugging using COM6
0x401750c9 in sys_sem_signal (sem=0x1364a)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/port/freertos/sys_arch.c:133
133         xSemaphoreGive(*sem);
(gdb) bt
#0  0x401750c9 in sys_sem_signal (sem=0x1364a)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/port/freertos/sys_arch.c:133
#1  0x4016232d in event_callback (conn=<optimized out>, evt=<optimized out>,
    len=<optimized out>)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/api/sockets.c:1922
#2  0x401641e8 in recv_tcp (arg=0x3ffec348, pcb=<optimized out>,
    p=<optimized out>, err=<optimized out>)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/api/api_msg.c:303
#3  0x4016ac10 in tcp_input (p=<optimized out>, inp=<optimized out>)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/core/tcp_in.c:469
#4  0x4016de05 in ip4_input (p=0x3ffef2f8, inp=0x3ffec414)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/core/ipv4/ip4.c:711
#5  0x40174369 in ethernet_input (p=0x3ffef2f8, netif=0x3ffec414)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/netif/ethernet.c:176
#6  0x40164c1c in tcpip_thread (arg=<optimized out>)
    at C:/msys32/home/Davide/esp/esp-idf/components/lwip/api/tcpip.c:143

2.Allocated socket integer was 8193

Code: Select all

assertion "select_cb.prev == NULL" failed: file "C:/msys32/home/Davide/esp/esp-idf/components/lwip/api/sockets.c", line 1764, function: lwip_select
abort() was called at PC 0x400e88d3 on core 0

Backtrace: 0x400924d8:0x3ffedf20 0x4009267b:0x3ffedf40 0x400e88d3:0x3ffedf60 0x40162fca:0x3ffedf90 0x400e8509:0x3ffedff0

Entering gdb stub now.
$T0b#e6

xtensa-esp32-elf-gdb ./build/Avior_ESP32_WiFi.elf -b 115200 -ex 'target remote COM6'
GNU gdb (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a5) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-host_pc-mingw32 --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/Avior_ESP32_WiFi.elf...done.
Remote debugging using COM6
0x400924d8 in invoke_abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:139
139             *((int *) 0) = 0;
(gdb) bt
#0  0x400924d8 in invoke_abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:139
#1  0x4009267e in abort ()
    at C:/msys32/home/Davide/esp/esp-idf/components/esp32/panic.c:148
#2  0x400e88d6 in __assert_func (
    file=0x3f429e60 "C:/msys32/home/Davide/esp/esp-idf/components/lwip/api/sockets.c", line=1764, func=<optim                             ized out>,
    failedexpr=0x3f42a098 "select_cb.prev == NULL")
    at ../../../.././newlib/libc/stdlib/assert.c:63
#3  0x40162fcd in lwip_select (
/builds/idf/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: As                             sertion `!frame_id_eq (*this_id, outer_frame_id)' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) [answered Y; input not from terminal]

This is a bug, please report it.  For instructions, see:
<http://www.gnu.org/software/gdb/bugs/>.

/builds/idf/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: As                             sertion `!frame_id_eq (*this_id, outer_frame_id)' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Create a core file of GDB? (y or n) [answered Y; input not from terminal]

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Thanks in advance for help.

[davdav]

anyone has idea?
Thanks

[kolban]

Howdy,
You have a lot of "stuff" here. For me, what would be useful would be a diagram illustrating what you are connecting to and from. What is the partner to the ESP32? Have you the ability to run a WireShark trace to examine the underlying requests? How are you simulating a loss of connection? When the connection is lost, what does the partner think has happend? etc etc

[WiFive]

What esp-idf version ?

[davdav]

@WiFive: version is esp-idf v3.0 commit f586f5e690e687f8db219a8b509bbf483fc416d6 (I guess latest because I did "git pull" a couple of days ago).

@Kolban: The partern of ESP32 (the server) is a C program which accepts connection on a specific TCP port and do not close the connection but wait for data from the client. The data are exchanged in raw (no HTTP, MQTT or other standard application protocol). The server side is OK (tested with other devices and GSM modules). To simulate connection lost I just kill the server program. ESP32 recognizes the lost of connection and wait for 10 seconds before attempting a new connection to same TCP port of the server.
Then, I start again the server program but ESP32 fails to connect.
As I wrote if I use the "http-request" example in esp-idf it works, so the procedure is OK. Discarding for the moment the two crashes reported in my first post, the only difference I can see between "http-request" example and my project is the integer number reported when socket is created (8192 against 4096).

I will try to do a wireshark debug session to see if I can find other hints.

Thanks

[kolban]

Nothing obvious comes to mind. If it were me, I'd write the simplest sockets application on the ESP32 that I could get away with. Something like (pseudo code)

Code: Select all

while(1) {
   s = socket();
   connect(s, address);
   send a test message;
   wait for a disconnect;
   close(s);
   sleep 10;
}

No application code or anything else. If this doesn't work, you will then have the simplest sample application you can post that would allow other community members to have a test and see what happens on their environment. Building samples like this is obviously a waste of energy ... but I consider it the tax on programming as it is likely to get to an answer overall quicker.

[davdav]

Thank you kolban for suggestion. As I wrote using a sample code ("http-request" in esp-idf example folder) it works so I'm not saying that esp-idf core is not working and it is for sure something in my code (even if it is quite similar to the above example). Obviusbly, the "http-request" is 200 lines of code, my project have a lot of more code.

Since I was not able to catch with wireshark any trace, I ask if someone knows how to enable LWIP debugs for socket. I have checked in the component folder and there are some define to activate debugs. For example in

Code: Select all

~\esp\esp-idf\components\lwip\include\lwip\lwip\opt.h
~\esp\esp-idf\components\lwip\include\lwip\lwip\debug.h
~\esp\esp-idf\components\lwip\include\lwip\port\lwipopts.h

And I have enabled some flags but at the end I don't get any trace on com port. How to enable the debug for LWIP?

Thanks

[ESP_Mahavir]

Try changing debug options from

components/lwip/include/lwip/port/lwipopts.h

Following are few options that might be helpful here,

#define TCP_DEBUG LWIP_DBG_ON
#define TCP_INPUT_DEBUG LWIP_DBG_ON
#define SOCKETS_DEBUG LWIP_DBG_ON

[davdav]

I have found the problem. It was the TCP port not correctly setup when it reconnection. My mistake... Thanks to @Kolban and @WiFive for help.