lwip blocking send problem
Posted: Tue Aug 29, 2017 9:26 am
Hi all,
I've searched this forum and quite a lot on google as well, but I can't find an explanation.
I am working on a project which uses the ESP32 as a TCP socket client which only sends data to a TCP server (no receiving, just sending). I am using the lwIP stack with standard UNIX sockets API (connect, send, recv, etc), and I see a strange (to me, at least) phenomenon happening: the blocking send() function that I am using blocks for a duration of time which is dependent on the frequency at which I call the function!
I am using the ESP32 dev kit with the ESP-WROOM-32 module running a slightly modified version of the 'tcp_perf' example as a client, while for the TCP server I use netcat (started with 'nc -l 1985'). I use the esp-idf release v.2.1 for development.
My scenario is: call send() with ~28kB of data to send, then sleep for an amount of milliseconds. I have made some tests, with the server being on my local network and on an Amazon EC2 VM. Here are my results:
- local network: 100 ms sleep: send takes ~27 ms
500 ms sleep: send takes ~110 ms
1000 ms sleep: send takes ~120 ms
- Amazon EC2 VM: 100 ms sleep: send takes ~900 ms
500 ms sleep: send takes ~810 ms
1000 ms sleep: send takes ~820 ms
My connection to the EC2 VM is not the limiting factor, because I can wget to/from it with a rate of about 8 Mbit/s.
So my question is, why is this happening? What causes send() to block for longer amounts of time when calling more rarely?
Is this a known lwIP issue?
I am using the tcp_perf example with the default configuration.
Here is the code:
I have tried to increase the TCP_WND size to 4*TCP_MSS, but the results are the same.
I have also written a small C application which runs on Linux, based on the tcp_perf example, to check the behavior of the TCP/IP stack on a PC, but I don't get the variation of send() duration as I see on the ESP32. Also, I get from 60 to 90 ms send() duration when sending data to Amazon EC2.
Here is the PC code (the entire C file).
I've searched this forum and quite a lot on google as well, but I can't find an explanation.
I am working on a project which uses the ESP32 as a TCP socket client which only sends data to a TCP server (no receiving, just sending). I am using the lwIP stack with standard UNIX sockets API (connect, send, recv, etc), and I see a strange (to me, at least) phenomenon happening: the blocking send() function that I am using blocks for a duration of time which is dependent on the frequency at which I call the function!
I am using the ESP32 dev kit with the ESP-WROOM-32 module running a slightly modified version of the 'tcp_perf' example as a client, while for the TCP server I use netcat (started with 'nc -l 1985'). I use the esp-idf release v.2.1 for development.
My scenario is: call send() with ~28kB of data to send, then sleep for an amount of milliseconds. I have made some tests, with the server being on my local network and on an Amazon EC2 VM. Here are my results:
- local network: 100 ms sleep: send takes ~27 ms
500 ms sleep: send takes ~110 ms
1000 ms sleep: send takes ~120 ms
- Amazon EC2 VM: 100 ms sleep: send takes ~900 ms
500 ms sleep: send takes ~810 ms
1000 ms sleep: send takes ~820 ms
My connection to the EC2 VM is not the limiting factor, because I can wget to/from it with a rate of about 8 Mbit/s.
So my question is, why is this happening? What causes send() to block for longer amounts of time when calling more rarely?
Is this a known lwIP issue?
I am using the tcp_perf example with the default configuration.
Here is the code:
Code: Select all
char databuff[19*EXAMPLE_DEFAULT_PKTSIZE];
...
//send data
void send_data(void *pvParameters)
{
int len = 0;
memset(databuff, EXAMPLE_PACK_BYTE_IS, EXAMPLE_DEFAULT_PKTSIZE);
vTaskDelay(100/portTICK_RATE_MS);
ESP_LOGI(TAG, "start sending...");
#if EXAMPLE_ESP_TCP_PERF_TX && EXAMPLE_ESP_TCP_DELAY_INFO
//delaytime
struct timeval tv_start;
struct timeval tv_finish;
unsigned long send_delay_ms;
#endif /*EXAMPLE_ESP_TCP_PERF_TX && EXAMPLE_ESP_TCP_DELAY_INFO*/
while(1) {
#if EXAMPLE_ESP_TCP_PERF_TX && EXAMPLE_ESP_TCP_DELAY_INFO
total_pack++;
gettimeofday(&tv_start, NULL);
#endif /*EXAMPLE_ESP_TCP_PERF_TX && EXAMPLE_ESP_TCP_DELAY_INFO*/
//send function
gpio_set_level(14, 0);
len = send(connect_socket, databuff, sizeof(databuff), 0);
gpio_set_level(14, 1);
vTaskDelay(100/portTICK_RATE_MS);
#if EXAMPLE_ESP_TCP_PERF_TX && EXAMPLE_ESP_TCP_DELAY_INFO
gettimeofday(&tv_finish, NULL);
#endif /*EXAMPLE_ESP_TCP_PERF_TX && EXAMPLE_ESP_TCP_DELAY_INFO*/
I have also written a small C application which runs on Linux, based on the tcp_perf example, to check the behavior of the TCP/IP stack on a PC, but I don't get the variation of send() duration as I see on the ESP32. Also, I get from 60 to 90 ms send() duration when sending data to Amazon EC2.
Here is the PC code (the entire C file).
Code: Select all
/* tcp_perf Example
This example code is in the Public Domain (or CC0 licensed, at your option.)
Unless required by applicable law or agreed to in writing, this
software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied.
*/
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/time.h>
#define EXAMPLE_DEFAULT_SERVER_IP "52.53.111.152"
#define EXAMPLE_DEFAULT_PORT (1985)
#define EXAMPLE_PACK_BYTE_IS 97 //'a'
#define EXAMPLE_DEFAULT_PKTSIZE (1460)
/*socket*/
static struct sockaddr_in server_addr;
static int connect_socket = 0;
int total_data = 0;
char databuff[19*EXAMPLE_DEFAULT_PKTSIZE];
struct timeval timeBefore, timeAfter;
//send data
void send_data(void *pvParameters)
{
memset(databuff, EXAMPLE_PACK_BYTE_IS, EXAMPLE_DEFAULT_PKTSIZE);
while(1) {
(void)gettimeofday(&timeBefore, NULL);
//send function
(void)send(connect_socket, databuff, sizeof(databuff), 0);
(void)gettimeofday(&timeAfter, NULL);
long int elapsedTimeMs = (timeAfter.tv_sec * 1000000) + timeAfter.tv_usec - ((timeBefore.tv_sec * 1000000) + timeBefore.tv_usec);
printf("-- Elapsed time: %ldms\n", elapsedTimeMs);
usleep(100*1000);
}
}
//use this esp32 as a tcp client. return ESP_OK:success ESP_FAIL:error
void create_tcp_client()
{
connect_socket = socket(AF_INET, SOCK_STREAM, 0);
if (connect_socket < 0) {
printf("could not create client socket!\n");
return;
}
printf("socket = %d\n", connect_socket);
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(EXAMPLE_DEFAULT_PORT);
server_addr.sin_addr.s_addr = inet_addr(EXAMPLE_DEFAULT_SERVER_IP);
if (connect(connect_socket, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) {
printf("could not connect to server socket!\n");
}
}
//wifi_init_softap
int main(void)
{
create_tcp_client();
send_data(NULL);
return 0;
}