Serve HTML from SPIFFS

[pushtoopen]

Hi All, i've integrated the standard AP webserver code in with this example for using SPIFFS filesystem https://github.com/espressif/esp-idf/tr ... age/spiffs

Normally once i detect a "GET" from the connected socket, i return a HTML header, then body, written as a char string. I.E.

"netconn_write(conn, http_html_hdr, sizeof(http_html_hdr)-1, NETCONN_NOCOPY);"
and
"netconn_write(conn, http_index_hml, sizeof(http_index_hml)-1, NETCONN_NOCOPY);"

Where http_html_hdr is a standard http 200 response and http_index_hml is my actual html file.

This works great and i have no issues for the most part. I integrated in the SPIFFS filesystem and was able to read a flashed HTML file out as a char string (to serial) using the following code (copied/modified from the above SPIFFS example)

char readFile(char *fname)
{
printf("==== Reading from file \"%s\" ====\r\n", fname);

int res;
char *buf;
buf = calloc(1024, 1);
if (buf == NULL) {
printf(" Error allocating read buffer\"\r\n");
return NULL;
}

FILE *fd = fopen(fname, "rb");
if (fd == NULL) {
printf(" Error opening file\r\n");
free(buf);
return NULL;
}
res = 999;
res = fread(buf, 1, 1023, fd);
if (res <= 0) {
printf(" Error reading from file\r\n");
}
else {
printf(" %d bytes read [\r\n", res);
buf[res] = '\0';
printf("%s\r\n]\r\n", buf);
}
return(buf);
//free(buf);

res = fclose(fd);
if (res) {
printf(" Error closing file\r\n");
}
printf("\r\n");
}

When i use readFile, it prints to serial no problem (not the printf near the return of buf). But when i try to run readFile while in the middle of my HTTP GET response script, i get stack overflows or assertion failures.
I.E.

SERIAL OUT:
DNS: Q (type 0x1 class 0x1) for connectivitycheck.gstatic.com
buffer = GET /generate_204 HTTP/1.1
User-Agent: Dalvik/2.1.0 (Linux; U; Android 6.0.1; XT1254 Build/MCG24.251-5-5)
Host: connectivitycheck.gstatic.com
Connection: Keep-Alive
Accept-Encoding: gzip

Sc5+?l?
buf[5] = g
==== Reading from file "/spiffs/images/HtmlExample.txt" ====
***ERROR*** A stack overflow in task http_server has been detected.
abort() was called at PC 0x40088340 on core 1
0x40088340: vApplicationStackOverflowHook at C:/msys32/home/esp/esp-idf/components/esp32/panic.c:572


If i free up the buffer of the client before reading in new data, i get this

SERIAL OUT:
DNS: Q (type 0x1 claass 0x1) for connectivitycheck.gstatic.com
buffer = GET /generate_204 HTTP/1.1
User-Agent: Dalvik/2.1.0 (Linux; U; Android 6.0.1; XT1254 Build/MCG24.251-5-5)
Host: connectivitycheck.gstatic.com
Connection: Keep-Alive
Accept-Encoding: gzip


buf[5] = g
assertion "next > (intptr_t)block" failedd: file "C:/msys32/home/esp/esp-idf/components/heap/multi_heap.c", line 116, function: get_next_block
abort() was called at PC 0x401085c3 on core 1
0x401085c3: __assert_func at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdlib/../../../.././newlib/li
bc/stdlib/assert.c:63 (discriminator 8)

any thoughts?
Below is my http request read & response code

static void http_server_netconn_serve(struct netconn *conn)
{
struct netbuf *inbuf;
char *buf;
u16_t buflen;
err_t err;

/* Read the data from the port, blocking if nothing yet there.
We assume the request (the part we care about) is in one netbuf */
err = netconn_recv(conn, &inbuf);

if (err == ERR_OK) {
netbuf_data(inbuf, (void**)&buf, &buflen);

// strncpy(_mBuffer, buf, buflen);

/* Is this an HTTP GET command? (only check the first 5 chars, since
there are other formats for GET, and we're keeping it very simple )*/
printf("buffer = %s \n", buf);
if (buflen>=5 &&
buf[0]=='G' &&
buf[1]=='E' &&
buf[2]=='T' &&
buf[3]==' ' &&
buf[4]=='/' ) {
printf("buf[5] = %c\n", buf[5]);
/* Send the HTML header
* subtract 1 from the size, since we dont send the \0 in the string
* NETCONN_NOCOPY: our data is const static, so no need to copy it
*/

netconn_write(conn, http_html_hdr, sizeof(http_html_hdr)-1, NETCONN_NOCOPY);

if(buf[5]=='h' || buf[5]=='g') {

/* Send our HTML page */
//free(buf);
//heap_caps_check_integrity(MALLOC_CAP_8BIT,true);
char filebuf= readFile("/spiffs/images/HtmlExample.txt"); // <----------------- fails here
netconn_write(conn, phork, sizeof(filebuf)-1, NETCONN_NOCOPY);
}
else if(buf[5]=='l') {

/* Send our HTML page */
netconn_write(conn, http_index_hml, sizeof(http_index_hml)-1, NETCONN_NOCOPY);
}
else if(buf[5]=='j') {
netconn_write(conn, json_unformatted, strlen(json_unformatted), NETCONN_NOCOPY);
}
else {
netconn_write(conn, http_index_hml, sizeof(http_index_hml)-1, NETCONN_NOCOPY);
}
}

}
/* Close the connection (server closes in HTTP) */
netconn_close(conn);

/* Delete the buffer (netconn_recv gives us ownership,
so we have to make sure to deallocate the buffer) */
netbuf_delete(inbuf);
}

[kolban]

Howdy my friend,

This forum is full of great info but can be hard to search. I use Googles site specific search:

site:esp32.com stack overflow

which works great. Let me suggest you run that the search above and have a review at some of the preceeding comments. At the highest level, what your error is telling us is that you are using some ESP32 FreeRTOS tasks. When those tasks are created, they are initialized with modest amounts of stack. It would appear that you have run low on task stack. The solution will be to run the offending tasks with more stack.

[fly135]

My early experience was that just printing in a task added ~2K to the stack req.

[burkulesomesh43]

Hi all,
I am using file system to store html code of webserver. but in my code there are some css, js file references which are runs over internet. so if network dont have internet my webserver won't see properly like bootstrap.

Code: Select all

"<link rel=\"stylesheet\" href=\"https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css\">"
		"<script src=\"https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js\"></script>"
		"<script src=\"https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js\"></script>"

so I want to store this css and js files in flash. when code will run html code will take references of this files.
but how to do it with idf?

[fly135]

Hi all,
I am using file system to store html code of webserver. but in my code there are some css, js file references which are runs over internet. so if network dont have internet my webserver won't see properly like bootstrap.

Code: Select all

"<link rel=\"stylesheet\" href=\"https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css\">"
		"<script src=\"https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js\"></script>"
		"<script src=\"https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js\"></script>"

so I want to store this css and js files in flash. when code will run html code will take references of this files.
but how to do it with idf?

My experience with spiffs is that you would just refer to the file path and can open the file. My guess is that you would simply put that path (eg. /storage/xyz.css) in your HTML and the client will send a GET with that filepath. You would open the file, read it in, and feed it back to the client. I'm not sure how much support the webserver handles. So the web server may do the open/read/send for you.

John A

[burkulesomesh43]

Hi all,
I am using file system to store html code of webserver. but in my code there are some css, js file references which are runs over internet. so if network dont have internet my webserver won't see properly like bootstrap.

Code: Select all

"<link rel=\"stylesheet\" href=\"https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css\">"
		"<script src=\"https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js\"></script>"
		"<script src=\"https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js\"></script>"

so I want to store this css and js files in flash. when code will run html code will take references of this files.
but how to do it with idf?

My experience with spiffs is that you would just refer to the file path and can open the file. My guess is that you would simply put that path (eg. /storage/xyz.css) in your HTML and the client will send a GET with that filepath. You would open the file, read it in, and feed it back to the client. I'm not sure how much support the webserver handles. So the web server may do the open/read/send for you.

John A

is there any example available on esp idf or elsewhere?
I am just confused which api's will be usefull for that.

[fly135]

is there any example available on esp idf or elsewhere?
I am just confused which api's will be usefull for that.

I had an issue with creating a burnable file system, which you'll need to do. This link to my issue should provide you with some answers to your question since I went over what I did in somewhat detail.

viewtopic.php?f=2&t=8387

John A

[burkulesomesh43]

is there any example available on esp idf or elsewhere?
I am just confused which api's will be usefull for that.

I had an issue with creating a burnable file system, which you'll need to do. This link to my issue should provide you with some answers to your question since I went over what I did in somewhat detail.

viewtopic.php?f=2&t=8387

John A

I got ur point. actually you have to wear levelling example from esp idf.
but my problem is that I have to store js and css file in flash. and it will take upto 350Kb of space to store it.
and i have used component embed files which can be stored in flash and consumes code size also. I don't want to waste code size.
then how to use it.
here are some links I raised issues in forum so that you can understand my problem->>
viewtopic.php?f=13&t=1933&start=10#p35423
viewtopic.php?f=13&t=8429#p35373

[fly135]

I don't have any experience with an http server on the ESP32. But you can get a larger flash than 4MB. We have 16MB on our design. If you need more space, there may be other options out there. Not sure what would be the best approach. Maybe an SD card.

[burkulesomesh43]

I don't have any experience with an http server on the ESP32. But you can get a larger flash than 4MB. We have 16MB on our design. If you need more space, there may be other options out there. Not sure what would be the best approach. Maybe an SD card.

Ok. thank you for your response.