How to let webserver access file system of the SD Card

[Gfast2]

Hi ESP-IDF,

I just wunder, how to let ESP32 plays like a traditional static webserver?

My goal ist to let Mongoose Webserver access the virtual filesystem of SD card, and fetch the file that the webserver need.

I got this idea from Mongoose's stock example HERE
This is the code snippet that do the magic, and which also not works on my machine:

Code: Select all

  
  mg_set_protocol_http_websocket(nc);
  s_http_server_opts.document_root = ".";  // Serve current directory
  s_http_server_opts.enable_directory_listing = "yes";
  

---
The following is actually another question:
Right now i just specify one task for Webserver, one for SD Card I/O.
It that possible to pass file pointer between tasks? If yes, is that good practise?

Thanks

Gfast2

[kolban]

I think the answer is going to be a function of what software you use as a Web Server / HTTP Stack. The excellent Mongoose library will have one answer and my work in C++ classes has a different answer. As for sharing an "open file descriptor" between tasks, that should work fine as long as you are using the Virtual File System libraries. A file descriptor should be global across all tasks across a device.

[Gfast2]

I think the answer is going to be a function of what software you use as a Web Server / HTTP Stack. The excellent Mongoose library will have one answer and my work in C++ classes has a different answer. As for sharing an "open file descriptor" between tasks, that should work fine as long as you are using the Virtual File System libraries. A file descriptor should be global across all tasks across a device.

Hi Kolban,

It's my great honor to talk to you. Right now my webserver task is kind mimc your works.
But the problem here is:

• Firstly, if I convert my webpage like you did, the size of the file would be very big.
  Secondly, if it is huge, I can't save it in flash anymore.
  Thirdly, since they are huge, I've to try to transport them like how "fread()" dealing with file I/O: Only a certain number of character each time.

I did investigate some extra time on how to send the file in multiple parts inspired by your other brillient Code "Webserver.cpp".
Today I will going on consentrate on this topic. If I did get good result, I will post my result somehow.

Thanks for your Book and "ESP32 Technical Tutorial". Please consider make more tutorials about advanced topics like you've did on "Concurrent Socket".

[kolban]

I have stopped tinkering with the Mongoose web server. Don't get me wrong, I think its a fantastic piece of work. My choice to not use the Mongoose server was because I wanted something that was based on C++ and I wanted to "play" with what is involved in making a Web Server. To that end, we have build a new set of classes called "HttpServer". The docs for these can be found here:

https://github.com/nkolban/esp32-snippe ... Server.pdf

One of the goals for this is to be able to "stream". What I mean by that is that instead of reading the data one wants to send into RAM and then sending it as a complete unit, the goal is to treat the source as an "output stream" and attach that to the "input stream" to the browser. This will be accomplished by the C++ streaming technology ... (see also: https://en.wikipedia.org/wiki/Standard_streams)

One of the benefits of this is that within an application, we can create:

* A string stream for sending data that we generate in our app
* A file stream for sending that that is sourced from a Posix file

The conversion of the web page to a large header file is only useful if the web page is small (exactly as you said). If the web page were larger or was many web pages, then I would use SPIFFS to make a file image and move that into flash. If the total amount of my web data that I wanted to server was larger than would fit in flash, then that begs the question ... where will it be stored? The likely answer is on an SD card or equivalent. At that point we would use a Virtual File System attached to the car and write the data into the card in some other way.

[Dig Kleppe]

Hi guys,
The attatchment of my post viewtopic.php?f=2&t=3522&p=16307&hilit= ... art#p16307
contains a http server that reads from SD card connected in SPI mode.
The problems described in this post are not related to the http part.
Dig

[Gfast2]

I have stopped tinkering with the Mongoose web server. Don't get me wrong, I think its a fantastic piece of work. My choice to not use the Mongoose server was because I wanted something that was based on C++ and I wanted to "play" with what is involved in making a Web Server. To that end, we have build a new set of classes called "HttpServer". The docs for these can be found here:

https://github.com/nkolban/esp32-snippe ... Server.pdf

One of the goals for this is to be able to "stream". What I mean by that is that instead of reading the data one wants to send into RAM and then sending it as a complete unit, the goal is to treat the source as an "output stream" and attach that to the "input stream" to the browser. This will be accomplished by the C++ streaming technology ... (see also: https://en.wikipedia.org/wiki/Standard_streams)

One of the benefits of this is that within an application, we can create:

* A string stream for sending data that we generate in our app
* A file stream for sending that that is sourced from a Posix file

The conversion of the web page to a large header file is only useful if the web page is small (exactly as you said). If the web page were larger or was many web pages, then I would use SPIFFS to make a file image and move that into flash. If the total amount of my web data that I wanted to server was larger than would fit in flash, then that begs the question ... where will it be stored? The likely answer is on an SD card or equivalent. At that point we would use a Virtual File System attached to the car and write the data into the card in some other way.

Hi Kolban,

Thank you again for your informative answer. Your new webserver implementation is very very impressive (again) I will check that out very soon.
I have a colleage who is more then 50 yesrs old now. He told me, many thing in C++ is much more powerful, but if I've the interest to check out how the code compiled from C++ to C looks like, I'd really wanna avoid to do everything in C++. So since I'm really new here. I'd like to know what is your Motivation to build C++ Utilitys for ESP32. Is it because you are a Java Guy, OOP for you is just the most absolute priority thing? To tell the truth, I have always the feeling that C is the geniue language for ESP32 as the ESP-IDF do. C++ is more powerful, but still not bullet prooft. But anyway, if I got much more understanderation from your cool C++ library, I will do deliver it in my up comming project.

BTW:
This is the description of my test and its error logs.

[Gfast2]

Hi guys,
The attatchment of my post viewtopic.php?f=2&t=3522&p=16307&hilit= ... art#p16307
contains a http server that reads from SD card connected in SPI mode.
The problems described in this post are not related to the http part.
Dig

Hi Dig Kleppe,

Thanks for your Post. Since I do not have much experiences with the functions you mentioned. I can not tell much from your advice. But thank you all the same.

MfG

Gfast

[kolban]

Gfast2,
I also am more than 50 years old ... so don't listen to a word I say

I'm afraid I was programming in C and C++ before Java was even a concept. There is a lot of FUD when folks thing of C++. The simplest way to think of it (in my opinion) is that C++ is a superset of C. What I mean by that is that if you take a C program, by and large it will compile with a C++ compiler and produce compiled code that is going to be pretty darn equivalent to that produced by the C compiler. So don't let anyone say that "I know C but I don't know C++". If one knows C, one knows 90% of the C++ language. What C++ does is add the concept of "classes" (amongst a few other things). If one chooses not to use classes, then I'd argue that C and C++ are so close as to not make a difference.

As for the ESP-IDF, so far, I have no found a single ESP-IDF API that I couldn't call from C++ just as easily as from C.

But ... we are rapidly getting into a red vs blue vs green discussion. Don't let anyone tell you that "their language is best". My advice, study enough of them all to determine what their distinctions may be. If someone says "My language is the best" or "YOU should use X" then just nod and say "thank you"

I'm not trying to push C++ on anyone ... rather, the most I am saying is "for me, I've been pleased with C++ and offer to assist anyone with C++ on ESP32 as desired".

[Gfast2]

Gfast2,
I also am more than 50 years old ... so don't listen to a word I say

I'm afraid I was programming in C and C++ before Java was even a concept. There is a lot of FUD when folks thing of C++. The simplest way to think of it (in my opinion) is that C++ is a superset of C. What I mean by that is that if you take a C program, by and large it will compile with a C++ compiler and produce compiled code that is going to be pretty darn equivalent to that produced by the C compiler. So don't let anyone say that "I know C but I don't know C++". If one knows C, one knows 90% of the C++ language. What C++ does is add the concept of "classes" (amongst a few other things). If one chooses not to use classes, then I'd argue that C and C++ are so close as to not make a difference.

As for the ESP-IDF, so far, I have no found a single ESP-IDF API that I couldn't call from C++ just as easily as from C.

But ... we are rapidly getting into a red vs blue vs green discussion. Don't let anyone tell you that "their language is best". My advice, study enough of them all to determine what their distinctions may be. If someone says "My language is the best" or "YOU should use X" then just nod and say "thank you"

I'm not trying to push C++ on anyone ... rather, the most I am saying is "for me, I've been pleased with C++ and offer to assist anyone with C++ on ESP32 as desired".

Hi Kolban,

Thanks for your sharing about how to dealing about new stuff from your experiences.
My colleage told me: the most useful software language is the one fit yourself best.

I'll keep on looking new suffs. Thanks again. If I did got something really interesting. I'll post it here later.

cheers

Gfast2

[Gfast2]

Hi ESP-IDF,

I've been thinking about this question for a while. Since today I successfully figured out where is the problem (Big Thanks to iggr he is very awesome!). I'd very pleased to share the linke to you -> some one very curiously experienced the same problem as I did:

https://github.com/espressif/esp-idf/is ... 1333870401