HTTP requests (GET, POST etc.) on ESP32

[andrew_p]

Thanks! I think I figured it out...

Do I understand it correct that when we set MBEDTLS_SSL_VERIFY_NONE mode, the request will be served over https (encrypted), but no certificate verification will be performed?

Code: Select all

mbedtls_ssl_conf_authmode(&conf, MBEDTLS_SSL_VERIFY_NONE);

(in fact, for my task I don't need a certificate verification, since I'm using basic auth in my POST request)

[kolban]

That's my understanding. SSL provides encryption and authentication. Encryption is that the data flowing over the wire is encrypted while authentication is validating that the party to which you have connected are who they claim to be. Obviously, if the remote party is not who they claim to be then encryption is worthless ... but if you "trust" the network, then you will be encrypted.

[WiFive]

Thanks! I think I figured it out...

Do I understand it correct that when we set MBEDTLS_SSL_VERIFY_NONE mode, the request will be served over https (encrypted), but no certificate verification will be performed?

Code: Select all

mbedtls_ssl_conf_authmode(&conf, MBEDTLS_SSL_VERIFY_NONE);

(in fact, for my task I don't need a certificate verification, since I'm using basic auth in my POST request)

If you don't use certificate, there can be man in middle attack and attacker can steal your basic auth credentials. If your server is on public internet and doesn't give a warning for HTTPS in browser then it does use a valid certificate.

[andrew_p]

OK, so a certificate is not about authentication/authorization, we need a valid and signed certificate in order establish encrypted channel, right?

And if I want to:
1. Have a secure/encrypted channel with my remote server (it serves https only)
and
2. Make sure nobody can sniff traffic and steal basic auth credentials

...I have to use a valid signed certificate, right? Just found that I can purchase one for $50-$1500/year.. wow

[WiFive]

https://letsencrypt.org

https://en.m.wikipedia.org/wiki/Self-signed_certificate

[andrew_p]

If you don't use certificate, there can be man in middle attack and attacker can steal your basic auth credentials. If your server is on public internet and doesn't give a warning for HTTPS in browser then it does use a valid certificate.

I feel like I'm still a bit confused about necessity of a trusted certificate on a client ESP32 device.

And I think there is no way how to get a signed trusted certificate for an IoT device, you need to have a valid domain name for that.

Can someone explain mbedtls_ssl_conf_authmode options in regards to ESP32(client)->remote server communications?

This is what documentation says:

Code: Select all

MBEDTLS_SSL_VERIFY_NONE: peer certificate is not checked (default on server) (insecure on client)
MBEDTLS_SSL_VERIFY_OPTIONAL: peer certificate is checked, however the handshake continues even if verification failed; 
MBEDTLS_SSL_VERIFY_REQUIRED: peer must present a valid certificate, handshake is aborted if verification failed. (default on client)

My understanding is that since my remote server accepts self-issued certificates, I can just issue one and then my communication will be performed over HTTPS, right? But how those mbedtls_ssl_conf_authmode options will affect encrypted channel in my case?

[kolban]

Howdy @andrew_p,
I'll be the first to state that I'm no expert on SSL ... however I'm studying the mbedtls_*** apis as quickly as I can. In this discussion, let us assume that the ESP32 is going to be an HTTP client and there is a remote web server out there that is an HTTP server. From what I can see, SSL happens "outside" of the HTTP protocols ... and by that I mean that the underlying transport protocol (TCP in this case) is encrypted. So when our ESP32 wishes to send an HTTP request ... eg:

Code: Select all

GET / HTTP/1.1

Then that is the text of the HTTP request our app will send ... and it will be passed to mbedtls ... which encrypts the data and then transmits it over a regular socket. At the receiver, the encrypted text will be received (again over regular sockets) and will then be passed through the SSL stack on the receiver side to decrypt.

If we are thinking straight so far ... then lets delve deeper.

At a high level, I understand SSL uses large random numbers to encrypt the data. My basic knowledge says that there is a public and a private key (a pair of numbers). And the ESP32 will have one PAIR (a public and private) and the partner will have another pair. When an SSL session starts, the ESP32 will ask the receiver for its public key ... when the ESP32 receives that, it will send ITS public key encrypted using the public key of the receiver. The receiver will now decrypt the message using its private key and now both ends of the session know each other's public keys. Now they can exchange data freely using each other's public keys for encryption and only the correct receiver should be able to decrypt as they are the only ones who know the secret (their respective private keys).

At the simplest level, we now have encryption at play ... and no certificates our other "bits and pieces" were used.

Any problems with this story? Not superficially ... it can and does work. However, there are "issues" associated with this story IF we need deeper (better?) security. The first issue is the question of "are we actually talking to who we think we are talking to?" ... if I connect to IP address 1.2.3.4 and start exchanging SSL encrypted data ... am I "really" talking to 1.2.3.4? That's where certificates can come into play. Those certificates can validate that entity I am talking to is actually who it claims to be. There is also the concept of mutually exchanged certificates ... where the ESP32 could send a certificate that validates it is who it claims to be such that the receiver can know that the ESP32 is who it claims to be.

And this is where you have to ask yourself ... how far do I need to go?

For example ... when I use a browser on my desktop and connect to my bank over SSL, I want my browser to know that the bank is who it claims to be (a reason in a second). Once I connect my browser to the bank, I then enter my account number and password ... it is at THAT point that the bank knows who I am and trust that I am who I claim to be (by virtue of the userid/password pair ... that should only be known by me). Because the bank sent me a certificate AND I validated that the certificate was correct .. THEN I trust that I now have a secure connection to the bank ... and not someone impersonating that bank. If there was an impersonator, they would be able to get my userid/password. However, when the bank sent me their certificate at the start of the session ... it was MY responsibility (i.e. my browser or ESP32) to validate that the certificate was correct for whom I was trying to contact. If not correct, then it is up to ME to terminate the conversation ... not the bank ... it is happy to carry on ... because it doesn't need to trust my "physical browser" or "physical ESP32" ... as it doesn't use that for authentication ... but rather uses the supplied userid/password pair.

[andrew_p]

Neil, thanks for the great explanation! This definitely makes sense.

In my case, I suppose, my remote server handles connection over https, but it doesn't care about the client's certificate, it cares only about valid username/password in order to authenticate/authorize transaction (store payload).

On the other side, ESP32 as a client may or may not want to verify server's certificate to make sure it can be trust.

So there I see 3 use-cases, and I would really want to understand what mbedtls_ssl_conf_authmode options we should use for those. In all cases I suppose we use mbedtls to establish a secure connection:

Server - remote server, which handles GET, POST etc. requests over https with basic authentication (login/password)
Client - ESP32

Use Case #1:
- server doesn't care about the client's certificate, it cares only about credentials sent by a client
- client doesn't care about the server's certificate, it just wants to send data payload and credentials over a secure connection

User Case #2:
- server doesn't care about the client's certificate, it cares only about credentials sent by a client
- client wants to verify server's certificate

User Case #3:
- server wants to verify the client's certificate
- client wants to verify server's certificate

[WiFive]

Use Case #1:
- server doesn't care about the client's certificate, it cares only about credentials sent by a client
- client doesn't care about the server's certificate, it just wants to send data payload and credentials over a secure connection

Well it can't really be secure if it is not verified, so it is just encrypted....by someone


Verification is based on the certificate you load into the esp32 and by doing so you are saying it is trusted. This can be from a public certificate authority or you can be your own certificate authority if you control both client and server and manage your own chain of trust.

[andrew_p]

Well it can't really be secure if it is not verified, so it is just encrypted....by someone

Yes, I finally got it.. I just took a root certificate from my remote server, installed on ESP32 and now ESP32 can verify that the server is my real and indeed trusted server. Thanks!