[Solved] fmt2rgb888() returns TRUE but doesn't work

[Star-Vision]

Hello,
I have a problem in my program and I haven't figured out how to fix it, so I'm calling on your enlightenment!

I am currently programming with an ESP32-CAM and trying to get an image in RGB888 format. As we cannot take the image directly in the desired format, the most obvious way is to take the image in JPEG format and then transform it into RGB888 format via the function fmt2rgb888().
I come to my problem: when I introduce a buf, let's say "_rgb888_buf" in the function, fmt2rgb888() returns TRUE to me and returns me the buf that I introduced to it with exactly the same value as when it goes in.

I created a little program a bit outside of my project to highlight the problem. I take a JPEG image, transform it to RGB888 via fmt2rgb888 (), then apply a filter to it that removes all green and red components at each pixel and then reshuffle the image as JPEG and send it to a web page to see the result.

Code: [Select all] [Expand/Collapse]

1. void startCameraServer(){
2.   httpd_config_t config = HTTPD_DEFAULT_CONFIG();
3.   config.server_port = 80;
4.   config.stack_size = 15000;
5.  
6.   httpd_uri_t index_uri = {
7.     .uri       = "/",
8.     .method    = HTTP_GET,
9.     .handler   = capture_handler,
10.     .user_ctx  = NULL
11.   };
12.  
13. static esp_err_t capture_handler(httpd_req_t *req) {
14.   camera_fb_t *fb = NULL;
15.   esp_err_t res = ESP_OK;
16.   size_t _jpg_buf_len = 0;
17.   uint8_t * _jpg_buf = NULL, * _rgb888_buf = NULL;
18.  
19.  
20.   res = httpd_resp_set_type(req, "image/jpeg");
21.   if (res == ESP_OK)
22.   {
23.     res = httpd_resp_set_hdr(req, "Content-Disposition", "inline; filename=image.jpg");  //capture
24.   }
25.   if (res == ESP_OK) {
26.     ESP_LOGI(TAG, "Take a picture");
27.     fb = esp_camera_fb_get();
28.  
29.     if (!fb)
30.     {
31.       ESP_LOGE(TAG, "Camera capture failed");
32.       httpd_resp_send_500(req);
33.       return ESP_FAIL;
34.     } else {
35.       Serial.println("we came by [2]");
36.  
37.       if(!fmt2rgb888((const uint8_t *) fb->buf,fb->len,fb->format, _rgb888_buf)) Serial.println("fmt2rgb888 function failed");
38.       else {
39.         if(_rgb888_buf == NULL) Serial.println("Why _rgb888_buf is NULL ????");
40.  
41.         Serial.println("we came by [3]");
42.         _rgb888_buf = photoshop_with_rgb888(_rgb888_buf, 600, 800); //We apply a Blue Filter on image
43.  
44.         if(!fmt2jpg(_rgb888_buf, fb->height*fb->width*3, fb->width,fb->height, PIXFORMAT_RGB888, 15, &_jpg_buf, &_jpg_buf_len)) Serial.println("fmt2jpg function failed");
45.  
46.         res = httpd_resp_send(req, (const char *)_jpg_buf, _jpg_buf_len);//Send the picture to the web page
47.  
48.         free(_jpg_buf);
49.         _jpg_buf = NULL;
50.         _jpg_buf_len = 0;
51.  
52.         esp_camera_fb_return(fb);
53.       }
54.     }
55.   }
56.   return res;
57. }
58.  
59. //Apply a Blue Filter on an image
60. uint8_t* photoshop_with_rgb888(uint8_t *buf, size_t height, size_t width)
61. {
62.   for (int y = 0; y < height; y++)
63.   {
64.     for (int x = 0; x < width; x++)
65.     {
66.       uint32_t offset = (y * width + x) * 3; //we are in the 'offset' pixel.
67.  
68.       Serial.println("we came by [4]");
69.  
70.       //I go in 1st byte of the pixel
71.       buf[offset] = buf[offset] & 0b11111111; //I didn't change the byte for Blue
72.  
73.       Serial.println("we came by [5]");
74.  
75.       //I go in 2nd byte of the pixel
76.       buf[offset + 1] = buf[offset + 1] & 0b00000000; //I change the byte for green to 0
77.  
78.       Serial.println("we didn't come by [6]");
79.  
80.       //I go in 3rd byte of the pixel
81.       buf[offset + 2] = buf[offset + 2] & 0b00000000; //I change the byte for red to 0
82.     }
83.   }
84.   return buf;
85. }

GeSHi © Codebox Plus Extension

At runtime, the ESP32 returns this to me:

WiFi connected
Camera Stream Ready! Go to: http://192.168.1.45
we came by [2]
Why _rgb888_buf is NULL ????
we came by [3]
we came by [4]
we came by [5]
Guru Meditation Error: Core 0 panic'ed (StoreProhibited). Exception was unhandled.

We see that it entered the test "_rgb888_buf == NULL" while fmt2rgb888 () returned TRUE and we see that it crashed when it tried to change the byte of the green component of the pixel (Normal seen that the buf = NULL).

If anyone has any idea why fmt2rgb888 isn't working, I'm interested!

[ESP_Sprite]

You never actually allocate memory for _rgb888_buf, and fmt2rgb888 won't do that for you from what I can see.

[Star-Vision]

Thank you for the quick reply.
Hmm I added a few lines to allocate a memory of the desired size but it looks like the buf always points to NULL after allocation.

Code: [Select all] [Expand/Collapse]

1. static esp_err_t capture_handler(httpd_req_t *req) {
2.   camera_fb_t *fb = NULL;
3.   esp_err_t res = ESP_OK;
4.   size_t _jpg_buf_len = 0;
5.   uint8_t * _jpg_buf = NULL, * _rgb888_buf = NULL;
6.  
7.  
8.   res = httpd_resp_set_type(req, "image/jpeg");
9.   if (res == ESP_OK)
10.   {
11.     res = httpd_resp_set_hdr(req, "Content-Disposition", "inline; filename=image.jpg");  //capture
12.   }
13.   if (res == ESP_OK) {
14.     ESP_LOGI(TAG, "Take a picture");
15.     fb = esp_camera_fb_get();
16.  
17.     if (!fb)
18.     {
19.       ESP_LOGE(TAG, "Camera capture failed");
20.       httpd_resp_send_500(req);
21.       return ESP_FAIL;
22.     } else {
23.       Serial.println("we came by [2]");
24.  
25.       _rgb888_buf = (uint8_t *) malloc(3*800*600); //I use FRAMESIZE_SVGA [800x600]
26.       if(_rgb888_buf == NULL) Serial.println("Why _rgb888_buf after malloc is NULL ????");
27.  
28.       if(!fmt2rgb888((const uint8_t *) fb->buf,fb->len,fb->format, _rgb888_buf)) Serial.println("fmt2rgb888 function failed");
29.       else {
30.  
31.         esp_camera_fb_return(fb);
32.         if(_rgb888_buf == NULL) Serial.println("Why _rgb888_buf is NULL ????");
33.  
34.         Serial.println("we came by [3]");
35.         _rgb888_buf = photoshop_with_rgb888(_rgb888_buf, 600, 800); //We apply a Blue Filter on image
36.  
37.         if(!fmt2jpg(_rgb888_buf, 800*600*3, 800, 600, PIXFORMAT_RGB888, 15, &_jpg_buf, &_jpg_buf_len)) Serial.println("fmt2jpg function failed");
38.        
39.         free(_rgb888_buf);
40.         _rgb888_buf = NULL;
41.  
42.         res = httpd_resp_send(req, (const char *)_jpg_buf, _jpg_buf_len);//Send the picture to the web page
43.  
44.         free(_jpg_buf);
45.         _jpg_buf = NULL;
46.         _jpg_buf_len = 0;
47.  
48.       }
49.     }
50.   }
51.   return res;
52. }

GeSHi © Codebox Plus Extension

At runtime, the ESP32 returns this to me:

WiFi connected
Camera Stream Ready! Go to: http://192.168.1.45
we came by [2]
Why _rgb888_buf after malloc is NULL ????
Why _rgb888_buf is NULL ????
we came by [3]
we came by [4]
we came by [5]
Guru Meditation Error: Core 0 panic'ed (StoreProhibited). Exception was unhandled.

Is it possible that it is because the memory is full? a JPEG image + an RGB888 image takes too much space?

[Star-Vision]

Since I am using the AI-Thinker ESP32-Cam and have psram memory associated with it. Now that I've educated myself on these intricacies, it's pretty easy to fix that actually. I just have to use the malloc variant (ps_malloc) to save _rgb888_buff in the psram.

I just have to replace line 25 of my "capture_handler" function with:

Code: [Select all] [Expand/Collapse]

1. _rgb888_buf = (uint8_t *) ps_malloc(3*800*600); //I use FRAMESIZE_SVGA [800x600]

GeSHi © Codebox Plus Extension