How can I upload a large size of data by mqtt?

[jackjackdev]

Greetings, I have a project with esp32 and using the Arduino framework to make a sniffer and upload to the cloud.

Browsing I found a sniffer code https://github.com/ESP-EOS/ESP32-WiFi-Sniffer
I plan to capture 1000 packets, stop the sniffer, upload the data and repeat but I have problems uploading the data, I think that the mqtt does not allow to publish much data.

How can I upload the data in a correct and optimal way to complete the cycle?

Code: Select all

    void wifi_sniffer_packet_handler(void* buff, wifi_promiscuous_pkt_type_t type)
    {
     if (type != WIFI_PKT_MGMT)
     return;

    const wifi_promiscuous_pkt_t *ppkt = (wifi_promiscuous_pkt_t *)buff;
    const wifi_ieee80211_packet_t *ipkt = (wifi_ieee80211_packet_t *)ppkt->payload;
    const wifi_ieee80211_mac_hdr_t *hdr = &ipkt->hdr;
    char s[200];
 
    snprintf(s, 200,"PACKET TYPE=%s, CHAN=%02d, RSSI=%02d,"
      " ADDR1=%02x:%02x:%02x:%02x:%02x:%02x,"
      " ADDR2=%02x:%02x:%02x:%02x:%02x:%02x,"
      " ADDR3=%02x:%02x:%02x:%02x:%02x:%02x\n",
      wifi_sniffer_packet_type2str(type),
      ppkt->rx_ctrl.channel,
      ppkt->rx_ctrl.rssi,
      /* ADDR1 */
      hdr->addr1[0],hdr->addr1[1],hdr->addr1[2],
      hdr->addr1[3],hdr->addr1[4],hdr->addr1[5],
      /* ADDR2 */
      hdr->addr2[0],hdr->addr2[1],hdr->addr2[2],
      hdr->addr2[3],hdr->addr2[4],hdr->addr2[5],
      /* ADDR3 */
      hdr->addr3[0],hdr->addr3[1],hdr->addr3[2],
      hdr->addr3[3],hdr->addr3[4],hdr->addr3[5]
    );
    pakagesSniffer = pakagesSniffer + s;
    count++;
}

My Loop:

Code: Select all

    void loop() {
  
    // -------------------------------------------------------------------
    // WIFI
    // -------------------------------------------------------------------
      if(wifi_mode == WIFI_STA){
        wifiLoop();
      }else if (wifi_mode == WIFI_AP){
        wifiAPLoop();
      }
      // -----------------------------------------------------------------
      // MQTT
      // -----------------------------------------------------------------
      if((WiFi.status() == WL_CONNECTED) && (wifi_mode == WIFI_STA)){
        if(mqtt_server != 0){
          // Función para el Loop principla de MQTT
          mqttLoop();
          if(mqttClient.connected() && mqtt_time_send){
            // Funcion para enviar JSON por MQTT
            if(millis() - lastMsg > mqtt_time_interval){
              lastMsg = millis();
              mqtt_publish();
            }
          }
        }
      }

      if(sniffer == false){
        // init sniffer
        wifi_sniffer_init();
      }else{
        if(count >= 5 && mqttClient.connected()){
          count = 0;
          wifi_sniffer_stop();
          mqtt_publish_sniffer();
          mqttTX();
          delay(10000);
        }else{
          snifferLoop();
        }
      }
    }
  

I appreciate any help or suggestion, I'm new to C++.

[tommeyers]

Max packet size in protocol is 256MB.

The broker can limit that what is your broker's limit? Probably a lot less. Check the mqtt broker first.

Tom