ESP32CAN missing messages.

[acln99]

Hi - I am a complete newbie to ESP32. Please HELP !
My task has been to capture on an Chrysler Automotive Can Bus.
I am able to monitor messages on the bus. I use a diagnostic tool to initiate a "dump" of data and the ESP32 to monitor the packets.

Problem is - some of the message packages are missing :

Can anyone help as to why I am missing messages ?? Thank You !!!

I am using Arduino ver1.8.10
I have ESP-Wroom-32 board.
SND65HVD230 transceiver.
I choose arduino "board" - DOIT ESP32 DEVKIT V1.

ESP32 is power from USB 2.0 port connected to Notebook
Can bus speed set at 500kbps and serial monitor set at 500,000 baud


The Arduino code is as follows. I used the sample "esp32can-basic.ino" , added a blink to show me that the program loads and also removed any "send" lines.

THANK YOU for Reading this !




#include <Arduino.h>
#include <ESP32CAN.h>
#include <CAN_config.h>

CAN_device_t CAN_cfg; // CAN Config
unsigned long previousMillis = 0; // will store last time a CAN Message was send
const int interval = 1000; // interval at which send CAN Messages (milliseconds)
const int rx_queue_size = 10; // Receive Queue size
int ledPin = 2;


void setup() {
pinMode(ledPin, OUTPUT);
digitalWrite(ledPin, HIGH);
delay(3000);
digitalWrite(ledPin, LOW);
delay(1000);


Serial.begin(500000);
Serial.println("");
Serial.println("Basic Demo - ESP32-Arduino-CAN ACG 5");
CAN_cfg.speed = CAN_SPEED_500KBPS;
CAN_cfg.tx_pin_id = GPIO_NUM_5;
CAN_cfg.rx_pin_id = GPIO_NUM_4;
CAN_cfg.rx_queue = xQueueCreate(rx_queue_size, sizeof(CAN_frame_t));
// Init CAN Module
ESP32Can.CANInit();
}

void loop() {

CAN_frame_t rx_frame;

unsigned long currentMillis = millis();

// Receive next CAN frame from queue
if (xQueueReceive(CAN_cfg.rx_queue, &rx_frame, 3 * portTICK_PERIOD_MS) == pdTRUE) {

if (rx_frame.FIR.B.FF == CAN_frame_std) {
printf("New standard frame");
}
else {
printf("New extended frame");
}

if (rx_frame.FIR.B.RTR == CAN_RTR) {
printf(" RTR from 0x%08X, DLC %d\r\n", rx_frame.MsgID, rx_frame.FIR.B.DLC);
}
else {
printf(" from 0x%08X, DLC %d, Data ", rx_frame.MsgID, rx_frame.FIR.B.DLC);
for (int i = 0; i < rx_frame.FIR.B.DLC; i++) {
printf("0x%02X ", rx_frame.data.u8);
}
printf("\n");
}
}
// // Send CAN Message
// if (currentMillis - previousMillis >= interval) {
// previousMillis = currentMillis;
// CAN_frame_t tx_frame;
// tx_frame.FIR.B.FF = CAN_frame_std;
// tx_frame.MsgID = 0x001;
// tx_frame.FIR.B.DLC = 8;
// tx_frame.data.u8[0] = 0x00;
// tx_frame.data.u8[1] = 0x01;
// tx_frame.data.u8[2] = 0x02;
// tx_frame.data.u8[3] = 0x03;
// tx_frame.data.u8[4] = 0x04;
// tx_frame.data.u8[5] = 0x05;
// tx_frame.data.u8[6] = 0x06;
// tx_frame.data.u8[7] = 0x07;
// ESP32Can.CANWriteFrame(&tx_frame);
// }
}



SAMPLE of the data collected follows: I have manually entered the decoded data beside each message. blank lines I have inserted to show that lines are missing and the data expected. The dump is over 100,000 messages.

7E8 8 23 0 0 0 0 0 0 0
7E8 8 24 0 0 1 6 0 0 0
7E8 8 26 43 6F 70 79 72 69 67 Copyrig
7E8 8 27 68 74 20 32 30 30 37 ht 2007
Daimler
7E8 8 29 20 43 68 72 79 73 6C Chrysl
er LLC.
7E8 8 2B 20 41 6C 6C 20 52 69 All Ri
ghts Re
7E8 8 2D 73 65 72 76 65 64 2E served.

7E8 8 2F 6F 66 74 77 61 72 65 oftware

7E8 8 22 65 6E 74 69 61 6C 20 ential

7E8 8 24 70 72 69 65 74 61 72 prietar

7E8 8 26 72 74 79 20 6F 66 20 rty of
Chrysle
7E8 8 28 72 20 4C 4C 43 2E 20 r LLC.

7E8 8 2A 68 74 20 6E 6F 74 69 ht noti
7E8 8 2C 72 65 63 61 75 74 69 recauti
7E8 8 2E 6E 64 20 64 6F 65 73 nd does
7E8 8 20 70 6C 79 20 70 75 62 ply pub
7E8 8 22 6E 2E 0 0 1C B 21
7E8 8 24 0 0 1C B 21 5 2
7E8 8 26 1C B 22 5 2 0 1C
7E8 8 27 B D8 2 0 0 1C B
7E8 8 29 1 0 0 1C B 14 5

As you can see , several lines are missing.

[ESP_Sprite]

Are you sure you can output on your serial port as fast as the CAN messages arrive?

[acln99]

After trying for 2 months first with Ardunio Uno and Can Shield, ( same problem) I'm not sure of anything....
But -

I have tried Ardunio Serial Monitor set at 1,000,000 and receive garbled data so for sure 1,000,000 is too fast for my PC.
At 500,000 I do not get any garbled data. As in the sample - very clean.
Serial Print Setup message from the program does appear properly on the monitor at 500,000

Also have tried Terminal programs with the same result.

Since I used the same PC and USB port for Arduino and ESP32 and receive the same result, it does make me suspect that it is the serial connection. It is a large "dump" - about 4 minutes of receiving data.

Is there a better way to capture the data rather than serial monitor ? Before I switched to ESP32 , I purchased a Arduino Can bus shield with microSD card but could never get card to initialize.

Any help is greatly appreciated !!

[CollinK]

The answer to that last question is, of course, no. You can't set the serial port speed to 500000 bits per second and hope to output all the CAN frames coming in on a 500k CAN bus. The reason is fairly straight forward. The CAN bus is sending data binary encoded. It's as compact as it can be, all things considered. The serial traffic is much more verbose in comparison. Let's take an example line:

7E8 8 26 43 6F 70 79 72 69 67 Copyrig

That is 37 bytes long and the line terminator is either 1 or 2 bytes so let's be conservative and say it's 1 byte. That's 38 bytes for that one message. Expect a CAN message to have 11 bits ID, about 3 bits of other stuff, 3 bits for DLC, some other overhead. Just to be extra lenient, let's say a CAN frame is about 5 extra bytes on top of the data bytes. So 5+8 = 13 bytes on the bus to receive the frame. The OP is exploding these 13 bytes to 38 bytes of output. That's nearly three times as many output bytes as input. Thus, if the bus load dares approach 33% on CAN then the serial port will saturate while the CAN bus will happily keep receiving more and more data that the serial port cannot display. Eventually this will fill up the transmit buffer on the serial port. When this happens CAN frames will queue into the buffer. At a certain point there will be no more space in the buffer to store a new frame since the serial port just keeps getting further and further behind. This will cause frames to start being dropped. Eventually it reaches equilibrium where enough frames are dropped to even up the load.

Long story made short: Crank that serial speed. The ESP32 can easily do 1M baud, and even really should be able to do 2 or 3M without hardware flow control. Even if you bork a few bytes every so often from the serial speed you'll still be better off than having too low of a speed and filling up the serial buffer. If you set the serial speed to 3M and it still does the same thing then you know I'm a liar and not to be trusted and it was something else going on. Hey, it happens. But, the serial speed still is way too slow for the given use case.

[ESP_Sprite]

Alternatively: use a faster protocol, like a TCP connection over WiFi. Make sure to buffer enough data in that case, though.

[acln99]

Thank you both !! I'm testing with faster serial speed and results are better. Only loosing every 5th line but also sending a lot of extra data that is not needed. Once I get the serial data down to a minimum, I hope I will get all the messages.
Thanks again !!

I'm over 65 and have never worked with these "little" processors. They are amazing !!! ( I remember terminal programs at 1200 baud )

[acln99]

CollinK - Reading your answer a few times... is it possible to send the whole Can message in binary to the PC and decode it later.

[CollinK]

CollinK - Reading your answer a few times... is it possible to send the whole Can message in binary to the PC and decode it later.

Yeah, sure, I do it all the time. I have whole complicated programs that run on the embedded and PC side (in this case ESP32RET on the ESP32 and SavvyCAN on the PC) but basically you can just boil it down to this: Use the write function to blast things out in binary.
For instance, serial.write(0x10); sends hex value 0x10 (16) over the wire. You can also write out buffers that way: serial.write(buffer, 16) writes out 16 bytes from the buffer named buffer. You can use that functionality to directly write out a frame.

CAN_frame_t tx_frame;
<read the frame>
serial.write(&tx_frame, sizeof(CAN_frame_t));

That will directly blast the actual bits of the frame out the serial port without any modification. This will end up slightly smaller than the actual # of bits that were sent over the wire on CAN for various reasons that aren't super relevant here. Then you problem is how do you interpet the frames on the PC side? Well, almost certainly with custom code.

[acln99]

CollinK - Thank You! I'm going to give that a try .