ECU Diagnostics – part 5 : The Correlator Dead-end

Hopefully this is a quick post…

The story so far is that:

1. We’ve got a test bench made of a Raspberry Pi, CAN bus decoder board and an OBD Y-cable
2. The Raspberry Pi can “sniff” all the packets sent between the car’s ECU and something that can talk to the car, like Easimap or an OBD scanner
3. We can decode the CAN bus packets captured in the last bullet point and view them using software such as Wireshark
4. We knew that the MBE ECU sends continuous data on the CAN bus (I call the protocol “MBE-Broadcast”) but there’s not much real data in those packets
5. We know that the MBE ECU can talk the standards based protocol of OBD-II, but again there’s not much more than simple data there either
6. SBD provides the Easimap software and USB to CAN bus adapter (MBE 985) that can get all the data from the ECU, but we don’t know how it does that

Easimap EC2 Files

The Easimap Windows software app can get all the data it needs from the ECU. The app is designed to work with all of MBE’s ECU’s… not just the Caterham 9A4 but all the other ECU’s they make and covering all (probably) of their past ECUs.

The flexibility that these diverse ECU requirement brings, is achieved by having configuration files that Easimap loads once it’s decided which ECU it is talking to.

And those configuration files are loaded from a standard location on the Windows c:drive after the software is installed, and have an .ec2 extension. You can find the ec2 files in the Projects directory that Easimap installs.

When Easimap sees the Caterham 9A4 ECU it loads an EC2 file called:

9A4be52a.ec2

This file is not for the faint hearted. It’s a text file, but over 25,000 lines long. There are multiple sections to the file and it all cross references itself. I’ll talk about EC2 files and how I pulled out what I needed in a future post.

But for the moment, and at this stage of the testing, I was interested in the way the basic data definitions worked in the ec2 files. This might tell us a bit more about Easimap was talking to the ECU.

A Simple EC2 Data Variable Example

For instance the Engine RPM data variable (RT_ENGINESPEED) is defined in the ec2 file like this:

[RT_ENGINESPEED]<br>Number of Dimensions = 0<br>Page = F8<br>Address = 237C<br>Bytes per Cell = 2<br>0 = SCALE_ENGINESPEED

The files are formatted like the Windows config file format, but it’s multi-layered and causes standard configuration file parsers to barf, but we can see basically what’s going on.

I think the RT in RT_ENGINESPEED means “real time”. But that’s just a guess.

Here’s what I think the rest means:

• Number of Dimensions: This defines how many axies the variable has. Zero dimensions means its a single data item variable, containing a single data item. If it was a 1 dimension then it would be a simple one-dimensional array – containing a list of numbers. 2 dimensions is a 2D array with two axies. At least that’s my thinking at the moment.
• Page: In computer speak we often talk about pages of memory. This “Page = F8” refers to a “page number” in hexadecimal of 0xf8.  In the old days CPU’s couldn’t deal with the large amounts of memory that were required of them and they had to load pages of memory one at at time into the main processor memory (cache). These days that concept is not quite as prevalent, even though the MMU (Memory Management Units) of modern CPU’s still have throw backs to this system. Anyway, a page is a block of memory and each “page number” refers to that block. 
• Address: This is also a reference to a computer memory location. It is the “index” into the page in hexadecimal – in this case 0x237c. We call it an index because it indexes the memory location in the page, like the index of a book it points to a memory location in the block we accessed with the page number.
• Bytes per cell: Is telling us the data variable can be a maximum of 2 bytes, which is 16 bits and can therefore take a value from 0 to 65,535. The ec2 file also defines  1, 3 and 4 byte variables elsewhere in the file.
• 0 = SCALE_ENGINESPEED: This line is a bit more complex and I’ll leave its discussion to another post.

Right… so the reason I’m getting into this here is that the page and address parameters in this variable definition are what intrigued me. If Easimap was using the ec2 files as a definition of how it was going to talk to the car then the page and address parameters seemed to be very important.

Hold that thought. Now back to the packets on the CAN bus.

Some CAN bus Easimap data

So, we knew nothing more about how Easimap was talking to the ECU on the CAN bus. The first packet captures we took from the car just showed us data like the following:

#  Time        Protocol    ID         Data<br>1  0.000000000 CAN 32 XTD: 0x0cbe1101 03 04 00 0d ef fb ff f7<br>2  0.000990262 CAN 32 XTD: 0x0cbe0111 10 0d e4 00 0d 23 39 41<br>3  0.001320498 CAN 32 XTD: 0x0cbe0111 21 34 62 65 35 33 30 00<br>4  0.058836571 CAN 32 XTD: 0x0cbe1101 03 04 00 5e ef fb ff f7<br>5  0.059825611 CAN 32 XTD: 0x0cbe0111 07 e4 00 5e 2b 07 01 00<br>6  0.506864643 CAN 32 XTD: 0x0cbe1101 10 0a 01 00 00 00 00 12<br>7  0.507713945 CAN 32 XTD: 0x0cbe1101 21 66 67 a8 a9 00 00 12<br>8  0.508917926 CAN 32 XTD: 0x0cbe0111 05 81 aa aa 16 00 12 66<br>9  0.517519386 CAN 32 XTD: 0x0cbe1101 10 09 01 00 00 00 00 1a<br>10 0.518228338 CAN 32 XTD: 0x0cbe1101 21 52 5c 5d 00 00 00 1a<br>11 0.519227915 CAN 32 XTD: 0x0cbe0111 04 81 84 00 00 00 1a 52<br>12 0.530752292 CAN 32 XTD: 0x0cbe1101 10 0a 01 00 00 00 00 e2<br>13 0.531488040 CAN 32 XTD: 0x0cbe1101 21 cc cd ce cf 00 00 e2<br>14 0.532530228 CAN 32 XTD: 0x0cbe0111 05 81 ff ff ff 07 e2 cc<br>15 0.545266012 CAN 32 XTD: 0x0cbe1101 10 23 01 00 00 00 00 f8<br>16 0.546032834 CAN 32 XTD: 0x0cbe1101 21 30 31 36 37 44 45 4c<br>17 0.546786859 CAN 32 XTD: 0x0cbe1101 22 4d 4e 4f 50 51 5a 5b<br>18 0.547536866 CAN 32 XTD: 0x0cbe1101 23 5c 5d 64 6a 6b 7c 7d<br>19 0.548296910 CAN 32 XTD: 0x0cbe1101 24 9e 9f a0 a1 d8 d9 da<br>20 0.549005102 CAN 32 XTD: 0x0cbe1101 25 db 9f a0 a1 d8 d9 da

Remember it’s the last eight columns of two Hexadecimal digits that, the data, that we’re really interested in.

And it was at this point that we figured out that Easimap was using a request/response protocol. It was sending data to the ECU with an ID of 0x0cbe1101 and was getting data back from the ECU with an ID of 0x0cbe0111. You can see the changing ID’s in the packets above, if the line has an 0x0cbe1101 in it then it means Easimap is sending that data to the ECU and vice versa for 0x0cbe0111.

Now back to the pages and the address…

It seemed to me, that if Easimap was so configurable and the ec2 files defined the configuration, then the page and addresses were probably being sent to the ECU in some form. And that meant that they would probably show up in the “data” we were seeing on the CAN bus.

In the packets just above, I knew that Easimap was asking the car for Engine RPM (RT_ENGINESPEED) and that had a page of 0xf8 and an address of 0x237c from the ec2 file.

But I couldn’t see the two bytes of 0x237c in any of the data to or from the car.

Bit Shifting

It then seemed that if we couldn’t see the 0x237c in the data stream, and we were sure that Easimap must be sending it somehow to the car, then perhaps the address was being sent with a “bit shift”.

The CAN bus protocol itself is a very non-byte aligned protocol. What I mean by that is that the CAN bus protocol uses transmissions of things like the ID, CRC, start and stop bits that do not align to byte boundaries – they don’t align to the zero’th bit of 8 bit (byte) chunks.

So, perhaps the data being sent to and from the ECU was not “byte aligned” either.

I needed a correlator.

Correlator

If the address 0x237c was being sent to the ECU non-byte-aligned then I needed to look for its “bit pattern” in the data at non-byte-aligned positions.

I was therefore going to have to look for 0x237c in binary, which is

0010001101111100 (binary)

And I was going to have to look for this bit pattern in the data. For example the first line of data in the capture above is:

03 04 00 0d ef fb ff f7 (hexadecimal)

Which is this in binary:

0000 0011 0000 0100 0000 0000 0000 1101 1110 1111 (binary)

 And so I would have to look for the first binary pattern (0010001101111100) within the second pattern, and then do that through the whole of the packet capture (which was about 3000 packets in total).

I needed a program… and in digital signal processing (DSP) terms this program had to correlate the address with the data.

So, I wrote this program here. And here’s some example output from it…

Bitwise search for: 0x237c
Row 1, Data:0x(ff95000000000000), 0b(1111111110010101000000000000000000000000000000000000000000000000)
Row 2, Data:0x(0495ff2800000000), 0b(0000010010010101111111110010100000000000000000000000000000000000)
Row 3, Data:0x(0327001d001d0000), 0b(0000001100100111000000000001110100000000000111010000000000000000)
Row 4, Data:0x(02aaaa008fd60000), 0b(0000001010101010101010100000000010001111110101100000000000000000)
Row 5, Data:0x(0152000039009549), 0b(0000000101010010000000000000000000111001000000001001010101001001)
Row 6, Data:0x(ff00000000000000), 0b(1111111100000000000000000000000000000000000000000000000000000000)
Row 7, Data:0x(ff00000000000000), 0b(1111111100000000000000000000000000000000000000000000000000000000)
Row 8, Data:0x(ff49000000000000), 0b(1111111101001001000000000000000000000000000000000000000000000000)
Row 9, Data:0x(ff95000000000000), 0b(1111111110010101000000000000000000000000000000000000000000000000)

Unfortunately I didn’t find anything – a dead-end. I couldn’t find any of the addresses from the ec2 file within the data I was capturing from Easimap talking to the car.

If my program had found something then it would have printed something out like this:

Bitwise search for: 0xff28
Row 1, Data:0x(ff95000000000000), 0b(1111111110010101000000000000000000000000000000000000000000000000)
Row 2, Data:0x(0495ff2800000000), 0b(0000010010010101111111110010100000000000000000000000000000000000)
Found something:
Row/Bit: 2/32, DW: 0x(ff28), DW: 0b(1111111100101000), Data: 0x(0495ff2800000000), Data: 0b(0000010010010101111111110010100000000000000000000000000000000000), SHIFTED: ff28

But it didn’t find anything other than the test data I had plugging into it, like that 0xff28 test above.

If I write the output of that test out a bit differently, you’ll see how the bits line up, we were looking for the first line, in the second line and we found it at bit position 16…

Found:
................1111111100101000................................
In:
0000010010010101111111110010100000000000000000000000000000000000

I’ve been involved in a number of attempts over the years to decode different communications protocols and these dead-ends happen a lot.

Ok, so we need to think about another way of decoding the protocol.