Ok, that’s enough of the theory about mapping. Lets get back to what Easimap is doing when it talks to the car.
When we left this subject last we had got to the point where we could see what Easimap was sending and receiving on the CAN bus, but we didn’t know what it meant. We had seen a set of startup packets like this:
# Time ID Data 28 0.265548270 0x0cbe1101 03 04 00 0d 26 40 42 43 29 0.266543253 0x0cbe0111 10 0d e4 00 0d 23 39 41 30 0.266872637 0x0cbe0111 21 34 62 65 35 33 30 00 37 0.326052544 0x0cbe1101 03 04 00 5e 26 40 42 43 38 0.327045620 0x0cbe0111 07 e4 00 5e 2b 07 01 00
Then we get this set of 31 packets that repeat continually but with small changes (probably the “real” data from the ECU):
# Time ID Data 80 0.736160089 0x0cbe1101 10 0a 01 00 00 00 00 12 81 0.736958836 0x0cbe1101 21 66 67 a8 a9 00 00 12 82 0.738230722 0x0cbe0111 05 81 aa aa 16 00 12 66 84 0.749027454 0x0cbe1101 10 09 01 00 00 00 00 1a 85 0.749733313 0x0cbe1101 21 52 5c 5d 00 00 00 1a 86 0.750875498 0x0cbe0111 04 81 84 00 00 00 1a 52 89 0.767924313 0x0cbe1101 10 0a 01 00 00 00 00 e2 90 0.768777632 0x0cbe1101 21 cc cd ce cf 00 00 e2 91 0.769825875 0x0cbe0111 05 81 ff ff ff 07 e2 cc 93 0.780880102 0x0cbe1101 10 23 01 00 00 00 00 f8 95 0.781565869 0x0cbe1101 21 30 31 36 37 44 45 4c 96 0.782301116 0x0cbe1101 22 4d 4e 4f 50 51 5a 5b 97 0.783061863 0x0cbe1101 23 5c 5d 64 6a 6b 7c 7d 98 0.783818702 0x0cbe1101 24 9e 9f a0 a1 d8 d9 da 99 0.784526524 0x0cbe1101 25 db 9f a0 a1 d8 d9 da 100 0.785768596 0x0cbe0111 10 1e 81 5e 39 cc 48 dc 101 0.786095776 0x0cbe0111 21 50 9c 89 1e 85 5e 39 102 0.786437863 0x0cbe0111 22 a8 0d 66 a0 00 b0 4f 103 0.786777913 0x0cbe0111 23 00 00 56 95 80 09 00 104 0.787102630 0x0cbe0111 24 80 00 80 a1 d8 d9 da 106 0.798260523 0x0cbe1101 10 0a 01 00 00 00 00 f9 107 0.799080454 0x0cbe1101 21 ba bb bc bd 00 00 f9 108 0.800070197 0x0cbe0111 05 81 78 1e dc 1e f9 ba 110 0.810033090 0x0cbe1101 10 09 01 00 00 00 00 fa 111 0.810853632 0x0cbe1101 21 64 65 6c 00 00 00 fa 113 0.812042780 0x0cbe0111 04 81 8c 1e 00 00 fa 64 115 0.821498922 0x0cbe1101 10 0e 01 00 00 00 00 fd 116 0.822377593 0x0cbe1101 21 20 24 25 26 40 42 43 117 0.823099192 0x0cbe1101 22 4d 24 25 26 40 42 43 118 0.824343987 0x0cbe0111 10 09 81 40 00 00 01 80 119 0.824665352 0x0cbe0111 21 b0 40 40 26 40 42 43
We’ve also seen that I’d had a diversion looking for bit-shifted pages and addresses which got us nowhere.
Lets get back to Wireshark again…
Wireshark Sub Dissectors
I’d played around with a lot of the CAN bus packet captures we’d taken from Easimap talking to our ECUs and I’d taken a bit of a diversion to fix Wireshark’s OBD-II dissector. I’d also spent time trying to see if Wireshark could dig deeper into the packets with it’s sub-dissectors.
Wireshark works by iterative dissection – meaning that it decodes a packet then passes that packet on to further dissectors, called sub-dissectors, to see if they can further decode the packet. So, Wireshark’s CAN dissector defines further sub-dissectors for these protocols:
- OBD-II (which I’ve talked about before)
- ISO 15765
- CanOpen
- DeviceNet
- J1939
- ISOBUS
- AUTOSAR NM
I thought I had tried to decode our Easimap stream with all of them before, but after coming up blank with my correlator and having read the Car Hacker’s Handbook, I thought I’d have another go. The Car Hacker’s Handbook talks about car manufacturers using proprietary protocols on their CAN bus implementations but it also talks about progress in standardising some of the protocols used.
And… low and behold… when I selected the ISO 15765 sub-dissector to decode our Easimap communications, it was showing some promise. Once I’d issolated just the Easimap traffic then it looked something like this:
[ 2] ISOTP OTHER_REQUEST :04000d [ 4] ISOTP OTHER_RESPONSE :e4000d23394134626535333000 [ 5] ISOTP OTHER_REQUEST :04005e [ 6] ISOTP OTHER_RESPONSE :e4005e2b070100 [ 8] ISOTP REQUEST :0100000000126667a8a9 [ 9] ISOTP RESPONSE :81aaaa1600 [11] ISOTP REQUEST :01000000001a525c5d [12] ISOTP RESPONSE :81840000 [14] ISOTP REQUEST :0100000000e2cccdcecf [15] ISOTP RESPONSE :81ffffff07 [21] ISOTP REQUEST :0100000000f83031363744454c4d4e4f50515a5b5c5d646a6b7c7d9e9fa0a1d8d9dadb [26] ISOTP RESPONSE :814e39094a9c52b886e8844e39a80d70a000b04f00005695800900800080 [28] ISOTP REQUEST :0100000000f9babbbcbd [29] ISOTP RESPONSE :81781edc1e [31] ISOTP REQUEST :0100000000fa64656c [32] ISOTP RESPONSE :81ef1d00 [35] ISOTP REQUEST :0100000000fd202425264042434d [37] ISOTP RESPONSE :814000000180b04040
ISOTP Basics
Ok. Lets step back a bit here. More info on ISOTP (ISO 15765-2) can be found here. It defines a bunch of packet types that can be used to extend the regular CANbus communications from packets of 8 up to packets with a total of 4095 bytes. To do that it steals the first byte or two (depending on the message type) to say what ISOTP packet type this is, and to tell us how long (in bytes) the packet is.
Those first two bytes are described as follows:
For those wanting some help with what that means:
- The top 4 bits (7..4) of byte zero define the message type as follows:
- 0 = “Single” Byte Message
- 1 = “First” byte of a multi-byte message
- 2 = a “Consecutive” byte sent after a “First” byte
- 3 = “Flow” Control message
- The bottom 4 bits of byte zero change their meaning depending on which message type is being sent
- If data is being sent (anything other than a flow control message) then the data starts in either byte 1 or byte 2 and continues for “size” bytes
Lets look at an example of what that means…
ISOTP Example Transmission
From the CAN bus packets shown at the top of this post, we have the first packet viewed as a CAN bus packet as follows:
# Time ID Data 28 0.265548270 0x0cbe1101 03 04 00 0d 26 40 42 43
So the data part of that is:
03 04 00 0d 26 40 42 43
If we look at the top 4 bits (0x0) of the first byte (0x03), we can see that we have a message type of ‘0’. And from the table above message type ‘0’ means we are sending one CAN bus frame and the bottom 4 bits of byte 0 tell us how many data bytes we have in this frame, which is 3.
So, even though there were 8 bytes sent on the CAN bus, only the first byte (0x03) and the subsequent 3 bytes (0x04, 0x00, 0x0d) are of any use. The last 4 bytes (0x26, 0x40, 0x42, 0x43) are junk. In some implementations of ISOTP the devices on the CAN bus would insert “padding” where these junk bytes are here, and we might see them as 0x00 or 0xff or 0xaa, or some other constant but essentially useless data.
In theory Easimap and the ECU don’t need to send these junk or padding bytes. And in my experiments the ECU’s are happy to be sent them or not.
That’s decoding the CAN bus packet and interpreting it as ISOTP.
I also wrote some Python code to decode and display ISOTP packets and it shows us the output of this single CAN bus frame as this:
[ 2] ISOTP OTHER_REQUEST :04000d
Which matches up with our manual decoding of that packet, the data is 0x04, 0x00, 0x0d.
But There’s a Twist – No Flow-Control
As you can see from the “Table of ISTOP Messages and Bit Patterns” above, there’s a flow-control message type defined… but I’ve not shown any flow-control messages so far.
ISOTP is a simple protocol, buts also not just a few lines of code to write the software for. I’d anticipated using a Python compatible library to decode this layer and didn’t really want to write my own.
However, flow-control is an implicit part of the ISOTP protocol and any the libraries I looked at had it intrinsic to the code.
I looked at two ways of doing ISOTP. Both were using the Python-CAN-ISOTP library. And this can either implement ISOTP in user space (meaning it is written in Python, communicates with SocketCAN directly, and is not a kernel module) or it can sit as a thin “wrapper” for the can-isotp kernel module.
I looked at hacking the user space Python version of python-can-isotp but decided that was going to be a lot of work, and there were speed and software architecture benefits to using the can-isotp kernel module.
So, I got to work “adjusting” (some may say hacking) the can-isotp module from Oliver Hartkopp. I changed the code to not wait for flow-control messages when receiving packets (this is almost exactly the same as how can-isotp does the same when in listening mode) and stopped it from sending flow-control when sending packets. All in all it was only some 40 changes to Oliver’s original code.
Normally I’d put a link to Oliver’s GitHub repository here, but that might confuse people. What I’m going to do is link here to my modified version of his repo. That way you’ll not go to all the trouble of installing his code, only to find it won’t work on an MBE ECU. If you want to find his unadulterated code then it can be found from my version of his repo. It might be that Oliver can support this removal of flow-control, but I can fully see how he might not want to do that – you’re supposed to use flow-control with ISOTP and so why would a library supporting the standard include some perversion of that standard. We’ll see how this one plays out and I might update this post to account for where it goes.
But also, for the record… Oliver has done an awesome job with his kernel module. I almost feel embarrassed to have chopped out the flow-control segment, which is a big part of the code. Sorry Oliver!
Back to our ECU’s: We’re not sure why MBE don’t implement flow-control. It’s possibly because it was thought to be too complicated. Or perhaps the ECU’s can respond fast enough not to need the timing / throttling that flow-control offers. Or perhaps it’s both.
One thing we’re going to have to keep in mind though… it might be the case that flow-control is required in an MBE ECU application. It might be that Easimap is doing the backing-off of the transmission to the ECU and if we “hit” the ECU to hard then it might not be able to keep up. We’ll have to keep an eye on this as we develop our applications going forwards. We certainly don’t want to overload the ECU and for it not to be able to do what it’s really meant to… i.e. keeping the engine running properly… that would not go well if we screw that up!
But with flow-control removed from the ISOTP layer we’re off to the races.
It All Starts to Make Sense
As I looked through the communications between Easimap and the car, it was clear that it made a lot of sense when it was decoded as ISOTP. I had no idea what 0x04,0x00,0x0d meant at this time, but at least I had a sense that another layer of the onion was understood.
This made even more sense when I looked at the response from the car…
Easimap was sending 0x04000d and the car was responding with 0xe4000d23394134626535333000 (I’ve just joined all the bytes together here instead of adding the 0x to each one).
That response may seem a bit random. But if we look at it by converting it to ASCII then we get the following:
...#9A4be530.
That also looks a bit random, but if we ignore the dots (which mean the byte is an unprintable ASCII character) then we’re left if 9A4be530… and pulling that apart… clearly 9A4 is the model of the ECU… and by checking back with Easimap… be530 is hexadecimal (0xbe530) and is the serial number of my ECU.
Voila!
By decoding the CAN bus data as ISOTP we’d managed to extract a string (of characters) that was the model of the ECU and the ECU’s serial number. That meant we had some meaningful data and we’re on the right track now.
It also made even more sense when we look at what Easimap does when connected to the car. When Easimap first communicates with the car, it pops up a window in the software showing that it needs to load a particular EC2 file. It must do that because it has found something out about the ECU it’s talking to. It makes perfect sense that as the first communications with the car, it will ask for its model number and serial number so it can go and load the correct EC2 file. It all adds up.
We just need to figure out what all the rest of the communications between Easimap and the car are.
And to do that, it seemed that I needed to know more about what the Ec2 file was telling Easimap when it loads at the start of the communications. So next up we’ll dive back into the EC2 files again…
Leave a Comment