ECU Diagnostics – part 2 : ECU, OBD and CAN

ECU

Caterhams, like all modern cars, have an Engine Control Unit (ECU) – a black box full of electronics, controlled by a microprocessor that manages how the engine runs.  And because it has a microprocessor it means it runs some software to control everything. It also connects to a bunch of sensors, like temperature, pressure, engine speed, air-flow, lambda etc which it then uses to set the engine’s inputs, things like ignition advance/retard, air/fuel mixture and other stuff.

The ECU on a factory, or self-build car, is manufactured by MBE Systems and seems to be supplied to Caterham through SBDMotorsport in the UK. The model used by Caterham is the 9A4 that runs Caterham specific software and mapping. It is also “locked” – meaning that there is no user accessible way of modifying the software or the map. We’ll talk about maps and software in a later post.

What these MBE ECU’s do allow though, is access to the internal sensor information used by the ECU to run the engine. There are many, many parameters used by the software to make an engine run properly and we hoped to be able to see all this information in “real time” – meaning we can see the data change in front of our eyes, hopefully being updated many times per second.

It’s also important to understand that there is both primary data (taken directly from physical sensors) and derived data (secondary, extrapolated or calculated data derived from the primary data). We’ll hopefully be able to see both.

The standard location for the ECU is underneath the battery in the engine bay:

OBD

In order for us to be able to interrogate the ECU and extract all this data, we need some way of connecting a computer into the ECU. Handily, legislation has been in place for a number of years (2001 for petrol and 2004 for diesel) that requires car manufacturers to fit a car with a diagnostic connector. This is known as the OBD port and looks like this:

On modern Caterhams the OBD connector is placed by the ignition switch under the dashboard. With some older cars (some R400’s for instance) it can be found behind the RHS knee trim panel, next to where the battery isolation switch is located. Going even further back, things become a little less simple and a serial lead is required to be connected to the ECU directly. We won’t be covering serial connections in this series of articles.

It’s through this connector that we’ll be getting access to the ECU and hopefully seeing what’s going on. The OBD port on a Caterham is located above the steering column underneath the dash. On my 2017 car it is loosely mounted, dangling on a short length of electrical cables that run to the connector. So you need to be careful when handling it so you don’t pull any of the cables out of the connector.

There are a bunch of standards that define the physical and electrical OBD connector used on cars. The data that’s sent through this connector (I’ll also call it a port) is mostly open to the car manufacturer as to what gets sent and what standard transmission protocols are supported. It’s also very common that the data sent on these ports is proprietary to the particular ECU manufacturer, each one having their own protocols.

For those reading this that aren’t quite so into communications protocols – a protocol is like a language. Computers and tech stuff communicate between each other using protocols. These protocols define what gets sent between two computers and how to interpret it. For instance, there are now over 8000 protocols that define how devices communicate over the internet, and more are being added every day. Check out the IETF RFC pages to learn more about those protocols and standards.

To make things more complicated, manufacturers of different equipment may interpret the protocol definitions (known as “standards”) differently and so end up with dialects of these languages. So the protocols differ from one manufacturer to another just like different dialects of French or English.

Fortunately, there is one protocol that does seem to be reasonably well followed by most car manufacturers and that gives us something to start with. That protocol is often known as OBD-II (OBD 2). We’ll go into more detail about the protocols supported by the MBU 9A4 in a future post.

As we started this investigation, learning about a Caterham’s ECU diagnostic port, we knew only a few things:

• SBD Motorsport supplies a piece of Windows software (Easimap) that can get all the available data points from the car
• OBD-II Scanners can access some data from the car (though not as much as Easimap)
• People have tried to access the diagnostic data on a Caterham before and not been able to find much

So, we started with a general assumption that because the Easimap software could interrogate the car for lots of data, then hopefully we could too. Though one thing on our minds as we started out on this journey, was whether the data exchange (protocol) used between the car and the Easimap software was encrypted – that would cause a whole further level of investigation and might lead to all this effort being fruitless.

Early on in our investigations we wondered if Caterham and MBE were doing something special (in an electrical sense) on the OBD port that was proprietary. However, a quick check under the dash and we can see that Caterhams have only 4 wires connected to the OBD port. There is 0v, 12v, CAN-L and CAN-H. CAN-L and CAN-H make up a twisted pair, see CAN bus below for more on those signals. So, we could see that none of the reserved connections on the OBD port were being used for anything special.

CAN Bus

There’s another twist to the OBD port. It’s one thing to know what it physically looks like, what its electrical connections are and how to connect to it, but its another to be able to understand the data that’s flowing through the connector.

The first layer of that puzzle is to know that two of the twelve electrical connections on an OBD port support a standard called CAN bus. CAN stands for Controller Area Network and is used in industrial and automotive systems. It’s called a network because you can have many CAN bus devices connected in a network. In computing terms a “bus” is something that transfers data from one device in the network to another, we often say that a “bus is shared” between the devices. Modern cars can have more than 50 devices in their CAN bus network; controlling everything from the engine, gearbox, infotainment systems, door locks, windows, lights, brakes and everything in between. My 420R 2017 Caterham has only one CAN bus device, the ECU. It seems that other cars (such as Sigma engined cars also have the rev-counter connected to the CAN bus – but not on my Duratec car).

Most modern cars also have multiple CAN busses: private ones, not directly connected to the under-dash OBD port, for the safety critical components and public ones that are connected to the OBD port and more readily accessible to people like us. These private and public CAN busses are often connected together via a CAN bus bridge. There’s no evidence so far that there is any more than one CAN bus on a Caterham, and therefore no bridges.

CAN bus defines both an electrical connection (wiring, what each wire does, voltages, termination impedances etc) but also a transmission protocol… i.e. what sequences of voltage changes mean (how fast are the changes (bitrate), is it digital or analogue, is it serial or parallel communication, etc.).

So the CAN bus standard defines all this with the following key points for Caterham tinkerers:

• There are two electrical signals making up a single serial data channel. The two signals are differentially coded, meaning you take the two electrical cables as a pair and use the voltage difference between the two to give you a single signal. The two signals are called CAN-L and CAN-H (CAN Low and CAN High). And Soto measure the data in the bus you measure the voltage difference between CAN-L and CAN-H.
• When there’s no data being transferred on the bus the two data lines, CAN-L and CAN-H sit at about 2.5V. It’s one of the main ways you can find a CAN bus on a car… looks for pairs of wires that have a voltage of about 2.5V on them, there isn’t much other wiring at that voltage on a car.
• The two CAN-L and CAN-H signals combine to create a single serial data stream – meaning digital bits are transferred one at a time in succession.
• CANBus can run up to 1Mbps (mega-bits-per-second). On a Caterham the speed is set to 500,000 bps.
• There are three main types of CAN bus signalling. Standard addressing (using 11 bit identifiers (IDs)), Extended addressing (using 29bit IDs) and CAN-FD (or Flexible Data – sending larger amount of data in one go). Caterham’s use Extended addressing and not FD.
• A stream of bits being sent on the bus forms a frame.
• Each frame contains information about:
  • when the frame starts
  • what type of frame it is (standard, extended, error, etc)
  • how much data is in the frame (up to 8 bytes)
  • error checking
  • acknowledgement bits and
  • end bits
• Data on the CAN bus can be either:
  • Broadcast: a device on the bus continuously sends data at pre-determined intervals. The 9A4 ECU does broadcast some information on the bus (I’ve called this protocol “MBE-Broadcast”) and I’ll talk about this in a future post
  • Request/Response: This is how the OBD-II protocol works and how Easimap communicates with the car. A device that want’s data (Easimap on a computer or a sniffer/scanner) “requests” data from the ECU and the ECU “responds” with the data. We’ll talk about the “OBD-II” and “MBE-ISOTP” protocol in later post.

For those interested, the format of a CAN bus packet is shown in the diagram below. Later on we’ll see that all we’re really interested in is the ID, Extended ID and the Data Field. The rest of the protocol is either handled by the CAN bus chipset or by the low level software (in my case that’s the Linux SocketCAN kernel driver).

If that diagram isn’t making sense then perhaps this paragraph is for you: Each of the oblong boxes running across the centre of the diagram represents a single bit transmitted on the CAN bus. Therefore, time runs from left to right, so the leftmost oblong (bit) on the diagram is sent on the CANbus first, followed by the second, etc etc until the last bit represented by the rightmost oblong on the diagram is sent last. All those bits together are a CANbus frame and can contain up to 8 bytes of data on the MBE 9A4 CAN bus.

If you want more info on what all the stuff means in the diagram then I’d recommend heading over to the CANbus wikipedia link, I can’t add much more to what they say there.

There’ll be a post later in this series about me connecting my Logic Analyzer to the CAN bus and hopefully the screenshots there will explain a little more about how data is sent on CAN bus.

Arbitration Fields, CAN bus IDs and Addresses

Having said that, I hope I can clear up one thing that caused me confusion at first. The diagram above and the wikipedia page shows a field called the “Arbitration Field”.

At the physical level (i.e. the 0’s and 1’s on the electrical wires) the arbitration field controls who gets access to the bus. It does this because if two devices start to transmit on the bus at the same time then the one with the lowest numeric around bitration field code “wins” and continues to transmit, while the second device with the higher arbitration code backs off and tries again later. However, at a higher level, such as SocketCAN (see below), these arbitration fields are also like addresses, in computer terms, and different devices will listen for different arbitration fields which are known as ID’s (identifiers).

And for even higher software levels such as ISOTP (see a future post) they are also known as transmit and receive addresses. 

That’s all a little confusing, but all you really need to know is that Arbitration Fields, CAN bus ID’s and the ISOTP Receive and Transmit Addresses are all the same thing.

CAN bus Software

We’ll be talking some more about the CAN bus software in a future post but for the moment it’s worth pointing out that there seem to be a few ways of accessing CAN bus data for writing your own code on a computer.

I’m mainly interested in using Python as a programming language to do my tinkering and in my opinion the only way to do that is with a Unix derived system. And that means either Linux (RedHat, Ubuntu and their derivatives) or MacOS (BSD based).

… and for those operating systems and for Python there are a couple of ways of poking at the CAN bus.

1. SocketCAN: this is the way I went. For me it gives closer access to the bus and allows me to use packet capture software like Wireshark. The CAN bus drivers are included in modern kernels and there is good support for other kernel mode protocols like ISOTP that we’ll need later. The CAN bus kernel module creates an interface that makes the CAN bus look like a network socket and so turns CAN bus frames into packets like an IP packet. These packets are then available to software applications like Wireshark so they can be captured and viewed. Python has a good wrapper for SocketCAN which is simply called python-can.
2. ELM327: Another way to get CAN bus frames into a computer is to use a ELM327, or similar, device. They can connect to phones and computers using Bluetooth or Wifi and have python support through the python-OBD project. They’re good because they mean your computer can be remote from the car. But I’m not a big fan of Wireless for critical connections and I’m hoping that my future OBD dashboard will be critical. They also have to work in “user space” (i.e. not in the kernel) and so can suffer from Linux’s susceptibility to not being as “real time” as some other OSes.

Yet another route for accessing CAN bus is through even smaller computing devices. I’ve spent a lot of time playing with the Arduino and Adafruit Feather devices over the years and I’m thinking that at some point I might switch from the Linux environment to Arduino for this project. These Arduino derivatives have their own programming environment and software drivers for things like CAN bus and they are also more “real time” than OSes like Linux and especially Windows. There are CANbus “shields” for Arduino and even Feather CANbus boards for the Adafruit feather line. But for the moment, Linux is a great way to figure out what’s going on with the car and to get rapid development done.

Example CAN packets

We’ll get into discussing the protocols used on a CAN bus in much more detail in furture posts, but for the moment here’s what three CAN bus frames look like to and from the car. The first two packets in the diagram below are to the car from Easimap and the third is the car’s response back to Easimap. The frames below are how Wireshark (a packet sniffer software tool) displays the frames once captured from the CAN bus, it takes the serial stream of bits and decodes them into what each bit means, displaying them in a more “human readable” form…

Frame 626:
Controller Area Network
...0 1100 1011 1110 0001 0001 0000 0001 = Identifier: 0x0cbe1101
1... .... .... .... .... .... .... .... = Extended Flag: True
.0.. .... .... .... .... .... .... .... = Rem. Tx Req. Flag: False
..0. .... .... .... .... .... .... .... = Error Flag: False
Data: 10 0a 01 00 00 00 00 12

Frame 627:
Controller Area Network
...0 1100 1011 1110 0001 0001 0000 0001 = Identifier: 0x0cbe1101
1... .... .... .... .... .... .... .... = Extended Flag: True
.0.. .... .... .... .... .... .... .... = Rem. Tx Req. Flag: False
..0. .... .... .... .... .... .... .... = Error Flag: False
Data: 21 66 67 a8 a9 00 00 12

Frame 628:
Controller Area Network
...0 1100 1011 1110 0000 0001 0001 0001 = Identifier: 0x0cbe0111
1... .... .... .... .... .... .... .... = Extended Flag: True
.0.. .... .... .... .... .... .... .... = Rem. Tx Req. Flag: False
..0. .... .... .... .... .... .... .... = Error Flag: False
Data: 05 81 aa aa 16 00 12 66

CANbus packets will be printed differently depending on which software you use to look at them. In this particular example from Wireshark:

• 0x0cbe1101 and 0x0cbe0111 are CAN bus identifiers (ID’s) or Arbitration fields written in hexadecimal (the 0x means hexadecimal). From the discussion above about arbitration, the device sending the ID of 0x0cbe0111 would take priority over the device sending 0x0cbe1101 , because 0xcbe0111 is less than 0x0cbe1101.
• The first two frames with an ID of 0x0cbe1101 are sent from Easimap and the last frame with an ID of 0x0cbe0111 is the response from the car’s ECU back to Easimap
• There are various flags shown, extended flag (meaning 29bit IDs), Remote Transmission Request Flag and the error flag.

• The hexadecimal numbers on the Data line are the data, ie the stuff we’re really going to be interested in like engine speed and fuel mixture. 

Well that’s about it for the basics of ECU’s, OBD and CAN bus. We’ll build on all of this in future posts.