RomRaider

All of the 16-bit guidance in the following posts is from Elevenpoint7five. Thank him for his help! Blame me for any mistakes!

Scope:

This document is intended for people whose ECUs already have RomRaider/EcuFlash definitions. If you want to understand in more detail how your ECU works, and you've read everything available on the forums, disassembling the ROM is the next step. It's tedious, and it takes a lot of time, but it can give you insights into how and why the ECU does what it does. Better yet, reverse engineering the ROM will allow you to create table definitions that you can use to modify new aspects of your tune. All of the existing table definitions were created this way, and there are lots more tables that have not yet been figured out and defined.

If your ROM has not yet been defined, I suggest choosing a ROM that has, and exploring that one first. The existing definitions will give you a big head start in understanding what's inside that ROM. Finding tables in an undocumented ROM will be easier after you gain some familiarity with a documented ROM.

Stuff you'll need:

1) A copy of the ROM you wish to reverse-engineer. You can acquire this with EcuFlash and a Tactrix OpenPort that is compatible with the vehicle in question.

2) An understanding of how a CPU works. Concepts like instructions, pointers, flags, memory, registers, stacks, addressing, branching, hexadecimal notation and floating-point representation should be familiar to you already. I'm not really sure how best to acquire this knowledge today. I started 25 years ago with an 8-bit home computer and a reference manual. It is routinely taught in college and perhaps advanced high school courses. There are books, however I don't have any recommendations.

I'll cover some fundamentals below, but it is intended mostly to refresh the memories of people who already have some familiarity with the concepts. If you have no formal eduction in this field I think that you will find it difficult to make progress based on this how-to alone - in that case I strongly suggest getting a book and/or taking a class.

3) A reference to the instruction set for the CPU for that ROM was created for. First you'll need to know what CPU your ECU uses. If your ROM has been documented, just open the ROM in EcuFlash and look in the "Rom Info" section. Then put "renesas 7058" into your favorite search engine (assuming your ROM uses a 7508 of course), and look for documentation.

http://www.renesas.com/products/mpumcu/ ... tation.jsp

If working with a ROM that has not been documented, you might be able to guess based on what similar ECUs used, but you might just have to open up the ECU and look for a part number.

4) You'll need time. Lots and lots of time. Plan to spend a couple weeks of evenings to get familiar with the basics.

5) You'll want my XmlToIdc utility. This is a tool for importing the information in the existing definitions into IDA - it will give you a significant head start. You can acquire it from this thread: viewtopic.php?f=32&t=6234

6) A copy of IDA Pro. You can acquire this in the same way that you would acquire any other software that sells for about $1000 per license. By that I mean you should bust out your credit card and call the distributor, of course.

http://www.hex-rays.com/idapro/idaorder.htm

On a related note, I am trying to get a reduced rate for us not-for-profit hobbyists. I will start a separate thread about that if I have any luck.

7) You should have a quest. Why are you looking into your ROM? This is tedious stuff, but having a purpose will help keep you focused and motivated. I started out with an interest in LC and FFS, so my first quest was to find the rev limiter code, and figure out how it works. If you are interested in a speed-density hack, you'll want to find the MAF code, and figure that out (hint: start by looking for references to the MAF scaling table).

What's in a ROM?

The ROM contains two types of information: code and data. The code is the set of instructions that the CPU executes to read the sensors and manipulate the actuators that keep the engine running. The data is the set of numbers that we think of as 'the tune.' I will also refer to the data as 'constants' below since their values don't change while the car is running (though it can be changed in RomRaider or EcuFlash). You'll also be analyzing RAM, which contains the data that changes while the engine is running. I'll refer to this data as 'variables' below since the values vary while the car is running (think of MAF and RPM, for example).

All of this stuff - code, constants, and variables, in stored in memory at various addresses. IDA will present the ROM and RAM to as one long series of memory locations and the information in memory at those addresses; in ROM the instructions and constant values are shown, and in RAM the values are not shown, since the values are only known when the car is actually running.

IDA will make an effort to determine which parts of the ROM contain code, and it will perform a basic analysis of the code - displaying instructions in mnemonic form, linking load and store instructions to the addresses they're loading from and storing to, and so on. But keep in mind that IDA's analyzer is not perfect - there will be times when you need to re-define sections of the ROM as code rather than data, and vice-versa.

Frequently used commands in IDA

1) Rename a memory location (press N). IDA will generate names like "sub_1234" when it finds instructions that appear to be a subroutine at address 1234, or unk_FFFF2345 when it finds a reference to memory location FFFF2345. If you suspect that FFFF2345 actually holds the engine coolant temperature, you should rename it CoolantTemperature so that you can recognize references to it in the code. I would actually call it ProbablyCoolantTemperature at first, then NsfwCoolantTemperature when I am confident that's what it really is.

I think it's helpful to distinguish things that I discovered (or think I have discovered) from names for things that Merchgod discovered. I have more confidence in his work than in my own.

2) Rename a function (press ALT+P). Same as above, but with a different keystroke, and function names are stored in a separate list from other memory location names.

3) Follow cross-references (press X). This is extremely useful. If you'e found a memory location that holds something interesting, you'll probably want to hae a look at all of the code that refers to that memory location. Pressing X will show you that list. As a nice bonus, IDA will remember which item in the list you clicked on last, so you can bring up the list, move to the next item in the list, and jump to it, working your way down the list.

4) Jump to an arbitrary address (press G). Sometimes you don't have a reference to where you want to go. Press G and enter the address directly, and you'll go there.

5) Mark as code or data (press C or D). As noted above, IDA's analysis isn't perfect. Pressing D repeatedly will cause IDA to interpret the current address as a single byte, two bytes (also known as a "word") or 4 bytes (also known as a "long word"). Under the "Options" menu, in "Setup data types" you should check "4 - Float" to get floating-point representation into the set of data type options.

How do you open a ROM? (32-bit)

1) Get the DSchultz' processor register listing from this thread:
viewtopic.php?f=25&t=5825&start=0
Copy it to IDA's cfg directory.

2) Use the "File, Open" menu just as you would with any other Windows application.

3) Choose your processor type.

4) IDA will prompt you to create a RAM segment as you open the ROM. Specify the RAM start address and length. The specifics are below, courtesy of DSchultz, but set the length to whatever it takes to make RAM extend to 0xFFFFFFFF. Special-purpose registers (for IO, etc) are located at the top end.

For 7055, Start is 0xFFFF6000, physical RAM length is 0x7FFF, but enter 0x9FFF.
For 7058, Start is 0xFFFF0000, physical RAM length is 0xBFFF, but enter 0xFFFF.
Be sure to check the box to make the RAM segment.

There's a way to create one later as well. After the ROM is opened, click "Edit, Segments, Create Segment" in the menu. For a 7058 CPU, fill in the box as follows:

Segment name: RAM
Start address: FFFF0000
Length: BFFF (End: FFFFBFFF)
Base: 0
Class: (empty)
32-bit segment

5) If you're working with an ECU for which definitions already exist, use XmlToIdc to tell IDA the names for the tables and extended parameters. See the XmlToIdc thread for details. You can't do standard parameters yet, but you will in a few minutes - keep reading.

6) Unfold the startup/reset code. The first 4 bytes of ROM are a pointer to the code that runs when the ECU is powered up. Jump to that location (AAC in the ROM I'm looking at) and press 'c' to mark it as 'code.' You'll see a small portion of the ROM disassembled.

7) Make an analysis pass. From the "Options" menu, click "General." Under "Kernel Options 1" select "Make final analysis pass." Click "Reanalyze Program." IDA should spend a minute analyzing the ROM, marking some sections as code and others as data. This is very helpful, but again it is not perfect - there will be sections of code erroneously marked as data, and vice-versa.

8 ) Find the SSM base, using DSchultz's method.

The "SSM base" is the base (starting address) of an array of function pointers that handle the standard SSM queries. Most of those functions just load a value from some memory location, do a conversion, and send the value back to the SSM device (i.e. the data logger). With the knowledge of what the standard queries are (from logger.xml and XmlToIdc.exe) you can determine what those memory locations are. More about this in the next post...

Finding the SSM base is a good first exercise in reverse engineering. I refer you to DSchultz's thread for a detailed description:

viewtopic.php?f=25&t=6295

How do you open a ROM? (16-bit)

>>> See this new and improved 16bit post for opening ROMs (dschultz)

1) You will need to convert your 160kb ROM to 192kb. To do this you will need a hex editor. EleventPoint7five recommends TinyHexer. Open the ROM in the hex editor of your choice and navigate to 0x20000. In TinyHexer you can hit ‘ctrl + g’ to enter an address. You then need to insert a RAM space(basically all 0’s) that is 0x08000 long. Attached is a file you can add in with TinyHexer to do this for you. You click ‘File’ the ‘Insert File’ then save your newly converted ROM.

2)Open your ROM with IDA.

3)Change the processor type to Motorola Series: 6816. Click Kernel Options 1 and make sure ‘Make Final Analysis Pass’ is checked. It should be by default, but just make sure. Hit ok.

4)Once the little bar at the top has stopped moving and changing colors, your ROM is loaded and analyzed!

5)Hit ‘g’, type 0x220 into the input field and hit ok, hit ‘c’ and wait a minute. Now your ROM is “unfolded” or “opened up”

0x220 is the entry point for every 16bit ROM known so far. It will open up a very large portion of the ROM, enough for you to edit/define almost anything you’ll ever want to. There are a few other smaller entry points that can be found in the processor data sheets. From here you can use the same commands with IDA as the 32bit ROMs.

The easy part is done.

At this point, you should have a ton of code and data in front of you, with some clues about what some of it means. The rest is up to you.

Additional resources

IDA Pro reference material, and also some data sheets for Evo ECUs:
http://www.evoscan.com/technical-vehicl ... ssassembly
(thanks to dschultz for this link)

Here are some tips on making sense of the rest of the code you have just disassembled.

It is useful to examine each of the SSM handlers to find what memory locations they look at. For example, SsmGetExhaustGasTemperature looks at one memory location, calls a subroutine to convert its value, and returns. So, that memory location is probably the exhaust gas temperature. Follow XREFs to that location to see where it gets used.

Pick some tables at random (XmlToIdc creates names that start with "Table") and follow XREFs to the code that uses those tables. You'll probably notice that the code sets fr4 and fr5, then jsr's to a subroutine. Rename those subroutines to Pull2d or Pull3d, as appropriate. Now you'll be able to spot 2d and 3d table lookups anywhere in the code.

When you find a call to Pull2d or Pull3d, look upwards from the jsr instruction and see if you can figure out what's in fr4 (and, for 3d tables, fr5). Sometimes you can find that they come from memory locations that are already known, so you can tell what the table axes are. Also look at what happens to fr0 after the table pull. The more variables you define, the more you'll be able to figure out about the various Pull operations, since you'll know what the inputs and outputs are.

When you find a memory location in RAM and you have a good theory about what it is used for, add it to your logger definition as an extended parameter. Then go for a drive, and see if the logger shows what you expected to see.

When you find a memory location in ROM and you think you know what's in it, add it to your ECU definition, open the ROM, and see if it shows what you expected to find. I prefer EcuFlash for this - the ROM definition file format is easier to work with, and EcuFlash can open a new ROM much more quickly than RomRaider. Also note that EcuFlash has UI for tweaking table definitions.

Quirks of the SHx instruction set.

1. Delayed Branching

For the most part, the SHx chips use a pretty straightforward RISC model. There are instructions to load values from memory, store values in memory, operate on those values, compare and branch, and so on.

But here's something that will drive you nuts for a while:



Why is there a no-op right after the bf/s instruction?
What's the point of storing the value of r0 after that last bra instruction?

Welcome to the magic of the 'delayed branch.' When you see a branch instruction, keep in mind that the CPU executes the instruction after the branch before actually taking the branch. You will see this with function calls as well (jsr instructions). Even after reading this in the instruction listing, it still took me a couple nights of occasional confusion before I got into the habit of looking at the instructions after the branches.

2. ZK = 0 (16-bit only)

With these ROMs, ZK is always 2. So, if you see an instruction like:

ldd 0F0h, Z

then:

Z = ZK & IZ
IZ = 0x00F0 (bits 0 - 15)
ZK = 0x2 (bits 16-20)

So, Z = 0x200F0 (20-bit RAM reference)

YK is not always 0x2, but where Y is referenced as it is relates to map offsets it is. You can verify this by going back far enough and seeing where yk is changed, which is usually through the tekb, tbky instructions. EK always equal to 2.

Indirect Addressing

The 32-bit CPUs support reading and writing memory using a base address and an offset from that address. The base is often R0 in earlier ROMs, and often the GBR register in later ROMs. R15 is effectively a stack pointer, so when you see R15 being used for indirect addressing, you're looking at references to what were local variables in the original C code (I'll admit that I have no solid evidence that the original code was C, but the assembly code strongly resembles the output of GCC.)

Indirect addressing makes the code a bit harder to follow, but there's an IDA feature that will help make sense of it:

Common patterns in the data - notable data structures and address ranges, etc

16-bit content by Eleventpoint7five, 32-bit content by NSFW. Editing (or lack thereof) also by NSFW. There's probably room for some merging of the two, but that hasn't happened yet.

Edit: dschultz - added more table/series structure definition detail for 16 bit ECU
16bit Table formatting IDA script

2D table structure layout:
---------------------
axis values
axis length - 1 (i.e.: 0 based)
type (this is the byte referenced in subroutines, i.e.: Table_)
data values
When reading the map:
idxY = table map type byte (function assumes yk=2).
regB (byte) or regD (word) = axis (depending on data type).


3D table structure layout:
---------------------
y-axis values
y-axis length - 1 (i.e.: 0 based)
x-axis values
x-axis length - 1 (i.e.: 0 based)
type (this is the byte referenced in subroutines, i.e.: Table_)
data values
When reading the map:
idxY = table map type byte 16-bit offset
regB (byte) or regD (word) = x-axis (depending on data type)
regE = y-axis


2D series structure layout:
----------------------
axis step
axis start
number of data items - 1 (i.e.: 0 based)
type (this is the byte referenced in subroutines, i.e.: Table_)
data values


3D series structure layout:
----------------------
y-axis step
y-axis start
x-axis series length - 1 (i.e.: 0 based)
x-axis step
x-axis start
x-axis series length - 1 (i.e.: 0 based)
type (this is the byte referenced in subroutines, i.e.: Table_)
data values

Type byte - bit definition:

Layout of a 2D table in memory (32-bit)

The table format information below was originally posted by XSWREX at the OpenEcu.org forums, and I have changed the presentation a little bit here for readability.

row_count: 2 bytes indicating the number of elements
data_bits: 2 bytes indicating the type of data (0 = 32-bit*, 4 = 8-bit, 8 = 16-bit)
row_address: 4 bytes pointing to the start of the row heading value array
data_address: 4 bytes pointing to the start of the data array

In some ROMs, if data_bits is not zero, the values above may be followed by:
multiplier: 4 bytes indicating a multiplier for data
offset: 4 bytes indicating an offset to the data values

Think "y = mx+b" where the multiplier and offset values are the M and B in that function.
Tables containing 32 bit data are not followed by scaling values.

Layout of a 3D table in memory (32-bit)

column_count: 2 bytes indicating the number of columns
row_count: 2 bytes indicating the number of rows
column_address: 4 bytes pointing to the start of the column heading array
row_address: 4 bytes pointing to the start of the row heading array
data_address: 4 bytes pointing to the start of the value matrix
data_bits: 4 bytes indicating the type of data (0 = 32-bit*, 4 = 8-bit, 8 = 16-bit)

In some ROMs this may be followed by:
multiplier: 4 bytes indicating a multiplier for data
offset: 4 bytes indicating an offset to the data values

The multiplier and offset are floating point numbers - 4 bytes, single-precision, IEEE format. These are not always present.

* A value of zero in the data_bits field usually indicates float values, but may just mean that there is no offset and multiplier. I'm not entirely clear on this.

Using the Pull routines to find more tables

If you look for references to tables, you'll find that they're typically loaded into a register right before a call to a function. That function is something that you'll want to label as "Pull2d" or "Pull3d" depending on the table type. Then keep an eye out for other calls to Pull2d or Pull3d. That will make it easier to recognize usages of not-yet-documented tables.

You may also find specialized versions of the Pull routines, for example there's one in the 32-bit ROMs which is used only for tables that have 8-bit cell values.

With an understanding these types and data layouts, you can find tables like Target Boost and Base Timing. If you're just learning, it's best to find stuff that is already defined study the code that uses them to familiarize yourself with the ROM.

1D Tables (16- and 32-bit)

There are also constant values scattered throughout the code. These are referred to as 1D tables in the ROM definition, but they're really just the addresses of const values. You'll just have to find those one at a time because there is no extra data structure around them that indicates they're special.

For example, when investigating the fuel pump duty cycle tables, I (NSFW) found some instructions that were loading values from memory into floating-point registers. Viewing those memory locations as floating-point values, I found values of 33% and 66% - since I recognized those values from logging Fuel Pump Duty Cycle, I knew I had some memory locations that were worth defining as a 1D table.

Interesting addresses and ranges (32-bit)
My Legacy's ROM has most of the table data from 0xC0000 and above. There are some tables stored at lower addresses (mostly in the 0x82A00+ range) but there is no code at higher addresses.

My ROM also has the calibration ID (A2WC522N) stored at 0xC0000. I am not sure how common this is - I may revise this text if I find out that it's not.

Our CPUs use an array of pointers to interrupt and exception handlers, which is typically located near the end of the ROM image. Mine for example starts at FFC50, and runs almost to the end. The first element of array is stored in a dedicated register called the Vector Base Register, or VBR.

The very last doubleword of my ROM image is a pointer to the calibration ID "A2WC522N" at address 0x2000 (yes, it's in the ROM in two places). The doubleword prior to that is a pointer to the code that sets the VBR.

Calls to the Pull routines in 16-bit code

The code will send axis information to the map parsing routines:

For a 2D map:
iy = table map type byte 16-bit offset (function assumes yk=2).
b or d = y-axis (depending on data type).

For a 3D map:
iy = table map type byte 16-bit offset
b or d = x-axis (depending on data type)
e = y-axis.

Calls to the Pull routines in 32-bit code

coming soon.

Any chance of adding the details for 16bit? I don't have access to 32bit, but would be willing to put in many many hours to learn via 16bit

Leslie

I don't actually know anything about the 16-bit ECUs, but I suspect it's very similar. Hopefully someone who is familiar with both can point out where what's different about 16-bit ECUs. It probably only requires a different processor choice and RAM address range. If that's really all it takes, I'll shorten the title.

This works for me every time.
When you use that dialogue box during the initial load, specify the RAM start address and length:
For 7055, Start is 0xFFFF6000 and Length is 0x7FFF
For 7058, Start is 0xFFFF0000 and Length is 0xBFFF
and check the box to make the RAM segment.
More processors details here: http://romraider.com/forum/viewtopic.php?t=5825

Like 2D and 3D table definitions...
viewtopic.php?f=34&t=5603&start=58

Thanks for this guide - I was missing a few key parts and you've definitely set me straight.

Very nice man! I wish I had some more time right now to get a better grasp on the 32bit stuff. I'm able to work with it, but it takes me forever! Took me 3 weeks to put together defs for the 2010 WRX! Bill could have had that done in a day! Lol!


I plan to have a similar write up for 16bit stuff over the winter when I am laid off. I will try to go as in depth as possible and explain as much as I can. If you have any specific questions let me know and I will try to help you out sooner.

Andy

Yes! Thanks for the link.

I just updated the floating-point value conversion utility in 2nd message to make it a little more usable.

CHeers Andy, that would be awesome mate.

Leslie

Here is a set of functions for Excel that you can use. I got them from the Interweb somewhere.
The HEX2SNG is useful to convert hex float to a single decimal value.
44960000 =HEX2SNG(A1) result-> 1200

Or this calculator on the web.