Hack Age of Empires III to change shader quality settings

The beginning of May 2020 - if you are like me, then quarantine made you redo games that have not been launched for many years.

And if you are even more like me, then somewhere you might have a disk of Age of Empires 3 lying around. Maybe you are playing on a Mac, maybe you have not upgraded to Catalina yet and want to command Morgan Black.

So, you start the game, get into the main menu, and immediately notice that something is wrong ... The menu looks disgusting .


If you are wondering what exactly is “disgusting,” then pay attention to the water. Everything else is terrible too, but it's less obvious.


So, you go into the options, raise all the parameters to the maximum ... But the game is still ugly.

You notice that the " Shader Quality " options are suspiciously locked to " Low ."

Attempt No. 1 - hack parameters


At this point, you begin to search for the game’s folder, which was supposedly created somewhere in the Documents folder, because the game was in 2005, and then everyone did it.

What are we looking for? Of course, the file in which the parameters are stored. The menu interface does not allow us to change the settings, but we are tricky, right?

We find the right XML, because the game is 2005 and it’s, of course, XML, where we come across the option " optiongrfxshaderquality ", which is set to 0. It seems that we were looking for it, so we increase the value to 100, because there is not much quality. Here I agree with you.


You may also notice that this is a terrible use of XML. Fortunately, he has no good uses.

Satisfied with ourselves, we launch the game. Alas, nothing has changed. Looking again at the XML file, we see that the parameter is again set to 0. Ensemble Studios (may it rest in peace) looks with contempt at all your ingenuity.

After that, we could give up. This is a Mac, therefore, the patches written by users are unlikely to be found (in fact, I checked - there are all sorts of dubious solutions that may work). It's all about the graphics. It seems to fix the problem will be difficult.

But we will not give up. Because we remember what Age 3 should look like. We remember how we admired this beautiful wavy water. These particle effects. These huge ships that just do not fit into the frame. And, it’s just that the camera was zoomed in, what was I just thinking?

Attempt No. 1 - hacking data


At the very beginning of the folder in Documents there is a log file. The game wrote a graphics card there and reported that it had selected the “generic dx7” setting. It says that you have installed Intel GMA 950, which really was on the previous computer.

But on this you have Intel Iris Graphics 6100. Something is really wrong. You make an assumption - the game determines the graphic capabilities of the computer, comparing it with a database of graphic cards instead of checking the capabilities of the card itself. Because that's exactly what AAA developers do, and if you worked on open-source RTS , then you know how this happens.


The contents of the log file. If you find it funny that Intel is designated as 0x8086, then you have chosen the appropriate article.

By chance, you find old patch notes that confirm your concerns. You look into the GameData folder, but there are only .bar and .xmb files that are mostly unreadable. You are looking for grep " Intel " and " dx7 ". Nothing happens. Delete several files ... Nothing happens. But you know that AAA games can store some resources in strange places, and this is the port of Windows games on Mac, so everything can be very bizarre here.

Ultimately, you find the file “render.bar”; if you move it, the game crashes when it starts. It’s not very clear, but not bad. We open the file in Hex Fiend , because this program usually allows you to learn something about binary files. And so it turns out. Part of this data is readable text. Some of them are shaders. But most of the data is strangely divided by empty space ...


Hex Fiend Interface It is minimalistic, but it works. The first bytes are the letters “ESPN”, which are most likely the .bar header.

You exclaim: “Obviously, this is UTF-16!” The game was created for Windows, so it is logical that it stores text in UTF-16, wasting almost half of a 30-megabyte data file. After all, Windows loved UTF-16 (although it shouldn't ), so why not use this encoding and ASCII shaders in one file. It is a logical idea!

(Actually, it took me about a day to figure it out)

Hex Fiend has the option of parsing bytes like UTF-16, so again we are looking for the grep string dx7. There are several entries. We replace them with the dx9 line, which is also found in the data, save the file and launch the game.

Yeah Logs again do not display "dx9". This is probably set elsewhere.

Let's try something else. Suppose that the game recognizes graphic cards, and their hex identifiers are stored in the logs (in my case, 0x8086 is Intel, and also 0x2773). We are looking for “8086” in Hex Fiend, we find something, and part of the text looks promising. The Intel 9 Series is the GMA 950 series, and Age 3 probably didn't work very well on it.


Some numbers are like card identifiers (2582, 2592); perhaps if you replace them, then something will change. We made a copy of the file anyway, so you can try.

Alas, this is also a dead end. In fact, we are studying the wrong file. Fortunately, I can tell you this because I spent a lot of time on it and tried too much. This damn thing took about 20 hours. And this is not counting the time for all the screenshots ... The

desired file is called "DataP.bar", and if we wisely replace the ID, we will see something completely new in the logs:


I don’t quite understand why this “generic dx7” is not the same as in the screenshot above.

Of course, in the game we are still limited by the “Low” settings, that is, the logs do not lie. We were hoping for a Medium, but alas .

It seems that more from hacking data can not be achieved. It’s time to get the tools seriously.

Attempt No. 2 - hacking


So, if nothing succeeds, we have one last thing left: to change the executable file itself (that is, as they said in 2005, “crack” it). It seems that today it is simply called hacking, but this term has been preserved in the field of game piracy (of course, I personally have never done this).

We move on to the most interesting part of the article (yes, you have already read a thousand words, but wasn’t it curious?).

Disassembler Hopper


At this stage, our tool will be a disassembler: an application that receives the binary code of the executable file and turns it into readable (ha ha) assembler code. The most popular of these is IDA Pro, but it's insanely expensive and I'm not sure if it works on a Mac, so I’ll use Hopper . It is not free (costs $ 90), but it can be freely used for 30 minutes at a time with almost all the necessary functions.

I will not explain the interface in detail, because everything is well laid out in the Hopper tutorial , but I will tell you what I will do and try to take enough screenshots so that you can learn something.

First you need to say: it took me five days to resolve, and many unsuccessful attempts were made. Basically, I will talk about successful steps, so it will seem magic. But it was difficult . Do not worry if you have any problems.

And one more note: “cracking” is most often illegal and / or performed for illegal purposes. I demonstrate a rather rare example of a fairly legitimate use, which at the same time is completely “white hat”, so everything is in order here. I will assume that, strictly speaking, this is illegal / contrary to the end user agreement, but, obviously, is a case of force majeure . Be that as it may, pay for what you like, and only for that, because anti-consumerism is cool.




Drag the Age 3 app to open it. Just select "x86" instead of powerPC, because the game is something 2005.

Stage 1 - search for the desired fragment


Surprisingly, this may be the most difficult. We have a disassembled code, but we have no idea where to look. Yes, some games have debugging symbols, so you can read the function names, but not in our case. Hopper gives us names like "sub_a8cbb6", which we have to deal with on our own.

Fortunately, we have a head start: our log file . It is highly likely that string literals are used when writing to the log, that is, ASCII are flashed in the executable file. We can look for them with grep, because we won’t get rid of them by any compilation (however, they can be hidden by obfuscation of the code. Fortunately, this did not happen.). Finding grep string literals is usually the first thing they do when disassembling a program. So, let's enter “found vendor” in the Hopper search box and it will find something:


Oh yes, string literals, I adore them.

This in itself did not advance us so much. But since the address of this "string of literals" is hard-coded, we can find links to it. Hopper highlighted them in green on the right: "DATA XREF = sub_1d5908 + 1252." Double-clicking on the line, we will move on to our hero - sub_1d5908 procedure .


Here we find the assembler code. It is believed to be difficult to read, but it is not. The hardest thing is to understand it.

By default, Hopper uses "Intel syntax", that is, the first operand is the receiver, and the second is the source. In the highlighted line we see mov dword [esp+0x1DC8+var_1DC4], aXmlRenderConfi. Let's make it out.

Our first line in assembler


movIs a MOV team known for being on x86 Turing-complete . It moves (actually copies) the data from A to B, which essentially means reading A and writing it to B. Based on the foregoing, aXmlRenderConfithis is A, and dword [esp+0x1DC8+var_1DC4]this is B, the recipient. Let's take a closer look at this.

aXmlRenderConfi- this is the value that Hopper helps us. This is actually an alias for the memory address of the string literal that we clicked a few seconds ago. If you split the window and look at the code in Hex mode (which is the preferred mode for hackers), we will see there 88 18 83 00.


Hopper conveniently highlights the same selected fragments in both windows.

If the “flip” value is correct, then we will get 0x00831888 - the memory address of the string literal (in one of the screenshots shown above it is even highlighted in yellow). So, one riddle is solved: the code executes MOV, that is, it writes the address 0x00831888.

Why is it written as 88 18 83 00, not how 00 83 18 88? This happened because of the byte order - a rather confusing topic that usually has little impact. This is one of those things that make people say that “computers don't work.” For us, this means that this problem will occur in all numbers with which we will work today.

You may have noticed that I said “write down the address”, and that’s true: we don’t really care about the contents, which turned out to be a string literal with a zero byte at the end. We write down the address because the code will later refer to this address. This is a pointer.

What about dword [esp+0x1DC8+var_1DC4]? Everything is more complicated here. dwordmeans that we are working with "double words (double-words)", that is, with 16 * 2 bits. That is, we copy 32 bits. Recall: our address is0x00831888, that is, 8 hexadecimal characters, and each hexadecimal character can have 16 values, i.e. 4 data bits. So we get 8 * 4 = 32. By the way, the notation “32-bit” for computer systems came from here. If we used 64 bits, then the memory address would be written as 0x001122334455667788, would be twice as long and 2 ^ 32 times larger, and we would have to copy 16 bytes. So we would have much more memory, but copying the pointer would require twice as much work, and therefore 64 bits are actually not twice as fast as 32 bits. By the way, a more detailed explanation of the term “word” can be found here . Because it is, of course, more complicated than my explanation.

So, what about the part in square brackets? The brackets mean that we calculate what is inside, and consider it a pointer. Therefore, the MOV command will write to the memory address obtained from the calculations. Let's look at it again in more detail (and don’t worry, when you are done with this, you will essentially know how to read assembler code with Intel syntax).

espit is a register that in assembler is the closest equivalent to a variable. There are a lot of registers in x86 architecture, and different registers have different ways of application, but we won’t really care. Learn more about this here.. Just be aware that there are special registers for floating point numbers, as well as “flags” that are used for comparisons and similar operations. Everything else is just hexadecimal numbers that Hopper reworked a bit to help us. var_1DC4- this -0x1DC4, we can see it at the beginning of the procedure, or in the right panel. This means that the result of the calculation [esp+0x1DC8+var_1DC4]will be [esp + 0x1DC8 — 0x1DC4]that equals [esp + 4]. In essence, this means "take the 32-bit value of the ESP register, add 4, and interpret the result as a memory address."

So, we repeat: we write the address of the string literal in “esp + 4”. This is correct information, but it is completely useless. What does this mean?

This is the difficulty in reading assembly code: in parsing what it should do. Here we have a head start: we know that he most likely writes something to the log. Therefore, we can expect a call to a function that writes to the log. Let's look for her.


In the screenshot above, there are several similar calls, as well as the command callthat calls the procedure. Hopper usually talks about this (I don’t know why he didn’t do it here), but in essence, we give arguments for calling the function. If you click on different subroutine calls, we find something called imp___jump_table___Z22StringVPrintfExWorkerAPcmmPS_PmmPKcS_. This is the decoded name of the function, and the “Printf” part is enough to understand that it really outputs something. We don’t care how she does it.

Take a step back


At this stage, we completely departed from what we did: we tried to convince the game that our 2015 graphics card was powerful enough to launch the 2005 game. Let's summarize. We found the place where the log is being recorded. If we are lucky, then here is the code that checks the parameters and capabilities of graphic cards. Fortunately, we were really lucky (there is a high probability that otherwise this article would not have been).

To get the big picture, we will use the Hopper Control Flow Graph function, which breaks the code into small blocks and draws arrows indicating different transitions. This is much easier to understand than in ordinary assembler, in which often everything is in a mess.


The call to the log entry is in the middle of a terribly long function, because this, again, usually happens in AAA games. Here we should look for other Hopper string literals and "hints", hoping to find something. At the beginning of the procedure is the rest of the logging, and below there are interesting parts about “forcing dx7” and “Very High” options, as well as a problem with the version of pixel shaders ... which we have.

The log of our "hacked" data reported that we had pixel shaders of version 0.0, and they are incompatible with the "High" parameters. This code is located in the lower left corner of our function, and if you look closely, you can see something curious (highlighted by your humble servant): it is the same code four times repeated for the parameters "Very High", "High", "Medium" and "Low". And yes, it took me several hours to find this.


I highlighted in blue the block in which “pixel shader version 0.0 High” is displayed in the log. I highlighted similar parts with other beautiful colors.

The only differences are that we write “0x0”, “0x1”, “0x2” and “0x3” in different places. This is suspiciously similar to the parameters file that we examined above and the optiongrfxshaderquality parameter (you may not understand this yet. But it is, believe me).

At this stage, we can try to go in different ways, which I did. But I will be brief - we will do the most necessary: ​​find out why pixel shader verification fails. Let's look for a branch. The output block to the log is linear, which means it should be somewhere above.


SEMVER fans, see a more advanced version format: floating point numbers.

And in fact, right above it there jbis a transition command (think of “goto”). This is a " conditional jump ", and the condition is " if less ." Assembler “ifs” work through register flags , which I briefly talked about above. It’s enough just to know that we need to look at the command above: most likely, it sets the flag. Often this is a command cmpor test, but used here ucomiss. This is barbarism. But she easily googles: ucomiss . This is a floating point comparison command, and this explains why the code hasxmm0: This is a register of floating point numbers. This is logical: the logs report that our version of the pixel shaders is “0.0”, which is a floating-point value. So what is located at memory address 0x89034c then? Hexadecimal data has a look 00 00 00 40and it can be confusing. But we know that we should expect floating point numbers, so let's interpret the data like that: that means 2.0. Which is a logical floating point value for the version of pixel shaders that Microsoft actually used in its High-Level Shading Languageon which DirectX shaders are written. And Age 3 is a DirectX game, even though the port on the Mac uses OpenGL. It is also logical that in 2005 pixel shaders of version 2.0 are required to enable the High parameter, so we are moving in the right direction.

How do we fix this? This comparison instruction performs a comparison with the value at xmm0which is given in the above line: movss xmm0, dword [eax+0xA2A8]. MOVSS is a special “move” command for floating point registers, and we write 32 bits from the address, which is the result of evaluating eax + 0xA2A8.

Presumably, this value is not 2.0, but rather 0.0. Let's fix it.

Doing real hacking


Finally, we are ready to get to the parameters of the High game. We want to go along the “red” path and skip all the logic with the “wrong version of pixel shaders”. This can be done by successfully passing the comparison, that is, by reversing the transition condition, but we have one more trick: the code is composed in such a way that when deleting the transition we will go the “red” way. Let's just replace the transition with nopan operation that does nothing (it is impossible to simply delete or add data to the executable file because an offset will occur and everything will break).

I recommend getting out of CFG for this, because otherwise Hopper starts to go crazy. If everything is done correctly, then in the Hex window we will see red lines.



Red shows the changes we have made. At this point, if you paid for Hopper, you can simply save the file. But I didn’t pay, so I will open the executable file in Hex Fiend, find the desired area (by copying the area from Hopper to the Find tab inside Hex Fiend) and change it manually. Be careful here, you can break everything, so I recommend copying the source executable file somewhere.

Having done this, start the game, go to the options ... And cheers - we can select "High". But we can not choose "medium". And we can not use the high parameters of the shadows. Nevertheless, we are making progress!


Just take a look at these magnificent reflections in the water!

Enhancements


In fact, you can fix this in the best way. Our solution worked, but we can make it so that the condition is met correctly: making the game think that our graphics card actually has shaders of version 2.0.

As we recall, the game loads the value at the address eax+0xA2A8. You can use Hopper to find out what was originally recorded there. We need to find a command in which 0xA2A8 is used as the recipient operand (I chose 0xA2A8 because it is a fairly specific value, and we cannot be sure that the same register is not used elsewhere). Here the Hopper backlight is very useful to us, because we can just click on 0xA2A8 and look for the yellow parts in CFG.


Our victory is a little higher: as expected, we write the value of some kind of floating-point register. I admit that I do not really understand what is happening before this (my best guess: the game checks the possibility of texture compression), but this is not particularly important. Let's write “2.0” and continue the work.

To do this, we will use the “Assemble instruction” of the Hopper application, and then do the same manipulations in Hex Fiend.




We use MOV instead of MOVSS because we write the usual value, and not the special value of the floating-point register. In addition, it is in direct byte order, that is, inverted.

You will notice that something strange is happening: we “ate” the team jb. It happened because the new command takes more bytes than the previous one, and as I said, we cannot add or remove data from the executable file to save offsets. Therefore, Hopper has no choice but to destroy the team jb. This can become a problem, and we can eliminate it by moving the command up (after all, we no longer need to write the xmm3 value). This will work here, because in any case, we are doing the branching on the right path. Lucky.

If you start the game ... then nothing will change much. I suggested that the Intel 9 series is not fast enough, so the game complains. This is not a problem, because in reality we are striving for “Very High”.

What was the most powerful card around 2004? NVIDIA GeforceFX 6800. Let’s trick us into convincing the game that we have it installed.

Down the rabbit hole


Yes, this is a completely new section.

We know that the game uses Hex codes to refer to graphic cards, and that it receives information from data files. We know that this should happen in the function we are studying (at least it would be strange if it weren’t). So we can find the code that does this. But it can still be difficult, because we do not know what exactly to look for.

We have one clue: options behave strangely; there is Low / High, but no Medium. So maybe there is another code that handles all of these “Very High”, “High”, “Medium” and “Low”. And he actually is, in the same function. He's a lot earlier at CFG, and seems interesting.


We can assume that the raw data is stored in some kind of XML, because it is very likely, because there are several other data files that are XML. And XML is essentially a bunch of pointers and attributes. Therefore, the code here most likely checks for some XML attributes (and in fact, below is “expected one of ID, Very High ...”, which is very similar to checking the correctness of the file). We can imagine a similar XML structure in which Boolean values ​​determine the quality of shaders:


It is worth considering that this is a very arbitrary guess. Everything may look a little different.

A very interesting part is shown on the left side of the CFG (for you it can be on the right, but shown on the left in the screenshot): we see the same var_CCone that is used in the code that writes the device ID to the log. That is ebp+var_CC, probably refers to the ID of our device, 0x2773. The strange thing is that there is no string literal in the first “check”, but this is a Hopper bug: the address 0x0089da8e contains hexadecimal data 69006400, which in UTF-16 stands for “id” (in fact, probably Hopper could not understand this because UTF16). And we are just looking for an ID.

You know what? Let's run the debugger and check everything in reality.

Game debugging


Open a terminal window and run lldb (just type lldb and press enter). First we need to tell LLDB to follow Age 3, and then we can start the game by calling run. The game starts, and we get nothing useful.


Now we need to add a breakpoint: to confirm our assumptions, we want to stop execution when we get to the code shown above. We do not have source code, but it doesn’t matter: we can set a breakpoint at the memory address. And we have the address in the code. Let's add a stop to a MOV call that receives an "id", its address is 0x001d5e68. To add a breakpoint, just enter br set -a 001D5E68. After starting the game, it stops and LLDB shows the disassembled code (in the AT&T syntax, not Intel, so all the operands are inverted, but we see that this is the same code). You may notice that LLDB in this case is actually smarter than Hopper; he tells us that we are performing operations related to XML and printf output.



Feel like a hacker?

To move on, let's take a “step instruction”. A short command for this
is ni. Repeating several times, we notice that we are back to where we started. This is a cycle! And this is completely logical: we are probably iterating around some kind of XML. In fact, Hopper showed us this - if we follow the CFG, we will see a (possible) cycle.


This means:if (*(ebp+var_CC) == *(ebp+var_28)) { *(ebp+var_1D84)=edi }

Its exit condition is for the value to ebp+var_1D84be non-zero. And we see an interesting code parameter in the one highlighted by me var_CC.

Let's add a breakpoint on this one cmpand we'll figure it out.



Left: set a breakpoint on the CMP and continue execution. On the right, we monitor the registers and memory.

We look at registers with reg r, and the value of memory with mem read $ebp-0x28. eaxand actually contains 0x2773, and the value in memory is now 0x00A0. If you execute thread cit several times , then we will see that iteration occurs over different IDs in search of matches. At some stage, the values ​​will coincide, the loop will exit, and the game will begin. Now we know how to manage it.

Recall the log file: it identified both the device and the manufacturer. Probably somewhere there is a similar cycle for the names of manufacturers. And in fact, it is very similar and located in the CFG directly above our cycle. This loop is a bit simpler, and we look for the string “vendor”.

This time the comparison is done withvar_D0. This variable is actually used as the manufacturer ID. If we put a breakpoint here and check everything, we will see the familiar 0x8086 at eax, and at the same time, a comparison with 0x10DE is happening. Let's write it in eaxand see what happens.



Wow.

That is, 0x10DE is NVIDIA. In fact, one could understand this when studying a data file, but it’s not very interesting. Now the question is: what is the identifier of the GeforceFX 6800? We can use LLDB and just check each identifier, but it will take time (there are a lot of them, I tried to find the right one). So let's take a look at render.bar this time.


Suppose this is “XMB”, the binary representation of XML used in Age 3.

This file is pretty chaotic. But after the Geforce FX 6800 tags, we see several identifiers. Most likely, we need one of them. Let's try 00F1.

The easiest way to test this is to use LLDB and set the desired breakpoints. I will skip this step, but I will say that 00F1 came up (fortunately).

At this stage, we need to answer the question: “How to make this change permanent?” It seems that it will be easier to change the values var_CCand var_D0on 0X00F1 and 0X10DE. To do this, we just need to get space for the code.

A simple solution would be to replace one of the log record calls with a NOP, for example, a subsystem call. This will free us several tens of bytes, and this is even more than necessary. Let's select all of this and replace it with NOP. Then we just need to write information using the MOV variables: assembler mov dword [ebp+var_CC], 0xF1and mov dword [ebp+var_D0], 0x10DE.



Before and after

Let's run the game. Finally, we have achieved something: you can choose from Low, Medium or High, as well as enable High parameters for shadows.


But we still can’t enable Very High, and this is sad. What's happening?


Ah, got it.

As it turned out, the Geforce FX 6800 should support version 3.0 pixel shaders, and we only set version 2.0. Now it’s easy for us to fix it: just write 3.0 floating point in ebp+0xA2A8. This value is 0x40400000.

Finally, the game is no longer capricious, and we can enjoy Very High settings.


Strange, but it looks completely different. The colors are much less vibrant, and range-dependent fog has appeared. There is also a small problem of shadow conflict (it is also in the screenshots above, this is due to the High shadow parameters, not the quality of the shaders), which I have not been able to solve yet.

And this is the end of our journey, thanks for reading.

In addition, I will give examples of how each of the parameters in the game looks like, because all the screenshots found online are terrible or incomplete.

The graphics on Medium are quite acceptable, but suffer from a lack of shadows. As far as I can tell, Very High for the most part adds a completely different LUT (a good explanation of this term can be found in the Color Grading section of this article.), fog in the distance, and perhaps even a completely different lighting model? The picture is very different, and it is rather strange.





From top to bottom: Very-High, High (with High shadow options), Medium (just with shadow enabled), Low (with shadow disabled).

Other useful links


The blog I linked to above contains a wonderful analysis of modern rendering pipelines: http://www.adriancourreges.com/blog/

Rate 0 AD is an open-source RTS game of the same genre: https://play0ad.com . She needs developers.

If you want to know more about the XMB format, you can read its code in 0 AD . I didn’t check, but I’m almost sure that this is the same format, because the person who wrote the XMB decoder for Age 3 on Age of Empires heaven also implemented these files in the 0 AD source code in 2004. So this will be a fun lesson in code paleontology. Thank you for all the work, Ykkrosh.

I remember exactly that I read an excellent tutorial on using Hopper, but now I can not find the link. If anyone knows what I'm talking about, then drop the link.

In the end, I would like to mention the Cosmic Frontier project: a remake of the classic Escape Velocity series (more of Override, but it is also compatible with Nova). This is an amazing game, and it is very sad that few people know it. Now the creators have launched a campaign on Kickstarter, and the lead developer has a very interesting dev blog , which talks about using Hex Fiend to reverse engineer data formats of the original game. This is a great read. If you think my article was insane. then in one of the posts they:

  • We launched the classic Mac emulator to use the long-obsolete APIs to read an encrypted data file.
  • Reverse engineering encryption from the game code
  • They used a file system hack to access the same data on a modern Mac.

Yes, these are truly passionate people, and I hope that their project will be successfully completed, because EV Nova is my favorite game.

All Articles