Dbi files

The current status of the extractor tool could recaped as: the only hard thing left are the dbi files. I’m focusing on the hardest reverse engineering first, so I’m leaving anything that is fairly easier for later.

Why are they hard? They are basically a custom format that I haven’t been able to relate to any other game, tool or software in general.

So far, I’ve been able to find out the following:

  1. It’s an indexed file.
    1. It contains the number of elements at the top of the file, in the header.
      This header takes 0x420 bytes where 0x400 bytes are a color palette.
    2. The index is at the bottom of the file and uses 8 byte strings for names and a 4 bytes index in the file.
    3. Elements start after the header.
    4. Each element is 0x2000 long
    5. You can get the start of a file by jumping at 0x420 + index * 0x2000
  2. It contains a color palette of 0x400 bytes in the header.
  3. It contains images. I’ve been able to get this from Listbox.dbiturret
    I’m not making this up, this a turret from the game 😀
    I still need to get this through the palette and it should look nice.
  4. Before each image data, there’s the size of the image. On Listbox.dbi you can use the width and height of the images (present in the header) + padding to get the size of the image. On others, the number is too small.
  5. Images in it are encoded/compressed/obfuscated/encrypted (except but Listbox.dbi), which is not.
    I’ve found a 32 bits field that is set to 0 on Listbox.dbi and 1 in the rest.

    My current focus is to understand what this field is really saying and interpret the image data correctly. I think images are compressed, as the sizes are too small.

  6. It contains other data, just after the image itself.
    After each image, there are a lot of indices. I think they could relate to terrain types, but I don’t know what is that used for; let me copy some examples:

    1. JUNG611 (jungle)
    2. ARID395 (arid)
    3. DESE830 (desert)
    4. BORE828 (???)

And these are the findings of the last week! If I’m able to crack down the compression for the dbi files I will start coding the game soon 🙂

 

Extractor tool integrations

Since the start of the project I’ve been adding integrations to the project.

In case anybody is interested, I’ve been doing the following:

  1. Setting up a gitter chat community.
    This was really easy to set up, just a bunch of clicks.
    I thought this would be a great way to enable others to reach me to collaborate with the project.
  2. Setting up a project on github.
    This is a simple agile-board, just to get a sense of what remains to be done. It could be great for collaborators to be able to pick a task and make sure that is going to be useful work.
  3. Setting up a build server on AppVeyor
    This has been a great way for me to make sure the project compiles successfully on other machines (other than testing on 2 PCs of mine). This should later allow to download a compiled version of the tool from the artifacts section.
    Also, it automatically runs unit tests, so it should give a good sense of the builds health as soon as there’s more unit tests.
  4. I’ve started to look into code coverage analysis.
    I’m currently using OpenCover with ReportGenerator, but it’s still work in progress and will probably need a bit of love before merging to master.
    I’ve got most of the info I needed from here.

Cracking the FF file

Hi there!

As I’m writing this post, I’m waiting for the extractor tool I have to give me some result after adding a bit of code with timing info (just total duration), since the export is taking an enormous amount of time.

So what I have been doing between the previous post and this one is trying to understand 2 file formats that the game uses that I don’t export from yet.
One of them is called DBI, which seems to custom made, it contains an Autodesk  license (on text, like you would write it on code file), and even error messages for things I don’t think I’ll ever seen inside the game.

As my experience reading through undocumented binary formats that I haven’t written is limited, I had to call it a day and leave the DBI files format unsolved.

FF file header

The other one is the FF file, that starts with the magic word “MFF”. I haven’t found any reference to it online, so I started looking at it on my own. These videos from Mr House helped a lot on this.

I think I understand the file format, so here’s the file structure for the FF file (there’s only one file like that in the game) is more or less as follows:

  • 4 bytes, magic word MFF, followed by \0.
  • Index of files in the archive header
    • 4 bytes with the amount of files in the archive.
    • An entry for each file
      • 256 ASCII chars, file path
      • 4 bytes, offset to the file contents on the file.
      • 4 bytes of 0’s.
  • Start of file data.
    • Each file here is an SMK2 file, so my converter for SMK files can read this.
    • There’s no hint (that I’ve been able to find) of each file size.
      I’m currently guessing the file size by reading the SMK2 file header and trying to add up, but I’m probably not doing that perfectly, since the SMK2 file format specs I’m using are not official.

After reading through the FF file more or less successfully, I will only need to read through the DBI one and I should have access to everything!

Liberation Day remake challenges

Hi everyone,

as mentioned on my previous post, I’m working on a remake for Liberation Day in Unity.

To be able to remake this game, I aim to extract assets from the original game during installation time and use those assets from the newly written code.

Legally speaking, I won’t be able to distribute the game assets, so each player will need to get a copy of the original game to play and run an extractor tool to get the assets from the original game into the remake.

installer
The installer/extractor tool

This game is old, 1998 old. This means it uses tools and libraries that are very old, like DirectX 5, Smacker Video or Microsoft Jet databases.

Understanding the game data formats, what are they used for and find how to implement converters for them is going to be the greatest challenge of the remake, I think.

So far, I’ve managed to read through the 2 formats I already mentioned and managed to read and export them to more “modern” formats:

  • SMK files. They all start with the Magic number SMK2, which helped me find that they are Smacker Video files. The game has some videos made with this, but also has interface assets stored with it. I’ve managed to open these with ffmpeg and export each keyframe to png. For that I’ve used FFmpeg.AutoGen, since I needed a C# wrapper around the C++ libraries.
  • MDB files. These are Microsoft Jet databases, opening them with Hex worshop revealed another magic header “Standard Jet DB”. Through mdbtools, I’ve been able to see the game always uses version 3 of MS JET DB, which I’ve been able to open through an OleDbConnection. Some of the databases on the game are password protected, but thanks to mdbtools being open source, I’ve been able to quickly read the passwords from the password protected ones and all of them are now TSV files.
11
Inside SMK files we have plenty of interface assets, like this one.

There are still big challenges ahead, specially since I’ve only been able to get interface assets and databases. There’s a file called ANMSUNIT.FF, that uses a completely unknown format (probably custom made) that seems to hold all animations for units, probably with the units assets. There’s also DBI files, which seem to hold some kind of images, since they are stored in an IMGS folder.

Feel free to give the code a look at github.

I also set up a gitter chat on the repository, so anybody that wants to know more or wants to contribute can reach me.

 

 

New project & blog resurrection

It’s been literally years since I don’t write on this blog. I’ve been very busy since then on work stuff, which will now allow me to write more interesting stuff than what I could write back at 2012 or 2013 😀

The reason I’m resurrecting this blog is because I have decided to work on remaking Liberation Day (gameplay here https://www.youtube.com/watch?v=6YtyTmItD6A), a turn base strategy game from 1998. This game is also known as Fallen Haven 2, since it’s the second part of a more famous title, also turn based strategy game.

I plan to create a github repository with the game and tools related to it, as soon as there’s something worth to share.

EDIT: you can find the repository here.

This project has many challenges, which will require me to learn a lot of new things, that will probably be well suited for blog posts.

I hope this blog will also be a good way to have somebody interested on working on this with me, so this project is not only something I work on weekends when I have some time to spare, but something bigger.

I’ll keep you posted!