Subfolder locations in PRJ

Binary formats and related hacking.

Moderator: Skyfaller

User avatar
Skyfaller
Clan 1st MechWarrior
Clan 1st MechWarrior
Posts: 1017
Joined: Sat Apr 12, 2008 2:58 am
Location: Germany
Contact:

Re: Subfolder locations in PRJ

Post by Skyfaller »

(Updated information above)

In case I forget updating this thread, you my find more accurate information on the PRJ and related formats in the file MW2-files.txt. It is part of the source download of MechVM (the non-win32 download).
User avatar
Col.Kell
House Steiner Archon
House Steiner Archon
Posts: 868
Joined: Sat Sep 27, 2008 7:44 am
Location: An Isolated Tennessee Valley.

Re: Subfolder locations in PRJ

Post by Col.Kell »

Sky! I just came up with something, I believe you can add files without messing things up. The 31cc PRJ doesn't seem to have anything that declares an official end-of-file.
I have added the Executioner EX1_HEAD.WTB to past the end of the file, the game seems to take it well.

I believe all you have to do is put your hexeditor into insert mode, and add the new geometry name into the GEOTABLE, and assign it with an OBJ tag that is unused. Next, you'll have to alter the DATA tag in the new geometry file to where it will be recognized in it's proper subfolder and such.
I don't know much on it, but I believe it can be done.

I'll have to take a look at your info when I get back from Arnold AFB. Thanks for keeping me updated! :)
MechWarrior 2: 31stCC
Image
User avatar
Skyfaller
Clan 1st MechWarrior
Clan 1st MechWarrior
Posts: 1017
Joined: Sat Apr 12, 2008 2:58 am
Location: Germany
Contact:

Re: Subfolder locations in PRJ

Post by Skyfaller »

Yes, you can add as much data as you want to the end of the PRJ. This is what MechVM does if a replaced file is larger than its predecessor. The problem is making MW2 aware of that additional data. The PRJ has an internal folder structure that is difficult to update. Maybe adding files will work if I also add a new symbol table ...
User avatar
Col.Kell
House Steiner Archon
House Steiner Archon
Posts: 868
Joined: Sat Sep 27, 2008 7:44 am
Location: An Isolated Tennessee Valley.

Re: Subfolder locations in PRJ

Post by Col.Kell »

:) Good luck with the work! In the mean time, I am on my way out the door. See ya!
MechWarrior 2: 31stCC
Image
User avatar
quota4stupid
House Steiner Private First Class
House Steiner Private First Class
Posts: 81
Joined: Mon Dec 29, 2008 7:40 am
Location: Australia

Re: Subfolder locations in PRJ

Post by quota4stupid »

G'day,

I've been poking around in the MW2 data files for the last two weeks (would like to write a MW2 engine in Java :-)) and stumbled across this forum a couple of days ago while looking for a copy of Dan Kegels Mercnet Mission Building FAQ. Thought I'd contribute my own findings on the PRJ format to this discussion (my apologies in advance if I screw something up in this post - I'm not only new to *this* forum, but to forums as a whole :-))

Code: Select all

struct MW2_PRJ_directory_header {
char marker[4]; // Marker string INDX
DWORD structSize; // Size of this record, plus MW2_PRJ_Directory_Entries, minus 8
[color=#FF0000]DWORD unknown;[/color]
[color=#FF0000]char directoryName[4];[/color]
[color=#FF0000]WORD maxEntries; // (see below) [/color]
[color=#FF0000]WORD entryCount; // Number of entries after this header, including an initial NULL record[/color]
[color=#FF0000]WORD unknown2; // (see below)  [/color]
}; // 22 bytes

struct SymbolTableHeader {
char marker[4]; // "SYMB"
DWORD size; // Size of this record, plus file name records, minus 8
DWORD unknown1;
[color=#FF0000]char directoryName[4];[/color]
[color=#FF0000]WORD maxEntries; // (see below) [/color]
[color=#FF0000]WORD entryCount; // Number of entries (no NULL record in SYMB table)[/color]
[color=#FF0000]WORD unknown2;[/color]
}; // 16h=22 bytes
As far as I can tell, for both the INDX and SYMB tables, there is space allocated for up to "maxEntries" records after the initial header. In both cases "entryCount" gives the number
of these entries that are actually used (contain valid records).

In the case of the INDX table, there is an initial NULL record (all zeros). I believe this is included so that fields in the BWD files that identify a MW2.PRJ entry by "FileNum" can be set
to NULL with a zero value.

I'm still not 100% sure about the "unknown2" field in "MW2_PRJ_directory_header" :-( In almost all cases it seems to be equal to the "entryCount" field, with a few exceptions (for example
the "MGI" directory in the Ghost Bears Legacy project file).

I believe that to add files you would simply add the extra INDX and SYMB records in the available empty space and increment the "entryCount" field in both tables (and probably the "unknown2"
field as well?). Also I think you need to keep the SYMB table entries ordered alphabetically.

Cheers,
quota4stupid
User avatar
Skyfaller
Clan 1st MechWarrior
Clan 1st MechWarrior
Posts: 1017
Joined: Sat Apr 12, 2008 2:58 am
Location: Germany
Contact:

Re: Subfolder locations in PRJ

Post by Skyfaller »

Welcome to mech2.org!

Thanks a lot for your help. I'll need a little time to digest all the info you posted.

Yes, using the reserved space in the tables is a good idea, but I think that once users start adding files to the PRJ, they won't want to stop even if the reserved table space runs out :?

We appear to have common goals. If you care about collaboration, there are some other MW2 riddles that I've lacked the time so far to dig into deeply enough. May I ask if your Java project designed to be distributed one day, and in which form (open/closed source, license)? How far did you get with the other file formats in MW2?
User avatar
quota4stupid
House Steiner Private First Class
House Steiner Private First Class
Posts: 81
Joined: Mon Dec 29, 2008 7:40 am
Location: Australia

Re: Subfolder locations in PRJ

Post by quota4stupid »

Thansk mate :-)

I think adding files easily was the reason the original programmers added that reserved space, but yeah, eventually it would come to an end. I guess at that point you'd have to move the respective INDX and SYMB tables to the end of file, make them bigger and then update their DDIT table entry at the start of the file (as well as the maxEntry fields). You would then of course have a gaping big whole elsewhere in the file where the original tables were located at (I *think* the "FREE" entry in the DDIT table has something to do with this, but I haven't really looked into it yet). Mind you, my goal at this stage is just to *read* the MW2 files, not edit them, so all of this is speculation on my part :-D

Yeah, plenty happy to share what I've got with you (at this stage I don't want anything in return - my goal is to try to decode all the files by myself, a kind of personal challenge thing). I haven't set any concrete goals for my MW2 engine yet. If I actually manage to get it off the ground then I will definately release it, probably open-source (not sure of which license I'll use, but probably not GPL).

My file decode status at the moment is as follows:
- WTB: Partial (I can extract the meshes), still got a lot of unknown fields though.
- SFL: Figured out that they play @ 11KHz, but haven't figured out more than the first 8 bytes of the header ;-)
- MEK: Got most of this done, although I still need to complete my list of weapon/equipment/ammo critical values.
- MW2: Done, but I haven't really looked at any of the files in it yet.
- MWM: Done (POSIX TAR file)
- PRJ: Mostly done, but still some unknowns.
- BWD: Working on this at the moment - I've managed to decode a pretty decent chunk of this so far.

This is mostly based on the files from mercs (I've checked a few things against the 31cc and GBL files, but don't plan to check them extensively until I'm finished with mercs).
User avatar
Skyfaller
Clan 1st MechWarrior
Clan 1st MechWarrior
Posts: 1017
Joined: Sat Apr 12, 2008 2:58 am
Location: Germany
Contact:

Re: Subfolder locations in PRJ

Post by Skyfaller »

Most important MW2 file format mysteries:
  • How does texturing work? Some of the fields in a WTB file probably reference texture files in the PRJ by number, but I did not find the time yet to figure out exactly how.
  • MechVM's decoded SFL files are often garbled. The file is split into blocks.
SFL File Format
Signature: SFLX
The files consist of blocks with varying length. The first of these blocks starts at offset 14. The first byte in a block says how to decode the block. Most often, the first byte in the block has value 5. That means 128 bytes of uncompressed PCM data follow, 11025 samples per second. If the first byte has value 4, the block length is 80 bytes, for block type 3, it's 37 bytes, and for block type 2, it's 19 bytes. I think I made some experiments decoding block types other than 5 with various decompression schemes, but it didn't work.
Here's what we know about the BWD files: viewtopic.php?f=17&t=406

About adding files to the PRJ, yes, the old directory tables would stay in place. I don't think it matters much, but if we can successfully add files to the PRJ, maybe we could even rebuild it later.
User avatar
quota4stupid
House Steiner Private First Class
House Steiner Private First Class
Posts: 81
Joined: Mon Dec 29, 2008 7:40 am
Location: Australia

Re: Subfolder locations in PRJ

Post by quota4stupid »

Yeah, haven't figured out texturing yet :-P Figured the two words after the X, Y, Z coordinates (in the WTB point records) were maybe texture coordinates, but I haven't really looked into it much yet. Figured I should try and finish work on the BWD files first (since the BWD seems to be the glue that ties everything else together). I'm pretty sure that the BMEN, BMID, BMPJ and BSEC tags in the BWD "xxxxMAP1" files are related to texture mapping too, but haven't started working on them yet (I'm currently completing the list of MEK critical slot values - when I go back to working on BWD it'll be the CPGN tag).

You're miles ahead of me on the SFL file (because playing with them tends to be noisy, I have limited opportunity to do so).

You can see my current work on the BWD format here: http://users.tpg.com.au/nkjd245/mw2/bwd.txt
User avatar
quota4stupid
House Steiner Private First Class
House Steiner Private First Class
Posts: 81
Joined: Mon Dec 29, 2008 7:40 am
Location: Australia

Re: Subfolder locations in PRJ

Post by quota4stupid »

Hit a wall with the MEK format and the BWD CPGN record, so started poking around in the PRJ format some more. Since we've been talking about adding files to the PRJ, I thought you might find this interesting.

Dunno if you've had a look at the FREE folder, but it's INDX table points to a series of FREE records (as opposed to the DATA records that the other INDX tables point to). The record format is:
0 - 3 : "FREE"
4 - 7 : Record size - 8
8 + : Garbage

I believe that these FREE records are unused space in the PRJ file that you could use to add new DATA records. If you do a hexdump of some of these records you can see that they were originally DATA
records used in the other PRJ folders (why they weren't simply removed completely is beyond me, but anyway...). In the Mercs MW2.PRJ, these records make up about 0.5MB of space. A lot of them are
too small to use, but there's a few that are over 1K in size.
Post Reply