Doing my part (.mob format)

Discussion in 'General Modification' started by dulcaoin, Apr 10, 2005.

Remove all ads!
  1. dulcaoin

    dulcaoin Established Member Veteran

    Joined:
    Mar 10, 2005
    Messages:
    213
    Likes Received:
    0
    Ok, so I ordered up a copy of Arcanum this evening, should be here in a week or so. I also found all the patches I could, most especially the "earlier" arcanum patch for 1070, with the "good" WorldEd included.

    How will this help us? Easy, as Agetian has pointed out, the version of .mob files used with TOEE hasn't changed from that used by Arcanum. So we should be able to use WorldEd to (say) change a (single) value in a container, and save out the file. As the change is made, it will be obvious what byte(s) have been affected. We than adjust that value in a .mob file for TOEE, and use the TOEE console to affirm that the same values (container_flags, as an example) change properly in TOEE. Through continued process-of-elimation, plus good use of a pro-level hex editor, we should truly limit the number of unknowns present in .MOB files. Even in cases where the format has changed a bit, this should still be helpful. As I look over earlier work in Arcanum, I am struck by how little things changed from one product to the other, so any changes should be as minimal as possible. These guys seem to have been very powerful creatures of habit.

    Despite best current efforts to get the modding tools handed to us, I submit that, by continuing to decipher .mob files on our own, we will learn a lot more about how this was all designed. I'll remind people that Arcanum modders didn't know .mob format (believe me, I've looked for decent docs, and they were nearly as stumped as we have been); and I think we'll be better for the efforts.

    -- d
     
    Last edited: Apr 11, 2005
  2. Agetian

    Agetian Attorney General Administrator

    Joined:
    Aug 14, 2004
    Messages:
    2,526
    Likes Received:
    0
    Hi dulcaoin,

    Glad that you'll join me up! Thanks! Of course we should NOT stop with modding, no matter whether we start our campaign for the official tools or not, and no matter how successful we are. I'll be glad if you could help me out.

    I was kinda busy with the 2D Map Discombobulator recently but I'll be ready to return to MOB-hacking in a couple days.

    - Agetian
     
  3. Firestrand

    Firestrand Member

    Joined:
    Mar 11, 2004
    Messages:
    79
    Likes Received:
    0
    Did you find a copy used? I can't find a new in the box copy anywhere.:(

    Currently the Arcanum WorldEd can't run in the ToEE directory because of DLL incompatibilities. So do you think you could zip those DLLs my way when you get the game so I can help?

    I think that they may have used a similar .mob format, except for the objects. I think by moving to the python scripting engine, they also changed the binary format of objects, so I don't think they will be compatible. I have been busy with a few other things, so I haven't had a chance to check for cPickle type information embedded in the .mob file. It might be a good idea to check.

    I agree, the more you know....

    -Firestrand
     
    Last edited: Apr 10, 2005
  4. Agetian

    Agetian Attorney General Administrator

    Joined:
    Aug 14, 2004
    Messages:
    2,526
    Likes Received:
    0
    I borrowed Arcanum from an acquaintance of mine and installed it. Tried using the Worlded - a lot of things resemble ToEE a lot (even though the Worlded there is definitely incompatible with ToEE). This way or the other it IS possible to draw certain parallels between Arcanum and ToEE in the .mob format. I'll try to join you in a couple days with this.

    So far, here is some information to ponder about (it's ARCANUM-related, not ToEE-related, but ToEE might turn out to be similar in this respect). I tried to use WorldEd, create a generic item (so the Worlded generates the G_xxxxxxxx_xxxxx_... .mob), and then add one property at a time to it, and see what happens to the file (in order to see where the game stores the offsets). As far as Arcanum mob files are concerned, the following offsets change:

    0x0038 0600 0700 0800 0900 0A00 0B00 0C00
    0x0042 4000 4400 4C00 5C00 7C00 7C02 7C22

    OK, the one in 0x0038 makes sense - it may be the number of properties. But what's with the lower one? It seems to increase in some sort of progression that I don't recognize. It kind of reminds me of similar (not the SAME, though) offsets I discovered in ToEE mob files recently (see the offsets in my posts above) that MIGHT be pointers into different mob file blocks.

    P.S. Note that these offsets should not be directly applied to ToEE mob files. They appear to be different in ToEE files.

    Any thoughts, guys?
    - Agetian
     
    Last edited: Apr 10, 2005
  5. Agetian

    Agetian Attorney General Administrator

    Joined:
    Aug 14, 2004
    Messages:
    2,526
    Likes Received:
    0
    Holi-schmakeroli! I've tried taking a look at different ToEE mob files that were used for two-item chests, and so far I've found out that the offset 0x0038 definitely holds some information regarding the number of defined properties. Look what I've found out so far:

    0x0038 = 08: file size is 180 bytes
    0x0038 = 09: file size is 184 bytes
    0x0038 = 0B: file size is 192 bytes (makes sense, 0A would be 190 bytes)

    So, somehow 0x0038 defines the number of 4-byte properties. It's yet unknown where exactly these properties are (I would assume in the item property block - check my refined chest mob and its legend for reference).
    So, Arcanum and ToEE definitely have similarities in MOB files!

    On the other hand, 0x0042 which I listed as important for Arcanum is empty for ToEE. However, 0x0046/0x0047 pair looks a bit similar to 0x0042/0x0043 in Arcanum. Either way I can't deduce how 0x0046/0x0047 are constructed, even if they are somehow associated with the number of items in a MOB. Look at this:

    180-byte files: 0006 @ 0x0046
    184-byte files: 0006 @ 0x0046
    192-byte files: 800E @ 0x0046
    196-byte files: C00E @ 0x0046

    The fact that the offset values are the same for 180- and 184-byte files leads me to an assumption that it's not what we are looking for. It might be something else, entirely different.

    Dulcaoin, could you please investigate this thing a bit further? Try taking a closer look at what 0x0038 might hold for us, OK? I'm still kinda short on time, I'll keep you in on the down-low of ToEE as I investigate things further ;-)

    IMPORTANT P.S.: Let's post all the unclear stuff that is to be discovered by drawing parallels between Arcanum and ToEE here. Whenever we actually discover something that is confirmed to work for ToEE, we post it in the "MOB File Format Breakdown" thread, OK?

    - Agetian
     
  6. dulcaoin

    dulcaoin Established Member Veteran

    Joined:
    Mar 10, 2005
    Messages:
    213
    Likes Received:
    0
    Yeah, I had trouble myself finding it; I'd read online that it was still available in stores, but couldn't find a copy. Tried online, same problem.

    Then last night, on a whim of sorts, tried one of my favorite online ordering spots, and there it is. I have no idea what condition it's going to be in as far as ancillaries (box, case, manual), in fact it's probably pretty bare bones, perhaps even with the manual in a .PDF on the disc, but that's not a concern at this price and for this issue.

    http://www.gogamer.com/cgi-bin/GoGamer.storefront/SESSIONID/Product/View/001ARCA

    Go-gamer can be weird in its persistance. If Arcanum doesn't show up the first time, try to go a second (because the site will have assigned you a SESSIONID by then, I think) time to the same link right afterwards.

    Yeah, I noticed that. I'm thinking of going looking for the bink .dll online (or in other game directories -- Diablo likes to use Bink video) soon. My game is on slow-boat-from-China shipping, and I'm impatient. :)

    By going through the arcanum modding docs, I'm seeing more that's the same, rather than things that have changed. And Agetian has told me that the file header is exactly the same, which is really encouraging. I'm pretty certain that I'll gain AT LEAST as much information from this approach as it will end up costing me.

    I'm not blind to the fact that Arcanum has been really popular as well, more and more after its release. I didn't like the demo; the character art seemed too "stretched" vertically, and the characters seemed to "jump" around. But I've gotten used to the Troika art style now (TOEE ;-)), and perhaps I was too harsh on it. Besides, the market is seeing a derth of good quality RPGs, and the modding tools for Arcanum look to have pretty nice features (ouch, I know).

    -- d
     
    Last edited: Apr 11, 2005
  7. Firestrand

    Firestrand Member

    Joined:
    Mar 11, 2004
    Messages:
    79
    Likes Received:
    0
    Yeah, have you seen what it goes for new on Amazon?? I suggested to Troika that they contact their rep at Sierra and have it re-released at a lower price point. Might as well make money with what you have....

    -Firestrand
     
  8. Agetian

    Agetian Attorney General Administrator

    Joined:
    Aug 14, 2004
    Messages:
    2,526
    Likes Received:
    0
    Hey dulcaoin,

    It seems that Arcanum's Worlded definitely holds some answers for us. Just in case you didn't see my post above, I've already found a byte that may define the number of properties in a mob file. I'll give it a whirl today.

    - Agetian
     
  9. dulcaoin

    dulcaoin Established Member Veteran

    Joined:
    Mar 10, 2005
    Messages:
    213
    Likes Received:
    0
    Yes, I saw that, and I tried to make something of it, but wasn't entirely able to.

    Best I did was figure out some of the format of the clipping.cgf file in the tutorial 1 directory as I've been learning the 010 hex editor (good stuff; I'll likely buy this one soon too, once my trial period runs out).

    -- d
     
  10. Agetian

    Agetian Attorney General Administrator

    Joined:
    Aug 14, 2004
    Messages:
    2,526
    Likes Received:
    0
    Hey dulcaoin,

    I'd be glad to hear any news about the clipping.cgf thing. I've assigned a friend of mine to work a li'l bit on the 2D Map Discombobulator so he's gonna stuff it with features. I don't have time to work on the clipping issues myself (it's personal between me and the MOB format :))))) but I can direct any data regarding that to a friend of mine who may work something out of that.

    I tried modding the 0x0038 byte a little bit - h*ll it's a crazy thing! It doesn't seem to work just on its own - sometimes you can even give it a smaller or bigger value and the game doesn't crash, but as soon as you start adding/removing the bytes within the property block (say, there were six 4-byte blocks there and 0x0038=06, so I set 0x0038 to 05 and remove the last 4-byte block) the game crashes, even if 0x0038 seems to be properly aligned. I wanna ask myself "WTF???" but I know it won't help. :))

    There's also a precarious byte at 0x003E. It's usually C2 but it can also be C0 or something else (rarely). I wonder what it does, since it seems to be somehow related to our thing under discussion (0x0038).

    Anyway, I'll keep you updated on my progress. Please post anything you find out or PM me if you're unsure it should be directly posted here, OK?

    - Agetian
     
  11. dulcaoin

    dulcaoin Established Member Veteran

    Joined:
    Mar 10, 2005
    Messages:
    213
    Likes Received:
    0

    Here's an exception for byte 0x38 that bothers me:

    The wooden orcish shield (G_50b4...) and the longsword (G_55d0) both have a 6 in 0x38, (and a 1 at 0x40), and are 136 bytes long; there is also the small gold key (G_9509...) also with 6 and 1, but it's 144 bytes long. :-(

    I was hoping the two word values together determined file size, but that's apparently not so.

    There's more to the puzzle...

    ----

    A couple hours of work, and little solid has come up, though I DID find my "standard" value offsets (location, off_x, off_y, etc.) for npcs, and sketched out more info on the rest of the objects on Tutorial Map 1 (including the "backlink" GUIDs you mentioned). I'm REALLY curious what a chest created from arcanum looks like compared to this one. And a key, and perhaps a sword. I'm anxious for the new tools to show up...


    -- dulcaoin
     
    Last edited: Apr 11, 2005
  12. Agetian

    Agetian Attorney General Administrator

    Joined:
    Aug 14, 2004
    Messages:
    2,526
    Likes Received:
    0
    Hi, dulcaoin!

    I gave 010 Editor a try - you're right, it's awesome!

    Yeah, I agree that there's a lot more to the puzzle than just the byte 0x0038. So anyway, I'll give Arcanum files a try today - I'll try and see what those files with items look like (something like a chest). I'll keep you in on the updates.

    P.S. Are the standard offsets you mentioned (e.g. off_x, off_y, etc.) the same for NPCs or no? This might be important as well.

    - Agetian
     
  13. dulcaoin

    dulcaoin Established Member Veteran

    Joined:
    Mar 10, 2005
    Messages:
    213
    Likes Received:
    0
    Yes, I'm building up a library of templates already; it's very nice because it keeps track of what you've discovered so far, without handwritten notes. Very handy.

    The ONE feature I REALLY wish it had would be for me to be able to select a byte, word, longword, or quad, and then right-click and just "name" the variable for me, and the template editor would add it for me; including any "padding" as a byte unknown_XXX[x]; in front of it for me. That would be VERY helpful (if there was already an unknown_XXX array "holding" those bytes, then I'd like to see it "auto-split" the array into two bookending arrays around my value for me.

    I think I'm getting spoiled at this point, obviously. :-D

    ----

    The attributes for the npc (location, off_x, off_y, etc.) are at a different offset, but seem to be in the same order. [No, having the offsets available to me did NOT help in deciphering byte 0x38 at all :-(]

    As a matter of interest, those items all change value (as you can imagine) as the animal does, so it was a LITTLE hard to figure out at first where the values were; I had to work from the non-changing attrs (radius, height) backwards. The damned rat almost TOOK ME OUT while I was working on this. :) I kept moving around the area where it had seemed to "start" from, so I could keep the numbers at least close.

    Interesting item #2: Radius has been the first and only thing I've found so far that seems to be in both the protos entry for an object, and in the .mob file for the object.

    And #3: I was shocked to discover that "Tutorial Room 1", "Tutorial Room 2", etc., are NOT scenery items as I'd guessed from the names, but actually non-drawing npc's placed in the rooms as "heartbeat generators". They are how the various tutorial messages pop up as you proceed through the tutorial portion of the game. This is actually a disappointment to me. I have felt that creating invisible critters to be used as triggers was extremely kludgey, and was hoping we could move past that once more was known from the formats. To find out that this is the "as designed" method for this is a bummer. The arcanum engine actually had true triggers that a PC would walk across to cause things to happen. I guess my main issue is that they didn't create an actual (mostly empty) object for triggers rather than use the heavy-overhead NPC object. OTOH, the argument is likely just academic, as it seems most every object is the same large structure anyway.

    Agetian, I'm sure you already knew all this last part already, as that is how your item generation has been done in the past, right? (create an invis npc to generate events properly on a level).


    -- dulcaoin
     
  14. Agetian

    Agetian Attorney General Administrator

    Joined:
    Aug 14, 2004
    Messages:
    2,526
    Likes Received:
    0
    Yeah, but I haven't yet seen an editor that would support this. :-(
    Hex Workshop is the second best after 010 Editor but it doesn't have this feature as well.

    I have no luck with 0x0038 either. Danged thing! Well, we'll get it in the end - I have some stuff to test for tomorrow, stay tuned. :)

    Oh, my! Good job figuring 'em out! I imagine how crazy it was!

    Yep, noticed that, too. The MOB file doesn't seem to double the protos.tab in very many respects.

    Yes, I don't like the idea either. There's another stunning thing - you can't place an inventory item (weapon, armor, and so on) on the floor, you are obligated to have some container for it (or a NPC that would carry it). That's how it works. Just creating a weapon MOB file and defining the coordinates doesn't work. The coordinates for all those MOB files are completely useless. Such MOBs have to be referenced by a container object and need to have a back-reference in the container object's MOB. Anyway, seems that we'll have to live with this way of doing things (that's how I worked out the creature spawning via scripted MOBs originally).

    I was glad to hear the news, keep it up! I'll try some more stuff out tomorrow, esp. the Arcanum containers!

    P.S. I have a question: did you find out what the rest of the crap might be for in the NPC mob files? It's a really large file (499 bytes), having a very large "tail" after the property block and the item block (where the key is referenced). Do you have any idea what that is for?

    - Agetian
     
    Last edited: Apr 11, 2005
  15. dulcaoin

    dulcaoin Established Member Veteran

    Joined:
    Mar 10, 2005
    Messages:
    213
    Likes Received:
    0
    so does Hex Workshop support this "templates" idea too?

    How sure are you about all that? It hasn't worked, so you're sure, or you have a definite proof?

    I didn't dig much, but the fields for an NPC are the most numerous of all, so it makes sense it'd have the largest .MOB file.

    The most curious thing I found was a series of about 20 bytes that was completely duplicated near the end. That definitely bears some scrutiny, eventually.

    -- dul
     
Our Host!