Thief 1/2 & SShock 2: DDFix and Enhanced Resolution Patch - discussion

[Timeslip]

I've been playing for a bit, and haven't come across any more major problems, (alpha'd textures that use mipmaps still can't be replaced, but there aren't many of them around,) so I'll replace 1.2.7 with 1.3.8 as the current version later today. (I'll leave Thief2Extensions=0 in the default ini, and leave 1.1.1 and 1.2.7 up for download too, just in case.)

Ahhh, okay. I was thinking that you'd hooked directly into the filesystem routines. How about this then-- Would it be possible to, at game start, automatically generate (if one doesn't already exist) a manifest file that contains an MD5 checksum of every original texture for which a replacement exists in the TXT32 (or whatever) folder? Then when you get handed the raw texture data, you'd have a mechanism for determining exactly which file it is.

Far too complicated for my tastes.

Then again, my ReadFile hook idea didn't work out either, (it works fine for pcx's, but breaks gifs,) so I have no better ideas.

Well, I guess I'm too late, but hiatus, I've got a zip version of ddfix 1.1.1 if you want it. (that way you don't have to have ddfix "install", cuz then it puts a installer registry entry and you have to uninstall it to get rid of it. I prefer the default ddfix way, non-installer.)

I already gave the zip version. It was the installer/GUI that I didn't have.

There isn't an installer anymore anyway. You wouldn't believe the confusion that the instructions 'extract the contents of the zip file to morrowinds installation directory' cause some people over on the morrowind forums, (just look for all the obse installation help threads. ) so I generally make installers for everything. Luckly you all seem to be a bit more computer literate over here.

Damn, so much updates! =o I haven't read all this because it is too much technical and stuff... but from what I read, I get the jist of it, which is that this new ddfix beta allows better world textures?

Yup. Lets you replace any .pcx, .bmp or .gif texture with a 32 bit replacement with no size limits. (Edit: well, there's still the hardware limit of 4096x4096 for DX9 cards and 8192x8192 for DX10, of course. )

Speaking of which, is anyone actually intending to make a texture replacer or FM that uses this?

[ZylonBane]

Speaking of which, is anyone actually intending to make a texture replacer or FM that uses this?

If you (or someone) can make an SS2 version of this, it will find immediate application in upgrading all the horribly low-res "tech" screens.

BTW, does this play well with animated textures? If memory serves, the Dark Engine has a 64K(?) limit on animated texture sequences.

[Timeslip]

BTW, does this play well with animated textures? If memory serves, the Dark Engine has a 64K(?) limit on animated texture sequences.

Not sure. Are there any animated textures around that don't use alpha? Replacing the clouds works, but that's just one texture moving and so presumably it's not animated in the conventional sense.

[Nameless Voice]

You never did say if you would be able to easily find the appropriate code for Shock2.exe / Thief.exe / DromEd.exe if you had .exe files?

Or give specifics so that someone who does have them could find the appropriate code locations.

[ZylonBane]

Not sure. Are there any anmated textures around that don't use alpha?Replacing the clouds works, but that's just one texture moving and so presumably it's not animated in the conventional sense.

I'm at work right now so I can't cite any specific textures, but just search in the FAM directory for anything ending with _1. That'll be part of an animated texture sequence. Dark supports up to 20 frames per sequence.

[Nameless Voice]

In thief 2: nearly everything in the "Mech" texture family.

Near the very start of Framed (Thief 2 mission 3 (technically Miss4.mis)), there is a room full of machinery that has a lot of animated textures in it.

[Timeslip]

You never did say if you would be able to easily find the appropriate code for Shock2.exe / Thief.exe / DromEd.exe if you had .exe files?

Or give specifics so that someone who does have them could find the appropriate code locations.

There are 7 code locations I modify in thief 2. You'd need to find the equivelents in the other exe's. Assuming that the dark engine hasn't been modified in any of those places, it could be as simple as looking at each address in thief 2, searching for a few dozen bytes around the area in the other exes and then changing the addresses. If the engine behaves differently, there'll need to be code changes.

//Where world textures first get loaded
SafeWrite32(0x0041D948, (DWORD)&CopyTex - 0x0041D94C);
//Where mip maps for world textures are generated
SafeWrite32(0x0041D96B, (DWORD)&MipMap - 0x0041D96F);
//Execution reaches here when thief tries to copy a system memory texture into video ram.
SafeWrite32(0x0062A67C, (DWORD)&CreateVidTextureHook);
//Hooks the level loading code to release all loaded textures
SafeWrite32(0x0040FCA4, (DWORD)&LevelStartHook - 0x0040FCA8);
//execution reaches here when a thief 2 is looking for an unused video mem surface to reuse and finds one. I free it, and then send execution back to where it would be if no suitable surface was found.
SafeWrite32(0x005BD427, (DWORD)&TextureExistsHook -  0x005BD42B);
//Override CreateFile to check for .override files first.
SafeWrite32(0x0060015C, (DWORD)&CreateFileHook);
//Not related to the texture fix; this is a function I force to return immediatly because it tries to blt from a null pointer each time you alt tab, which screws up ida
SafeWrite32(0x00591440, 0x000008c2);

I'm at work right now so I can't cite any specific textures, but just search in the FAM directory for anything ending with _1. That'll be part of an animated texture sequence. Dark supports up to 20 frames per sequence.

Yup, they appear to be working. I'm not 100% sure that they all are though; I didn't check many.

[Hiatus]

You never did say if you would be able to easily find the appropriate code for Shock2.exe / Thief.exe / DromEd.exe if you had .exe files?

AFAIR Timeslip mentioned that he had Thief (I assume Dark Project, not Gold) copy lying around somewhere, waiting to be found (Timeslip, have you found it?). As for SS2, I don't think he has a copy of it (Timeslip: why not? ) so we should get him one, really (and TG as well).

I'm afraid we won't be able to find these mem locations/offsets for other games/editors ourselves (which would be a real shame, now that we're so far along (ok, Timeslip is )).

There will be stuttering when an object comes into view for the first time; whereas world textures are all loaded from disc at the start of a mission, object textures will only be loaded when used for the first time. Once loaded, they stay in memory until the mission ends.

you mean there'll be stuttering with *all* objects in a level, or just with ones with replaced textures? Will the stuttering be severe? Would you think of any way to reduce/eliminate it somehow?

[Timeslip]

Edit: New download link.

you mean there'll be stuttering with *all* objects in a level, or just with ones with replaced textures? Will the stuttering be severe? Would you think of any way to reduce/eliminate it somehow?

Just the ones with replaced textures. Fixing it completely would require finding additional hooks for where the object textures are loaded at the start of each level, which is time consuming. (And was what I was moaning about having to do earlier. )

You can reduce it by making sure that multiple retextured objects don't come on screen at once, or by using smaller textures.

[ZylonBane]

Just the ones with replaced textures. Fixing it completely would require finding additional hooks for where the object textures are loaded at the start of each level, which is time consuming.

Would doing an initial preload of everything in the high-def object folder be... dangerous?

[Timeslip]

@BikerDude: Mind if I use the screenshots from your opening post for my website?

Would doing an initial preload of everything in the high-def object folder be... dangerous?

Depends what was in there. Thief ate 1GB of ram just from preloading the stuff from mission 1 when I replaced all objects with 1024x1024 textures, and I doubt that that mission uses more than half the objects in the game. With slightly saner replacers it would be a possibility.

Proper hooks the same as what I do for world textures would be the best option, as soon as I fiond where to put them.

[Hiatus]

whereas world textures are all loaded from disc at the start of a mission, object textures will only be loaded when used for the first time.

Q: and in vanilla T2, how obj textures are handled? Also preloaded at level start, as world textures? (hence no noticable stuttering on 1st seeing them? and of course smaller size). Just making sure..

[Timeslip]

Q: and in vanilla T2, how obj textures are handled? Also preloaded at level start, as world textures? (hence no noticable stuttering on 1st seeing them? and of course smaller size). Just making sure..

Yes, all textures used in the mission, both world and object, are loaded into system ram at level load. They are then copied into vram when they first appear on screen, and are removed from vram when they are no longer on screen.

[ZylonBane]

Interestingly, the HRP/SD Duke project for Duke Nukem 3D faces this exact issue. Their port of the Duke engine offers the choice between preloading all textures, or doing on-demand loading. The former gives you level load times sometimes over a minute, while the latter gives heavy stuttering every time you enter a new area. Scary stuff.

That being said, high-res object textures would be quite the rarity. The overwhelming majority of the objects in Thief are so small that 256x256 is more than enough (as the Thief Enhancement Pack aptly demonstrates). And the BIN format allows at least 16 textures per object, so pretty much the only thing you'd need high res for would be large flat objects-- paintings, signs, etc.

So a naive, brute-force preload algorithm might be entirely adequate in this case.

[Assidragon]

I have mapped the address changes needed to make it work with Shock2.

Thief2 addy -> Shock2 addy.

0041D948 -> 00419B5D (in Shock2, it originally calls: 0x00419710)
0041D96B -> 00419B82 (in Shock2, it originally calls: 0x004198A0)
0062A67C -> 0065A768 (in Shock2, it  original points to: 0x005E3360)
0040FCA4 -> 0040D136 (in Shock2, it  originally points to: 0x00418F60)
005BD427 -> 005E344F (originally calls: 0x005E34F8)
0060015C -> 0062A168
00591440 -> 005B89C0

Note the shock2 addys are not "adjusted" so to speak - those addresses point to the opcode before the to-be-overridden things themselves.

Looking forward seeing SS2 being supported.

[Hiatus]

@Assidragon: do you have TDP and/or TG as well? If yes, could you find the pointer mappings there as well (in TDP v1.33 and TG v1.37)?

also, let's not forget about editor .exe's for all DE games...

[Assidragon]

I don't own either. But ask and ye shell receive. I found a link to a Thief Gold executable in this thread, so I downloaded it and dug up a few things.

Thief2 addy -> Thief Gold addy.

0041D948 -> 00418BED (in TG originally calls: 0x004187A0)
0041D96B -> 00418C10 (in TG originally calls: 0x00418930)
0062A67C -> 005DA9B4 (in TG original points to function: 0x00576880)
0040FCA4 -> 0040C82D (in TG originally points to: 0x00417FF0)
005BD427 -> 00576976 (in TG originally calls: 0x00576A1F)
0060015C -> 005B614C
00591440 -> 00548480

Now I would not mind if someone who actually ran the thing once would double-check that, but I don't see any reason why they would be wrong (apart from me screwing up, but I trust myself ). Same story like with the Shock2 ones: these addresses point to the opcode, not to the bytes after them that need to be overwritten.

[Hiatus]

@Assidragon: big thanks for the pointers! Hopefully Timeslip will incorporate support for SS2/TG having this info and, it will actually work there as well (fingers crossed).

[sNeaksieGarrett]

I'm sorry, I did not make myself clear... I meant an extracted version of 1.1.1 that I packaged in a zip that IS the GUI version

Yup. Lets you replace any .pcx, .bmp or .gif texture with a 32 bit replacement with no size limits

Fantastic! So, it IS possible to change the world textures... or am I understanding wrong?

If this is possible, then could someone possibly up the resolution on TG/T2 textures instead of making new ones?

Also, nice redone of the ddfix links on your page Timeslip.... I kept thinking it'd be cool to have a description, and almost suggsted it, but I thought better of it because I figured everyone would obviously know what ddfix is (assuming everyone who knows of ddfix is from this forum)

Woops, just noticed... can't do it for Thief gold.

[Hiatus]

@sNeaksieGarrett: please up it to rapidshare or similar (I'll also PM you with email addy). Also, Timeslip could use it to add missing 1.1.1 GUI (which he didn't keep and thus doesn't currently have) to his site (if he wants to).

So, it IS possible to change the world textures... or am I understanding wrong?

yes, world AND game object textures - praise Timeslip

[sNeaksieGarrett]

All hail the Master Builder of DDFIX!

[ZylonBane]

If this is possible, then could someone possibly up the resolution on TG/T2 textures instead of making new ones?

Ummm... wouldn't a texture of higher resolution be, in fact, a "new one"?

[Nameless Voice]

Why do you need the GUI version?
I've never used the GUI, though it sounds like it will be needed for replacing textures. Which v1.1.1 didn't support.

I'm already planning to start upgrading Thief 1 / 2 world textures in the EP.
I'm not going to start right now, though!

[sNeaksieGarrett]

Ummm... wouldn't a texture of higher resolution be, in fact, a "new one"?

Um, I'm no texture guru. *shrugs* What I meant was, the same thief textures (not recreating them from scratch) with high res or whatever. Better quality on the originals? Or is that impossible? Like I said, I dunno about textures. I think maybe I'm getting confused with color depth versus dimensions.

[ZylonBane]

*headdesk*