Some tools for working with Thief 3 (and Deus Ex 2) files (WIP)

[snobel]

This thread is for posting some tools for extracting and changing content in various file formats used by Thief 3, Deus Ex 2 and possibly other Unreal 2 based games.

They've all been created to assist either Sneaky Upgrade work, or other people's modding or FM projects. As such I don't expect them to see much additional use, and so I'm not inclined to spend too much time on documentation. If you'd like to use one or more of these tools, and can't figure out how to use them, post in this thread or PM me.

They're all command line tools - sorry for the inconvenience...

• csc-tool
• mlb-tool
• ibt-tool
• cubemap-tool
• ibt_xbt_ibd_texture_extract
• cel_upscale

A zip of all published tools can be downloaded here. Download individual tools from the respective posts. <- currently not available

[snobel]

csc-tool: Used to fix or augment the existing CSC files, or to apply changes for use in FMs.

A full decompilation/compilation could be used to add subtitles in a language not covered by the original game. For FMs, the most convenient usage is probably to produce an override SchemaMetafile_HardDrive.csc file, containing changed and added schemas and sounds.
If the exe is renamed to Thief3SoundAddOn.exe it will change its function to mimick the tool of the same name that is used by the FM "'All the World's a Stage" - but it can change existing schemas and sounds in addition to adding new ones.

Current version: 0.43.

Download here.

Usage:
csc-tool --compile|-c   [--in|-i SCHEMA-DIR] [--out|-o CSC-DIR]
                        [--csc-suffix|-s STRING]
csc-tool --decompile|-d [--in|-i CSC-DIR] [--out|-o SCHEMA-DIR]
csc-tool --modify|-m    [--in|-i DIR] [--out|-o CSC-DIR]
                        [--csc-suffix|-s STRING] [--hd-only|-H]
csc-tool --help|-h

Commands:
--compile|-c           Read discrete schemas, sounds and LAD files, write CSC
                       block files.
--decompile|-d         Read CSC files, write discrete schemas, sounds and LAD
                       files.
--modify|-m            Read CSC files, apply replacement schemas, sounds and LAD
                       files, write modified CSC files.
Options:
--in|-i DIR            Schemas/sounds/LADs and/or CSC files as input. If not
                       specified, the current directory is used.
--out|-o DIR           CSC files or schemas/sounds/LADs as output. If not
                       specified, the current directory is used.
--csc-suffix|-s STRING This string will be appended to output CSC filenames.
--hd-only|-H           When used with modify, apply only to the HardDrive CSC
                       file. Schemas specifying DVD storage will go into the
                       HardDrive CSC instead.
--version|-V           Print version info and exit.
--help|-h              Print this message and exit.

TODO: Examples.

[snobel]

mlb-tool: Mainly used to produce override MLB files to fix or improve existing materials.

Current version: 0.4. This tool was formerly known as gen_matlib.

Download here.

Basic usage:
mlb-tool --dump|-d     --in|-i MLB-FILE [--out|-o CSV-FILE]
                       [--categories|-c TXT-FILE|t3|dx2]
mlb-tool --generate|-g --in|-i CSV-FILE [--out|-o MLB-FILE]
                       [--categories|-c TXT-FILE|t3|dx2]
                       [--orig-mlbs|-m MLB-DIR]
mlb-tool --help|-h

Commands:
--dump|-d            Read a matlib file, write the contents to a CSV file.
--generate|-g        Read a CSV file containing material definitions, generate
                     a matlib file from the contents. To modify existing
                     materials, use the --orig-mlbs option.
Options:
--in|-i FILE         An existing MLB file for --dump, an existing CSV file for
                     --generate.
--out|-o FILE|DIR    A new CSV file for --dump, a new MLB file for --generate.
                     If the path supplied is a folder, or if this option is
                     omitted, the filename will be derived from the input name.
                     Existing files will be overwritten without warning.
--orig-mlbs|-m DIR   Path to existing matlib files. When generating a matlib,
                     empty fields in the CSV will be filled in from existing
                     materials of the same names, which are read from these
                     files. Without this option, empty fields get default
                     values.
--categories|-c FILE In CSV files, use (and expect) category names from a
                     supplied text file. The file must contain each name on a
                     separate line. Names are case insensitive. Lines starting
                     with "//" and empty lines are ignored.
                     For T3 and DX2 no file is needed - "t3" or "dx2" can be
                     entered instead. Without this option, T3 category names
                     will be used.
--unknown|-u         Include four unknown values, which may or may not be
                     material parameters. Column headers in CSV dumps refer to
                     the hex offset relative to the start of the material data
                     blocks in the MLB files.
--version|-V         Print version info and exit.
--help|-h            Print this message and exit.

TODO: Examples.

[snobel]

Reserved for ibt-tool.

[snobel]

cubemap-tool: Basic tool, which will split a cubemap DDS file into six DDS files - one for each face, or join six DDS files into a cubemap texture.

Each face is extracted into the same folder as the cubemap, with a number postfix added to the filename. For joining, the same naming is expected.

Version: 0.2.

Download here.

Usage: cubemap-tool --split|--join <FILE> ...

[snobel]

ibt_xbt_ibd_texture_extract: Very primitive tool, which will extract all textures from all block files found in the current folder.

The output DDS files are organized after file name and compression types, inside a subfolder named "Extracted" in the current folder. Extraction works with block files from T3 PC (IBT), T3 XBox (XBT) and DX2 (IBD), except for uncompressed textures in XBTs, which come out corrupted. Also, for cubemaps, only the first face is extracted(?)

This tool should ideally be superceded by an improved ibt-tool.

Version: 0.1.

Download here.

[snobel]

cel_upscale: Basic tool to update a CEL file to match an upscaled font texture.

A Thief 3 font consists of a DDS texture and a textual CEL file defining the characters. To use an upscaled font texture, the values in the CEL file must be upscaled as well. Running this tool with a scaling factor and a CEL file as input will produce the resulting file - as "scaled.cel" in the current folder, it must be manually renamed before use.

The scaling factor is added to the end of the file for use by the Sneaky Upgrade patch (which is required for upscaled font textures to work). The Visible Upgrade doesn't have this patch, so upscaled fonts won't work with Deus Ex 2.

This tool should ideally be replaced by an equivalent function added to FontTaffer.

Version: 0.1.

Download here.

Usage: cel_upscale FACTOR FILE

[snobel]

Reserved for whatever I forgot.

[New Horizon]

Nice!

[ponaromixxx]

Hello, is there any software left from this page by any chance?

[snobel]

Dropbox has disabled all the tools because of excessive traffic. That's usually a temporary thing but apparently not this time. I'll get them up on Mod DB tomorrow - stay tuned.

[ponaromixxx]

Thanks!

[snobel]

Here's the zip on Mod DB - I've updated the first post as well.

Out of curiosity, what do you need them for?

[ponaromixxx]

Here's the zip on Mod DB - I've updated the first post as well.

Out of curiosity, what do you need them for?

Re-sound on Russian.

There was a problem with this file, it seems to be exported with an error.
Download

[snobel]

What was the problem? The .ogg file seems fine?

[ponaromixxx]

What was the problem? The .ogg file seems fine?

Not normal, the file is cut off and it contains part of the sound and part of the .lip

[snobel]

OK, I'll have a look. It may take some days, as I'm busy with something else.

Can you post the exact command you used to extract?

[ponaromixxx]

OK, I'll have a look. It may take some days, as I'm busy with something else.

Can you post the exact command you used to extract?

csc-tool -d -i "G:\Program Files\Steam\SteamApps\common\Thief Deadly Shadows_Editor\Content\T3\Sounds\Extr" -o Out

Thanks!

[ponaromixxx]

I figured it out, my SchemaMetafile_DVD2.csc file was damaged(
Thanks for helping! And sorry for the trouble!

[snobel]

No worries - good luck with your project.

[Jahouch]

Hi, I'm pretty new to modding. But thanks to your tools I finaly have a hang on the basics of command lines wich is all a new world for me. It's all pretty interesting, really.

Anyway, I stumbled upon a problem, it may sound stupid but I want to work with the textures from Deus Ex 2, do some heavy editing with them, like a complete revamp.
I used mlb-tool to generate a matlib out of a csv file and now...well, I don't know what to do with this mlb file. Is there a way to extract the textures from it in a dds format ? Or am I completly wrong and should look for another tool if mlb-tool is not for the textures ? If that's the case, do you know of a program that could extract the textures from the game and recompile them ?

Thanks in advance for your answer, have a nice day !

[snobel]

If you want to extract and edit the textures (i.e. the image files), use this tool on the IBD files.

Changed/replaced textures must be placed in Textures\DynamicallyLoaded, where they'll override those in the IBDs. I think there's a third-party tool which lets you replace textures in IBDs but the name escapes me right now.

mlb-tool is only needed if you want to fix or change the texture names or parameters of materials. A replacement MLB file must be placed in MatLib\DynamicallyLoaded.

[Jahouch]

Thanks a lot, I'll do that when I regain access to my PC !
Can't wait to try it.

[darkpadawan]

Hi, snobel:

I'm trying to use csc-tool to translate the subtitles in-game, but I have a problem.

-I use csc-tool -d (no arguments) in the ..\Content\T3\Sounds folder, and get the 'audio' and 'schemas' folders. But 'schemas' contains a 'conversations' subfolder, but not in 'audio'

-I can translate the *sch files in 'schemas'.

-I use csc-tool -c (again, no arguments), to compile, but then the subtitles (and voices) in-game disappear.

What I am doing wrong?
Thanks.

[snobel]

From the description, you're not doing anything wrong.

Which language are you changing? (And, out of curiosity, which language are you replacing it with?)

Can you give me an example of a schema file that you've changed, but the change failed in-game? It's best if it's something easy to test, like something early in the Inn or Castle missions. I'll try to reproduce the problem and if I can, I'll try fixing the bug.