Textures in the Mass Effect Trilogy (original) (raw)

NOTE: This page has been updated for Legendary Edition.

Textures in the Mass Effect Trilogy/Legendary Edition, and Unreal Engine 3 in general, are pretty tricky. They are one of several types of objects that heavily depend on file offsets to work properly, in that, they reference another file at an exact position.

Some quick information on textures:

• Mass Effect (Original Trilogy) games support up to 13 mips, which yields a maximum texture size of 4096x4096 (2^12), but without changing some configuration files, you will rarely see textures above 1024. Mass Effect Legendary Edition supports up to 13 mips, or 8K textures. You will not need to change configuration files to use them.
• Textures dimensions must be powers of two, however you will see a few that are not powers of two. Our tools do not support any textures whose dimensions are not powers of two.
• The normalmap that the game uses for a material is always one mip size lower than the diffuse. This means the maximum effective norm texture is 2048x2048 in OT and 4096x4096 for LE. You can install a higher mip norm, but they will waste a huge amount of memory, so don't bother.
• Textures are loaded according to the Unreal Engine 3 memory system. This means that when a package file is loading, any objects with the same full name already in memory will use the copy in memory rather than the one in the loading file. This also means that if you have duplicate textures with different full paths, they will use additional memory. Make sure your object paths are the same across all files that share the same texture.
• Texture data is mirrored in application memory as well as stored in GPU memory, according to the maximum LOD settings (and LOD bias). This means that textures can eat up a huge amount of process memory.
  • (ORIGINAL TRILOGY ONLY) The Mass Effect games will crash if they try to use more than 3.5GB (ME1 will crash if going above 2GB without patching the executable). The ALOT texture mods are the most popular mods in the modding scene, so if your mod + ALOT crashes the game, users will be unhappy. There are many features of ALOT that developers can use to tamp down their memory usage.
• Texture data that is package stored will be loaded when a package is loaded. Texture data that has external mips will be streamed in after the package is set to visible, the lower mips will be loaded with the package.

Mass Effect 1 (Original Trilogy)

Textures in Mass Effect are a real nightmare, as the externally stored textures are stored in UPK package files. As UPK package files often can change their file offsets (for example, adding an export will add an entry to the export table - which shifts all data after the table), it is extremely easy to break all of the inbound references. If you've tried to save packages in Legendary Explorer and are greeted with a message saying you cannot save this package because of texture references, this is exactly why, because that file has known inbound references. The only way to safely update textures in these files is by global replacement, with a tool like Mass Effect Modder.

In ME1 there is a concept of Master and Slave textures. Slave textures have externally referenced mips (extZLib, extUnc) on the higher values. The package name can be determined by traversing up the full path of the object and using the top level package as the package name to search for.

In the above image, the top texture is the slave texture, and the bottom texture is the master package file's version. You can see the slave version has the external mips defined as extLZO, where as the master one has them as pccLZO. The offsets are into the package file itself, rather than into the export. These files are (for some reason) loaded and parsed as package files, so they must behave and look like normal package files.

If a slave texture has multiple package export parents, such as BIOT_STA20_Presidium.Eyeballs.Eyes_Diff (Eyes_Diff is the texture), In BIOT_STA20_Presidium will be a package export named Eyeballs, with the master texture residing within it. Otherwise, it will directly be in the root.

Due to this complexity, as well as the waning Original Trilogy scene, master-texture replacement tools were never finished.

Mass Effect has a rather poor implementation of texture loading, and as such, will often hitch and stutter as higher resolution textures are loaded in, which due to Mass Effect's filesystem, happens all the time.

Mass Effect 2, Mass Effect 3, All 3 Mass Effect Legendary Edition games

Textures in other games are much easier to work with than Mass Effect, in that the external texture data is stored in a Texture File Cache, or TFC. TFC files begin with 16 bytes representing their GUID, and are followed directly by headerless texture data concatenated one after the other.

These textures are referenced by data offsets in the external mips of a texture (storage types of extUnc, extZlib, extLZO, or extOodle (LE)), along with the filename and guid as properties of the texture. Only mips higher than 6 are stored externally, all lower ones are stored locally in the package. Mips above the 6th one, if stored locally, must be in an export that contains the NeverStream flag, with its value set to true, otherwise the package will crash the game when LODs are raised and the file is loaded (OT only). You will see an I/O failure on the package file in ME3Logger/DebugLogger output if this condition occurs.

In DLC folders, there are restrictions on how TFCs can be named, depending on the game.

• LE1: TFCs can be named anything, with any amount of TFCs in the DLC folder. This feature will ONLY work if the Autoload ASI is installed (v4 and higher).
• ME2, LE2: TFCs can be named anything, with any amount of TFCs. The game will scan DLC folders for TFC files on DLC mount.
• ME3, LE3: TFC files must be named Textures_[DLCFolderName].tfc. The game will not look for any TFC files in a DLC folder beyond this one in the DLC's CookedPCConsole directory.

In all games, TFCs can only be up to 2GiB in size, and textures in them can be stored in several ways:

• Uncompressed (extUnc) - TFC files in ME3 SFARs are all extUnc, as the SFAR itself provides the compression for the TFC
• Compressed with LZO (extLZO, ME2)
• Compressed with Zlib (extZlib, ME3)
• Compressed with Oodle (extOodle, LE games)

Compression of textures with LZO/Zlib/Oodle has no effect on the texture quality, as these algorithms are lossless.

Textures in these games are loaded on a background thread, so slow disk access is typically not a big issue. Mass Effect 2 seems to have issues with texture loading causing FaceFX to desynchronize from the audio due to FaceFX not being synced up with anything.

Empty Mips

You may see the top mips of a texture stored as 'empty', and have no data. When the games were compiled, the source art sizes were often larger than the ones that could be used by the default Texture Level of Detail (LOD) settings. In ME2 and ME3 there are no graphical options, however you can think of LODs as ME1's texture quality settings - lower LODs = lower resolution textures. You can look at the OriginalSizeX and OriginalSizeY properties to see the original source asset size before game compilation. In Legendary Edition games, the shipping LODs were much higher, so it is much more rare to find empty mips.

When the game was compiled, any textures above the maximum vanilla LODs were replaced with stubs known as empty mips. In a vanilla setup, this causes no problems as those mips are never used. However, if LODs are raised, these empty mips are attempted to be used, which are invalid. In ME2/3 this can occasionally crash the game, but most times appears to be ignored. In ME1 loading any empty 'unused' mip will cause the game to assert a crash. In Legendary Edition, all texture tools use a workaround (LODBias) to allow the engine to request a higher mip, so we don't need to worry about empty mips.

In the above image, you can see the assertion message for ME1. The filename not found strings are because there is no .pdb symbols file for the game (as it's a release build), so there is no referenced filename available.

Problems with offsets

Because of these offset systems, it is very easy to break textures. A broken texture will appear 100% black in game, and is caused by the referenced offset not pointing to the start of texture data - each texture starts with a texture magic number, and if that magic number is not correct, the game completely ignores the texture data and uses a black texture instead.

This offset system is also why modding tools prevent installation of mods after installation of texture mods. In Original Trilogy, texture mods can change huge numbers of offsets, so installing or replacing a file will lead to a situation where all the existing external pointers are wrong immediately. When installing ALOT for example, the original TFC files are recreated with higher resolution versions assets, which means all pointers must be updated. Due to the sheer complexity of texture modding, the developers of ALOT/Mass Effect Modder do not support installation of files after texture modding the game.

For Legendary Edition, this is a lesser concern, as we don't have to worry about empty mips breaking games, and tooling was changed to only make new TFCs. However, mismatches can lead to undesirable game behavior and is not a supported configuration.

Mods should NEVER include TexturesMEM**.tfc files or references to them. Accessing a TFC (and other offset-based lookups) out of bounds of the file will result in a game crash. As these TFC are created by MEM, if a user installs a texture mod, but an existing mod points out of bounds of it (due to it being based on a different texture installation), the game will crash loading the texture.