xoreos is a FLOSS project aiming to reimplement BioWare's Aurora engine (and derivatives), covering their games starting with Neverwinter Nights and potentially up to Dragon Age II. This post gives a short update on the current progress.
Note: This is a cross-post of a news item on the xoreos website.
During the last 2 weeks, I implemented basic support for Neverwinter Nights 2 areas and campaigns/modules. In particular:
As a result, areas in Neverwinter Nights 2 are now in a similarly supported state as areas in the two Knights of the Old Republic games.
So, here's a few interesting things and/or issues with them.
Tiles and Metatiles
Like the first Neverwinter Nights game, areas in Neverwinter Nights 2 are (or can be, more about that later) made up of tiles. Meaning, a grid of models define the general structure: piece of indoors floor with walls, corners, windows and/or doors. Unlike the first game, though, there's also metatitles, which are bigger models spanning the space of several regular tiles. This is used, for example, for the piece of the cave where you wake up in the Mask of the Betrayer expansion.
xoreos correctly loads and displays both tiles and metatiles.
Terrain
Also unlike the first game, areas in the second game can also use terrain instead (or maybe even additionally to) tiles. The terrain is floor geometry unique to the area, and consists of patches of land and water, together with walk mesh definitions. Terrain is used very often in outdoor areas, to give those areas a more unique and less flat feeling.
xoreos correctly loads and displays the land and water patches, in a very rudimentary and stubby way.
Environmental Object
Neverwinter Nights 2 adds a new area object type: environmental objects. These seem to be quite similar to regular placeables, only that they are static and can't ever be moved. Apparently, their geometry is also baked into the area walk meshes.
As an extra weird bit, environmental objects store their position and orientation differently than all the other objects.
xoreos correctly loads and displays environmental objects in areas.
Trees
Neverwinter Nights 2 also adds new, non-static trees, courtesy of the SpeedTree middleware. Tree definitions seem to consist of just a few parameters and a random seed, the rest is done by the SpeedTree library.
xoreos does not load or display those trees yet. In fact, that will be a major task in the future. Luckily, trees aren't essential; them missing is not a showstopper.
Creature Models and Animations
xoreos also partially loads and displays the various creatures (animals, NPCs, monsters, ...) defined in the area files. There's one caveat, though: the different model parts (head, hair, gloves, boots) aren't actually attached to the body yet. They're loaded as separate models and those are already placed correctly for default character dimensions. Of course, this would fail with scaled models (which we do not yet support).
The underlying problem is this: where the character models in Neverwinter Nights and Knights of the Old Republic contain special nodes telling the engine where to place the different body parts, Neverwinter Nights 2 has no such thing. Instead, each skin geometry vertex has a "bone index" attribute that, I assume, are indices into the Granny 3D animation files.
Which means: to properly attach the body parts, and to animate the models, we first need to figure those files out.
Tile and Object Tinting
A lot of objects in Neverwinter Nights 2 areas use tint maps. These are special textures with intensity values in their color channels. When an object is loaded from the area definition, those also contain 3 color values, and those color values are combined with the intensity values in the tint map to create a unique tinting of the object. This gives you the ability to create blue-ish caskets with yellow-ish metal rings, purple carpets, green vases and the like, without having to manually create textures for each color combination.
Moreover, tiles also use this tinting.
Currently, xoreos can evaluate and use those tinting, but entirely in software. For each object, an uncompressed RGBA texture is created, and the color values are mixed on loading. Of course, this has certain drawbacks:
First of all, it's slow. But more damning, it takes up a lot of extra memory, both on the GPU and in the system's RAM. As a ballpark, a common large-ish area creates 1GB of extra textures; even more when there's a lot of objects around.
This is, of course, unacceptable. In the future, this tinting should most probably be done with shaders on the GPU.
Terrain Tinting
There is another tinting mechanism: terrain geometry uses a similar approach, only more extended. Instead of just 3 mix channels, there's 6 (using 2 mixing textures) and instead of just mixing colors, textures are mixing. I.e.: a terrain patch can specify up to 6 texture that will be mixed according to the color values in 2 mix textures. The result is a layered texture that removes all visible texture tiling effects.
Additionally, each patch also has a color value to modify the texture, increasing the flexibility again.
At the moment, xoreos does none of this, for obvious reasons. Instead, the singular color value is applied to each patch, creating a more flat look.
Doors and Chests
Currently, xoreos does not display most doors and chests entirely correctly: doors are missing their, well, door part (only the door frame and knob are visible) and chests are missing their lid. I have no idea if that's connected to the animation system, tinting, or what.
All in all, I'm pretty happy with the progress so far. :)
Note: This is a cross-post of a news item on the xoreos website.
During the last 2 weeks, I implemented basic support for Neverwinter Nights 2 areas and campaigns/modules. In particular:
- Reading the list of modules and the starting module in a campaign
- Reading module areas and starting conditions
- Loading general area information like music
- Loading area geometry and objects
- Tiles
- Terrain
- Doors
- Placeables
- Environmental objects
- Creatures
- Flying through the areas in a "spectator mode"
- Highlighting potentially useable objects
- Hooking up a debug console
- listcampaign/loadcampaign commands
- listmodules/loadmodule commands
- listareas/gotoarea commands
As a result, areas in Neverwinter Nights 2 are now in a similarly supported state as areas in the two Knights of the Old Republic games.
So, here's a few interesting things and/or issues with them.
Tiles and Metatiles
Like the first Neverwinter Nights game, areas in Neverwinter Nights 2 are (or can be, more about that later) made up of tiles. Meaning, a grid of models define the general structure: piece of indoors floor with walls, corners, windows and/or doors. Unlike the first game, though, there's also metatitles, which are bigger models spanning the space of several regular tiles. This is used, for example, for the piece of the cave where you wake up in the Mask of the Betrayer expansion.
xoreos correctly loads and displays both tiles and metatiles.
Terrain
Also unlike the first game, areas in the second game can also use terrain instead (or maybe even additionally to) tiles. The terrain is floor geometry unique to the area, and consists of patches of land and water, together with walk mesh definitions. Terrain is used very often in outdoor areas, to give those areas a more unique and less flat feeling.
xoreos correctly loads and displays the land and water patches, in a very rudimentary and stubby way.
Environmental Object
Neverwinter Nights 2 adds a new area object type: environmental objects. These seem to be quite similar to regular placeables, only that they are static and can't ever be moved. Apparently, their geometry is also baked into the area walk meshes.
As an extra weird bit, environmental objects store their position and orientation differently than all the other objects.
xoreos correctly loads and displays environmental objects in areas.
Trees
Neverwinter Nights 2 also adds new, non-static trees, courtesy of the SpeedTree middleware. Tree definitions seem to consist of just a few parameters and a random seed, the rest is done by the SpeedTree library.
xoreos does not load or display those trees yet. In fact, that will be a major task in the future. Luckily, trees aren't essential; them missing is not a showstopper.
Creature Models and Animations
xoreos also partially loads and displays the various creatures (animals, NPCs, monsters, ...) defined in the area files. There's one caveat, though: the different model parts (head, hair, gloves, boots) aren't actually attached to the body yet. They're loaded as separate models and those are already placed correctly for default character dimensions. Of course, this would fail with scaled models (which we do not yet support).
The underlying problem is this: where the character models in Neverwinter Nights and Knights of the Old Republic contain special nodes telling the engine where to place the different body parts, Neverwinter Nights 2 has no such thing. Instead, each skin geometry vertex has a "bone index" attribute that, I assume, are indices into the Granny 3D animation files.
Which means: to properly attach the body parts, and to animate the models, we first need to figure those files out.
Tile and Object Tinting
A lot of objects in Neverwinter Nights 2 areas use tint maps. These are special textures with intensity values in their color channels. When an object is loaded from the area definition, those also contain 3 color values, and those color values are combined with the intensity values in the tint map to create a unique tinting of the object. This gives you the ability to create blue-ish caskets with yellow-ish metal rings, purple carpets, green vases and the like, without having to manually create textures for each color combination.
Moreover, tiles also use this tinting.
Currently, xoreos can evaluate and use those tinting, but entirely in software. For each object, an uncompressed RGBA texture is created, and the color values are mixed on loading. Of course, this has certain drawbacks:
First of all, it's slow. But more damning, it takes up a lot of extra memory, both on the GPU and in the system's RAM. As a ballpark, a common large-ish area creates 1GB of extra textures; even more when there's a lot of objects around.
This is, of course, unacceptable. In the future, this tinting should most probably be done with shaders on the GPU.
Terrain Tinting
There is another tinting mechanism: terrain geometry uses a similar approach, only more extended. Instead of just 3 mix channels, there's 6 (using 2 mixing textures) and instead of just mixing colors, textures are mixing. I.e.: a terrain patch can specify up to 6 texture that will be mixed according to the color values in 2 mix textures. The result is a layered texture that removes all visible texture tiling effects.
Additionally, each patch also has a color value to modify the texture, increasing the flexibility again.
At the moment, xoreos does none of this, for obvious reasons. Instead, the singular color value is applied to each patch, creating a more flat look.
Doors and Chests
Currently, xoreos does not display most doors and chests entirely correctly: doors are missing their, well, door part (only the door frame and knob are visible) and chests are missing their lid. I have no idea if that's connected to the animation system, tinting, or what.
All in all, I'm pretty happy with the progress so far. :)
Some you may have missed, popular articles from the last month:
Kotor natively would be amazing too, all in all. Wish I could programme so I could help out. Will gladly donate though if it helps.
What I would like, in the future [1], is for xoreos to join a non-profit, like the Software Freedom Conservancy. The non-profit would hold the project's funds, which a) makes sure they are separate from me personally b) clears up tax problems c) would make donations tax-deductible in the country the non-profit operates in.
Money could then be transparently earmarked for certain things, like server costs, maybe IDA licences, maybe registrating a trademark. Or, on the slim chance there's a big windfall, paying people on a contractual basis to work on xoreos.
This way, donations would be transparent and I wouldn't have to worry about correctly declaring the donations as taxable income. And I think the money going to a non-profit instead of me would create some barrier against legal claims by Electronic Arts, maybe.
[1] The thing is, for a non-profit to consider xoreos as a member project, there needs to be an active and vibrant developer community. While I'm really damn grateful for the people who do work on xoreos with me (seriously, <3 you people), I fear it's not enough yet.
So, if you want to help me, spread the word and get more people to join me. And if you really have money burning a hole into your pocket, please consider donating to the Free Software Foundation, Software Freedom Conservancy, Software Freedom Law Center or the Electronic Frontier Foundation. Those organizations do a lot of things I care about in the realms of FLOSS. Thanks.
Aurora is in example missing PhysX, which DA: O with the Eclipse engine makes use of.
Sure there's been a lot of modifications along the way, but the base for each game is similar. If you read the Dragon Age Toolset Wiki, you'll see similar file formats and concepts. The game is still made up of areas, areas contain objects like placeables, doors and creatures. There's still triggers and the script system is still obviously NWScript-like.
I eagerly bought NWN 2 whe it came out, and was...disappointed. I did not like the graphics (so sue me), and the adventure included was uninspired, but so was the original NWN adventure. I did not buy the expansions, I'm afraid. There is also a raft of modules for NWN 2 on Neverwinter Vault, for what it's worth.
Now, this should not imply that I wouldn't like to give the sequel another try, especially with a Linux client.
Creating modules was harder in NWN2, though, and so the module community didn't take off quite as much as NWN1. Since they used the proprietary middleware FaceFX for lipsync, and BioWare of course couldn't give any information out, people couldn't even create lipsync'd cutscenes like the original campaign did. Instead, they started generating silly random lip flapping.
Harp & Chrysanthemum is excellent, I think. Too bad the promised sequel never came.
Dark Waters, by Adam Miller, the same guy responsible for the classic NWN1 campaigns Shadowlords and Dreamcatcher, I found disappointing. Especially after the long wait. It was riddled with bad dialogue and cringe-inducing voice acting, but then again I have a low tolerance for bad voice acting.
If so, maybe the TW1 on Xoreos would be the unique and genuine native port this game will have.
Good to read that!