RBDOOM-3-BFG 1.5.1

23 May 2023 - RBDOOM-3-BFG 1.5.1 Hotfix Patch

This is just a bugfix build that addresses several crashes and minor changes.

Quick Installation Guide for Beginners

1. Make a new DoomBFG folder
2. Copy base/ from your Steam Doom 3 BFG folder into DoomBFG
3. Download the RBDOOM-3-BFG 1.3.0 full package from the RBDOOM-3-BFG ModDB page and extract it over DoomBFG.
4. Do the same with the 1.5.1 patch.

If you already have a version of RBDOOM-3-BFG then you can just unpack this patch over it.

Changelog:

• Bumped version to 1.5.1

• Fixed "Failed to create a graphics pipeline state object" error in the Erebus 3 map. #770

• Disable DXGI automatic display mode switching for alt-enter borderless fullscreen

• Fixed crash in Delta Labs Sector 3 with Vulkan on Linux (Radeon RX 6700)

RBDOOM-3-BFG 1.5.0

29 April 2023 - RBDOOM-3-BFG 1.5.0 adds DX12, Vulkan and glTF 2 Support

So over 600 commits later and probably over 1200 hours provided by 4 programmers RBDOOM-3-BFG 1.5.0 is final after one year of development.

This update is huge and replaces OpenGL with DX12 and Vulkan using Nvidia's NVRHI API and adds support for glTF 2 models and animations.

DX12 & Vulkan through NVRHI

OpenGL is dead and won't be further developed. OpenGL was holding back the development of this source port for quite some time and all the HLSL -> GLSL translation code with different outputs for Windows, Mesa and Vulkan GLSL was a time eater.

I was excited when Vulkan was announced but it wasn't fun at all to work with because it is way too verbose and difficult to work with even after 17 years of OpenGL experience. It was even annoying to just render anything offscreen like shadow maps. Right now, Vulkan offers many extensions to make those tasks simpler but if you are using modern Vulkan then it is already so different that people demand to make a new Vulkan 2 API.
Vulkan became immediatly the extension hell OpenGL was and all I wanted was a nice and simple API like OpenGL ES 3 but with proper error feedback and more speed.

I was looking for several years for a good abstraction layer of modern graphics APIs like DX12 and Vulkan. I considered The-Forge, Dilligent Engine, Tiny Renderers (Google) or BGFX for reducing the maintencance costs of the RBDOOM-3-BFG renderer.

Then I stumbled over an excellent article by Alexey Panteleev about Writing Portable Rendering Code with NVRHI.

NVRHI (NVIDIA Rendering Hardware Interface) is a library that implements a common abstraction layer over multiple graphics APIs (GAPIs): Direct3D 11, Direct3D 12, and Vulkan 1.2. It works on Windows (x64 and ARM64) and Linux (x64 and ARM64) and macOS through MoltenVK. It is not tied to Nvidia GPUs but also works with AMD and Intel drivers.

I suggested it in Discord and that we should go into that direction if we move the renderer to an abstraction library. Stephen Pridham took the initiative and wrote the initial port to NVRHI. Porting a game like Doom 3 BFG to another render API is not trivial.
It also changed my plans for 2022 and ruined many weekends by fixing renderer related bugs.
However I want to continue to work on this project in my spare time for the next 10 years so the effort was worth it. In the end it turned out that NVRHI is as easy to use as OpenGL with the Donut samples provided.
There is definitely some lack of documentation but now you have a former AAA game rendered with NVRHI as an example.
RBDOOM-3-BFG has a similar renderer complexity to older Call of Duty titles or the Source engine.
The implementation is not optimal in terms of performance and very close to the original rendering architecture of Doom 3 BFG.
However it is nice and simple that way.

Stephen Pridham did the major porting effort and was a big help when addressing certain difficult issues.
Stephen Saunders helped out with many Vulkan related rendering bugs. He also got it working with MoltenVk on macOS and improved support for AMD Vulkan users by integrating AMD's Vulkan Memory Allocator (VMA). He also contributed many other smaller bugfixes and improvements. Big thanks for both guys for all the help!
Also thanks to Alexey Panteleev at Nvidia who helped us through Discord and by pointing us into the right directions.

NVRHI will be also the gate opener for cross platform hardware accelerated Raytracing in the future so it is overall a big win for this source port.

There are also some immediate advantages for RBDOOM-3-BFG besides the smaller renderer backend.

DX12 is faster than OpenGL

This is actually one of the most demananding scenes in Doom 3 with many shadow casting point lights. DX12 is now almost twice as fast as OpenGL. Vulkan is still lacking behind but DX12 is the primary API for Windows users which are the vast majority.
Stephen Saunders also provided triple buffering with NVRHI wich leads to a higher parallelization between the CPU and graphics card.

Temporal Anti Aliasing

MSAA/SMAA have been replaced with a Temporal Anti Aliasing solution by Nvidia. This not only fixes geometric aliasing at the edges but also shader based aliasing within the geometries like extreme specular highlights caused by the new PBR shaders.

New SSAO

The SSAO implementation has been updated with the latest code from Nvidia's Donut samples which outperforms the old implementation and also adds better stability on different view angles.
The old SSAO in 1.4.0 also lacked filtering and looked very noisy. The TAA would help but now it's really fixed and better.
It's hard to tell in these screenshots but the new SSAO gives most scenes a lot more depth.

glTF 2 Support

Harrie van Ginneken contributed a new glTF 2 loader written from scratch using the awesome id Tech 4 idlib framework.
Also a big thanks here because this is huge and took a lot of time to develop.

The importer is not designed to work with all glTF 2 models you can find on the internet but it serves as a solid import facility from Blender. It can read ASCII .gltf and binary .glb models. It is recommend to use .glb for faster load times.
Models imported as .glb will be cached as .bglb models in the generated folder like all other model formats.
Cool about the current implementation is that animations are converted straight to the .bMD5Anim binary format of the Doom 3 BFG edition. This makes it a very tight integration into the existing architecture.

The basic idea about the glTF 2 loader is to make it easier getting models into the engine without any third party plugins in Blender. Another big disadvantage of .md5mesh exporters is that the format simply lacks support for normals.
So all normalmaps baked in Blender or Substance Painter using the Mikktspace algorithm would look wrong.
glTF 2 fixes this problem and it not only has support for normals but also tangents and those have to be calculated the Mikkelsen way so that's a perfect fit.

You can also use .glb files for static models without a skeleton just like with the original 3ds Max .ase and Lightwave .lwo files.
On top of that you can also use glTF 2 for new camera animations but that needs to be covered in an upcoming tutorial.

I also want to go back and keep things simple so I don't want to import and configure a model like in Unity or Unreal. I just want to load models and use them and the maximum should be to specify which frame plays a foot sound or which bone needs to be used to attach an item to it.

However it might be the case that someone exports a model from Blender and in a team environment someone else needs an option to load it differently. E.g. the front in Blender is -Y and forward in Doom is +X. So we need to rotate a model by 90 degrees around Z axis. All these things did the Maya .mb -> .md5mesh converter plugin for Doom 3 and we allow similar options in model declarations.

model character_mcneil
{
  // scale from meters to Doom units (inches), add an "origin" bone from rig origin to pelvis and 
  // build automatically a root motion from the pelvis bone
	options -scale 39.37 -addorigin -transfermotion "pelvis"
	
	mesh models/characters/mcneil/mcneil.glb
	
	channel legs ( origin *pelvis *foot_l *foot_r)
	
	anim walk models/characters/mcneil/mcneil.Walk.glb
	anim idle models/characters/mcneil/mcneil.Idle.glb
}

Supported options are:

• -armature "armatureName" if a glTF 2 file has more than one skeleton

• -reorient pitch, yaw, roll

• -rotate yaw

• -transferMotion "boneName" to enable root animations like in Unity

• -addorigin so we don't need put that extra "origin" bone in every Armature and can use common rigs like Mixamo or Rigify

• -scale factor

• -nomikktspace, specify if you want to use old normalmaps from an original Doom 3 model

NOTE: Material names in Blender need to be 1:1 the material names defined in the corresponding materials/*.mtr files.

Models are expected to be exported by Blender with the following settings:

Format: glTF Binary (.glb)
Transform: +Y Up On
Materials: Export
Images: None
Compression: Off
Data: Custom Properties On
Animation: Animation On, Shape Keys Off, Skinning On

glTF 2 dmap Support and Blender Mapping Workflow

I rewrote dmap 8 years ago to compile a BSP from a polygon based map format instead of brushes.
Now, Harrie van Ginneken has extended it so we can compile glTF 2 files.
The trick is to use custom properties in Blender and the engine treats the .glb file like a .map file.

There is too much to explain here and this needs to be covered in a tutorial series along with a new Blender addon for editing the entities. But this is quite cool. You can now make maps without TrenchBroom or a Radiant based editor. Theoretically it is possible to use any 3D modelling package as a level editor now.

Just save your map as .glb file and compile it like:

dmap mymap.glb

Harrie managed to port entire asset packs with animations and hundreds of meshes by using a few clever Python scripts within Blender.

Here is a small demo of his WIP game which shows GPU skinning using M...

RBDOOM-3-BFG 1.4.0

6 March 2022 - RBDOOM-3-BFG 1.4.0

.plan

This version improves support for mapping with TrenchBroom. Until now you needed to extract and copy the vanilla Doom 3 models and textures over to the base/ folder to see the content in the TrenchBroom entity browser and texture viewer.
Owning the original game next to the BFG edition is not necessary anymore.
This version comes with a couple of new RBDOOM-3-BFG console commands that lets you export particular parts of the .resources files to the base/_tb/ folder.

You need to call exportImagesToTrenchBroom once and you are good to go to start mapping with the TrenchBroom level editor.

TrenchBroom comes with several more Doom 3 specific changes. After loading a map TrenchBroom generates unique entity names and also fixes missing or bad "model" keys for brush based entitites.
Also creating new entities like light will automatically receive names like light_2.

This patch also contains a couple of func_group related bugfixes. func_group works now with brush based entities, point entities and just regular brushes.

Changelog

[TRENCHBROOM]

• Tweaked exportFGD command for new icons

• Added new icons to TrenchBroom for certain entities like lights, speakers or particle emitters

• TrenchBroom offers a dropdown menu to select the Quake 1 light style for lights

• Drastically improved loading time of textures for materials in TrenchBroom

• Added RBDoom console command convertMapToValve220 <map>. You can also type exec convert_maps_to_valve220.cfg to convert all Doom 3 .map files into the TrenchBroom friendly format. Converted maps are saved with the _valve220.map suffix.

• Added RBDoom console command exportImagesToTrenchBroom which decompresses and saves all .bimage images to base/_tb/*.png files

• Added RBDoom console command exportModelsToTrenchBroom which saves all .base|.blwo|.bmd5mesh models to _tb/*.obj proxy files. This commands also generates helper entities for TrenchBroom so all mapobject/models are also available in the Entity Inspector using the DOOM-3-models.fgd.

• Added RBDoom console command makeZooMapForModels which makes a Source engine style zoo map with mapobject/models like .blwo, .base et cetera and saves it to maps/zoomaps/zoo_models.map. This helps mappers to get a good overview of the trememdous amount of custom models available in Doom 3 BFG by sorting them into categories and arranging them in 3D. It also filters models so that only modular models are picked that can be reused in new maps.

• TrenchBroom got several Doom 3 specific issue generators to help mappers avoiding pitfalls during mapping

• Changed TrenchBroom's rotation tool to use the "angles" key by default to remove some Quake related limitations

[MISCELLANEOUS]

• Stencil shadows work again (thanks to Stephen Pridham)

• Fixed black screen after using the reloadImages command

• Added CMake options STANDALONE and DOOM_CLASSIC

• Added command convertMapQuakeToDoom <map> that expects a Quake 1 .map in the Valve220 format and does some Doom 3 specific fixes

• The gamecode ignores func_group entities if they were created by TrenchBroom instead to warn that there is no spawn function

• dmap / idMapFile move brushes of func_group entities to worldspawn before compiling the BSP.
  This also means func_group brushes are structural.
  If you want to optimize the BSP then move those to func_static instead which is the same as func_detail in Quake.

• Fixed that dmap failed writing the BSP .proc file if the command was interrupted by an error

[COMMUNITY]

Steve Saunders contributed

• Updated mac OS support

• Improved Vulkan / Molten support

• Fixed FFmpeg 5 compatibility for newer Linux distros

• Bink videos can play audio if they contain audio tracks (merged from DOOM BFA by Mr.GK)

[ASSETS]

• Added TrenchBroom helper entityDefs like a Quake 3 style misc_model to comply with TrenchBroom's Solid/PointClass rules for editing entities

• Added new Creative Commons CC0 textures/common/ and textures/editor/ replacement textures because they didn't ship with the BFG edition

• Added base/convert_maps_to_valve220.cfg which lets you convert all maps to the Valve 220 .map format in one shot

• Added base/maps/zoomaps/zoo_models.map

RBDOOM-3-BFG 1.3.0 Halloween 2021

---

30 October 2021 - RBDOOM-3-BFG 1.3.0 - This is the Lite package. Download the Full package with over 6.3 GB precomputed global illumination data from the RBDOOM-3-BFG ModDB Page

---

RBDOOM-3-BFG 1.3.0 adds PBR, Baked GI and TrenchBroom Mapping Support

The main goal of this 1.3.0 release is enabling modders the ability to make new content using up to date Material & Lighting standards. Adding PBR is a requirement to make the new content look the same in RBDOOM-3-BFG as in Blender 2.9x with Cycles or Eevee and Substance Designer. PBR became the standard material authoring since 2014. Many texture packs for Doom 3 like the Wulfen & Monoxead packs were made before and are heavily outdated. With this release modders can work with modern tools and expect that their content looks as expected.

However the PBR implementation is restricted to standard PBR using the Roughness/Metallic workflow for now.
Specialized rendering paths for skin, clothes and vegetation will be in future releases.

Physically Based Rendering

Implementing Physically Based Rendering (PBR) in Doom 3 is a challenge and comes with a few compromises because the Doom 3 content was designed to work with the hardware constraints in 2004 and that even meant to run on a Geforce 3.

The light rigs aren't made for PBR but it is possible to achieve good PBR lighting even with the old content by tweaking the light formulars with a few good magic constants. However I also want to support the modding scene to allow them to create brand new PBR materials made with Substance Designer/Painter or other modern tools so multiple rendering paths have been implemented.

PBR allows artists to create textures that are based on real world measured color values and they look more or less the same in any renderer that follows the PBR guidelines and formulars.

RBDOOM-3-BFG only supports the standard PBR Roughness/Metallic workflow.

Baked Global Illumination using Irradiance Volumes and Image Based Lighting

To achieve the typical PBR look from an artistic point of view it also means to that it is necessary to add indirect lighting.
Doom 3 and even Doom 3 BFG had no indirect lighting.

Doom 3 BFG is a big game. Doom 3, Resurrection of Evil and Lost Missions sum up to 47 big single player levels with an average of ~60 - 110 BSP portal areas or let's call them rooms / floors. Each room can have up to 50 shadow casting lights and most of them are point lights.
I needed a good automatic solution that fixes the pitch black areas without destroying the original look and feel of the game.
I also needed to add environment probes for each room so PBR materials can actually reflect the environment.

So RBDOOM-3-BFG comes with 2 systems to achieve this and both are automatic approaches so everything can be achieved in a reasonable amount of time.
The first system are environment probes which are placed into the center of the rooms. They can also be manually tweaked by adding env_probe entities in the maps. They use L4 spherical harmonics for diffuse reflections and GGX convolved mip maps for specular reflections.
The second system refines this by using a light grid for each room which provides a sort of a localized/improved version of the surrounding light for each corner of the room.

Irradiance Volumes aka Light Grids

RBDOOM-3-BFG 1.3.0 brings back the Quake 3 light grid but this time the grid points feature spherical harmonics encoded as octahedrons and it can be evaluated per pixel. This means it can be used on any geometry and serves as an irradiance volume.
Unlike Quake 3 this isn't radiosity which is limited to diffuse only reflections. The diffuse reflectivity is built using all kinds of incoming light: diffuse, specular and emissive (sky, light emitting GUIs, VFX).

Lightgrids can be baked after loading the map and by typing:

bakeLightGrids [<switches>...]
<Switches>
 limit[num] : max probes per BSP area (default 16384)
 bounce[num] : number of bounces or number of light reuse (default 1)
 grid( xdim ydim zdim ) : light grid size steps into each direction (default 64 64 128)

This will generate a .lightgrid file next to your .map file and it will also store a light grid atlas for each BSP area under env/maps/<path/to/your/map/>

Limit is 16384 by default and means the maximum number of light grid points in a single light grid.
Quake 3 had one light grid that streched over the entire map and distributed lighting every 64 x 64 x 128 units by default.
If the maps were too big then q3map2 made the default grid size broader like 80 x 80 x 144, 96 x 96 x 160 and so on until the maximum number of light grid points was reached.

The Quake 3 approach wouldn't work with Doom 3 because the maps are too big and it would result in up to 800k probes for some maps or the grid density would very coarse.

RBDOOM-3-BFG uses the bounding size of the BSP portal areas and puts smaller light grids into those BSP areas.

This way we can maintain a good grid density and avoid wasting storage because of light grid points that are in empty space.

But what is an Irradiance Volume or Light Grid exactly?

It tells each model or lit pixel how the indirect diffuse lighting is coming from any direction. The Probulator page by Yuri O'Donnell has some good examples:

Left: The captured view using a panorama layout. Right: The Diffuse lighting using Level 4 Spherical Harmonics aka Irradiance.

Now think of this for each of the grid points in RBDOOM-3-BFG. If a model is placed between those probes the lighting will be interpolated by the nearest 8 grid points similar like in Quake 3.

Quake 3 only stored the dominant light direction, the average light color of that direction and an ambient color term for each grid point.
In RBDOOM-3-BFG you basically can have the diffuse lighting information for any world space direction instead.
This is a way more advanced technique.

Image Based Lighting and Environment Probes

Environment probes supplement the light grids. While light grids provide diffuse lighting information the signal isn't good enough to provide plausible specular light reflections. This is where environment probes are needed.

If a level designer doesn't put any env_probe entities into a map then they are automatically distributed through the map using the BSP area bounds and placed in the center of them.

Environment probes can be computed after loading the map and by typing:

bakeEnvironmentProbes

This will generate EXR files in env/maps/<path/to/your/map/>.

The environment probes use an octahedron encoding and the specular mipmaps are convolved using the Split Sum Approximation by Brian Karris [Epic 2013]. Convolving the environment probes is done on the CPU using the id Tech 5 multi threading code so it will consume all your available cores.

For artists this basically means if you increase the roughness in your material then you increase the mip map level of the environment probe it samples light from and it gets blurier.

Fallback for Missing Data

If you haven't downloaded the additional baked light data from the RBDOOM-3-BFG ModDB Page and just run RBDOOM-3-BFG.exe with the required DLLs (or you built it yourself) it will use an internal fallback.
RBDOOM-3-BFG.exe has one prebaked environment probe that is compiled into the executable.

It's the light data from the Mars City 1 lobby in the screenshot above. Using this data for the entire game is inacurrate but a better compromise than using a fixed global light direction and some sort of Rim lighting hack like in version 1.2.0.
The default irradiance / radiance data gives the entire game a warmer look and it fits for being on Mars all the time.

Some Examples of Indirect Lighting

Left: No extra ambient pass. Ambient is pure black like in original Doom 3. Right: Extra ambient pass with r_forceAmbient 0.5 using local environment probe for irradiance and radiance lighting.

Some examples that show additional environment lighting on all assets.

PBR Texture format

In Doom 3 a classic simple materials looks like this:

textures/base_wall/snpanel2rust
{
  qer_editorimage		textures/base_wall/snpanel2rust.tga
	
  bumpmap           textures/base_wall/snpanel2_local.tga
  diffusemap        textures/bas...

RBDOOM-3-BFG 1.2.0

---

October 2019 - RBDOOM-3-BFG 1.2.0

---

This is a maintenance release without Vulkan support even though it contains a lot Vulkan specific code.

• Experimental Work in Progress Vulkan renderer backend based on Dustin Land's Doom 3 BFG Vulkan port
  This renderer backend only supports ambient lighting and no light interactions but it also comes with a big renderer code cleanup.

  The changes also fixed several rendering bugs that occured in OpenGL:

• Fixed black dots when HDR and SSAO is enabled

• Hit/damage markers show a white background #422

• Refactored and simplified OpenGL renderer backend to match Vulkan backend

• Renamed the .vertex and .pixel shader files to .hlsl
  This allows better editing of the shaders within Visual Studio including Intellisense support.

• Integrated libbinkdec for video playback as a slim alternative to FFmpeg (thanks to Daniel Gibson)

• Added in-engine Flash debugging tools and new console variables.
  These tools help to analyse the id Tech view of Flash and what SWF tags are supported and how they are interpreted
  by id Tech's own ActionScript 2 interpreter

  • swf_exportAtlas
  • swf_exportSWF
  • swf_exportJSON
  • swf_show : Draws the bounding box of instanced Flash sprites in red and their names

• Added Steel Storm 2 Engine render demo fixes

• Merged LordHavoc's image compression progress bar which shows up in the map loading screen
  when loading and compressing new images from mods

• Added instructions how to use these Doom 3 port with the GOG installer

• Many smaller compiler related fixes like VS 2017 and VS 2019 support.
  If it fails to compile with GCC then it should at least build with Clang on Linux

RBDOOM-3-BFG 1.1.0-preview3

• True 64 bit HDR lighting with adaptive filmic tone mapping and gamma-correct rendering in linear RGB space
• Enhanced Subpixel Morphological Antialiasing
  For more information see "Anti-Aliasing Methods in CryENGINE 3" and the docs at http://www.iryoku.com/smaa/
• Filmic post process effects like Technicolor color grading and film grain
• Fixed issues with Mesa drivers and allowed them to use shadow mapping
• Defaulted fs_resourceLoadPriority to 0 so it is not necessary anymore to specify when running a modification
• Additional ambient render pass to make the game less dark similar to the Quake 4 r_forceAmbient technique
• Screen Space Ambient Occlusion http://graphics.cs.williams.edu/papers/SAOHPG12/
• Did some fine tuning to the Half-Lambert lighting curve so bump mapping doesn't loose too much details

For HDR comparison screenshots see:
http://imgur.com/a/dKfLH

RBDOOM-3-BFG 1.0.3

Community maintenance release:

• SDL 2 is the default for Linux
• CMake options like -DUSE_SYSTEM_LIBJPEG to aid Linux distribution package maintainers
• SDL gamepad support and other SDL input improvements like support for more mouse buttons
• Mac OS X support (experimental)
• XAudio 2, Windows 8 SDK fixes
• Better Mesa support (not including advanced shadow mapping)
• Added back dmap and aas compilers (mapping tools)
• Cinematic sequences can be skipped
• Localization support for other languages than English
• Improved modding support through loading of custom models and animations (see section 12 MODIFICATIONS in the README)

Many thanks to all contributors :)

RBDOOM-3-BFG 1.0.2

This release doubles the shadow mapping performance and allows the game to be played at solid 120 fps in 1080p with a Nvidia GTX 660 Ti.
It also adds a graphics option to enable/disable shadow mapping and fixes a few problems with FFmpeg playing Bink videos.

RBDOOM-3-BFG 1.0.1 with faster soft shadow mapping

This release improves the performance drastically and adds shadow mapping support for translucent surfaces like grates.

RBDOOM-3-BFG with soft shadow mapping