Nvidia announced a new version of its smart upscaling technology DLSS at the Gamescom fair in Cologne. The denoisers are replaced by an AI model, which should improve both image quality and ray tracing performance.
To understand what the changes in DLSS 3.5 mean, we first need to go back to the basics of how real-time ray tracing works. First, the engine generates a simple image without lighting effects, based on, among other things, the geometry. Rays are then shot onto the rendered world for both reflections and lighting. Because ray tracing is very computationally intensive, the process is cut short if the correct exposure is not yet known for all pixels. Manually tuned denoisers fill in the missing pixels, after which the image is upscaled to native resolution.
Such denoisers work, for example, with temporal accumulation, where the light information from previous frames is used, and spatial interpolation, where surrounding pixels are mixed. We have gone into this in more detail before. In any case, current denoising methods can result in image errors, such as ghosting of objects that are no longer in the picture (at that location), blurred lighting due to lack of sufficient data and loss of details in repetitive patterns. According to Nvidia, denoising and the upscaling that takes place later in the process ironically work against each other, because the denoising removes detail that could have been useful during upscaling.
In DLSS 3.5, those denoisers are replaced by an AI model, which Nvidia calls ‘ray-reconstruction’ (RR). According to Nvidia, this model has been trained on five times as much data as DLSS 3. In addition to the calculated rays, the model also takes into account information about movement and previous frames. The hardware optical flow accelerator is then used as before to recognize movement in parts of the image that are not yet part of the motion vectors that are fed to the AI model by the engine, such as shadows and reflections. If desired, frame generation can be applied to the end result for better performance, but this is not necessary.
The result, according to the examples shared by Nvidia, is both an improvement in image quality, because fewer graphic errors are visible, and an improvement in performance. The latter is because the AI model is less computationally intensive than the previously used denoising algorithms. In the game Cyberpunk 2077, the performance increase when enabling ray reconstruction, on top of super resolution and frame generation, would be an additional eight percent.
In contrast to frame generation, ray reconstruction can be used on all Nvidia GPUs with support for ray tracing, i.e. from the RTX 2000 series. Alan Wake II, Cyberpunk 2077: Phantom Liberty and the RTX version of Portal will be the first games to support DLSS 3.5 this fall. Rendering programs such as D5 Render and Chaos Vantage are also planning support for Nvidia’s new DLSS version.
DLSS, RTX and Reflex roadmap
In addition to the announcement of DLSS 3.5, Nvidia also shared a number of updates regarding the support of GeForce technologies in existing and upcoming games:
- Alan Wake II will support DLSS 3.5, full ray racing and Reflex upon release;
- Call of Duty: Modern Warfare III will support DLSS 3 and Reflex upon release;
- Cyberpunk 2077: Phantom Liberty will support DLSS 3.5, full ray racing and Reflex upon release;
- Payday 3 will support DLSS 3 and Reflex upon release;
- Fortnite will receive support for DLSS 3 this fall.
Community makes Half-Life 2 RTX
Developers not officially affiliated with Valve have started creating an RTX Remix version of the game Half-Life 2 under the name Orbifold Studios, for which they received early access to the RTX Remix toolkit from Nvidia. No release date has been announced yet, but Nvidia did release the trailer below for Half-Life 2 RTX: An RTX Remix Project.
Ace NPCs gain emotion
Nvidia also had an update on the Ace technology announced during the Computex, an AI model to allow NPCs to provide intelligent responses in games by means of a large language model. With the new addition of NeMo SteerLM, the traits of humor, helpfulness, creativity and toxicity can be set to influence the type of response. Nvidia had not yet made any concrete announcements about when Ace will be used in games.
Xbox PC Game Pass on GeForce Now
Finally, the GPU manufacturer announced that the first Xbox PC Game Pass titles will be available this week on cloud gaming service GeForce Now. From August 24, you can play all supported titles from Xbox Game Studios and Bethesda via Nvidia’s service, plus various titles from other developers. So if you have access to a supported game through Xbox Game Pass or a Microsoft Store purchase, you don’t need to purchase it again.