ARM’s “DLSS” for mobile phones will be open: this is how its NSS, NFRU and NSSD technologies work

As we already saw in architecture article on the new Mali G2-Ultra NX GPUARM has decided to implement a system of NX AI neural accelerators to be able to use rescaling, frame generation and ray reconstruction technologies for smartphones.

Together with the two SME2 engines from the CPU part of CSS For Mobile 2, they make up the hardware dedicated to the AI ​​of its new platform.

Broadly speaking, these new accelerators will allow the use of these technologies without penalizing the performance of the GPU shaders. The company will openly offer on platforms such as GitHub or Hugging Face technologies similar to what we already have on PCs with DLSS, FSR or XeSS, but adapted to the energy and power limitations of smartphones. These are the NSS, NFRU and NSSD technologies.

NSS (Neural Super Sampling)

NSS is the technology of temporary rescaling. It is based on a low-resolution image rendered by the GPU along with motion vectors and depth maps, and a neural network with AI is responsible for reconstructing it at a higher resolution without having to filter it later with antialiasing.

ARM has shown tests scaling from 540p to 1080p where they manage to double the FPS while maintaining a consumption of about 2.5 W and reducing traffic to memory by half.

This technology is capable of scaling 7 out of every 8 pixels of an image starting from 1 previously rendered pixel.

NFRU (Neural Frame Rate Upscaling)

Along with NSS, NFRU, the ARM system of frame generation; uses the motion engine built into the new GPUs to detect movements between previous and subsequent frames. With that information, the NFRY neural network generates an intermediate frame where the GPU has not had to render anything. According to ARM, this allows you to go from 60 to 120 stable FPS and reduces transfers to RAM by 33%.

NSSD (Neural Super Sampling and Denoising)

NSSD combines NSS temporal rescaling with a ray reconstruction/noise reduction, a technology that is especially practical in smartphones where the power and resources of the GPU are limited.

In this way, with NSSD you can start from raytraced images calculated with few light rays and at low resolution to subsequently rescale and reconstruct the illumination and filter the noise initially generated by AI.

ARM has shown demos such as Neural Dawn where, thanks to the combination of these AI techniques, it is capable of moving scenes with more than 1,400 light sources in real time, rendering at 540p and rescaling, all with a stable 60 FPS and a consumption of only 2 W.

This means multiplying FPS and efficiency by four from native mode to NFRU mode.