Sony patents Generation Super Sampling, its own predictive frame generation technology

Sony has filed a patent application titled Generation Super Sampling. In it, he talks about his own system for generating frames using neural networks. The document, signed by engineer Denis Dupeyron, proposes a method designed to mitigate poor performance in scenes with high graphic load through direct image prediction. We can assume that this is a technique aimed at the company’s future consoles, with the PlayStation 6 as the main candidate.

Sony’s approach seeks to solve one of the main limitations of current real-time generation methods. While the traditional generation of frames and multi-frame interpolations requires having two frames already calculated to interleave intermediate images, the latest proposals have taken different paths.

In fact, NVIDIA has taken the technology towards Neural Rendering with DLSS 5, where neural models not only reconstruct or interleave images, but they directly generate illuminationtextures and materials on the 3D scene, assuming a calculation cost that reduces the base frames by between 45% and 50% before compensating them with artificial generation.

Compared to these schemes or conventional interpolation that adds delay when waiting for the next frame, Sony’s proposal predicts forward from a single newly rendered frame. The synthetic image is displayed on the screen while the engine continues processing the next real image. To make the inference accurate without degrading the scene, the decoder network analyzes the motion vectors and adds recent keystrokes. In this way, the system interprets the player’s physical actions to anticipate where the camera or character will move.

Multi-frame prediction and learning embedded in the video stream

The registered mechanism works through an encoder that converts the processed frame into a compact representation. Then, it goes to a decoder network in charge of inferring the next image. The document points out that this model is capable of training directly on the video stream, comparing each prediction with the subsequent native frame, thus eliminating manual labeling.

Furthermore, it contemplates the capacity of project multiple consecutive frames simultaneously and incorporates a secondary mode without artificial intelligence that moves blocks of pixels following traditional motion vectors.

At the moment, the document is an application in the processing phase. This means that the information is scarce and does not provide latency figures or visual quality measurements. Nor does it explicitly mention any specific console. However, all this is known in a period in which Sony is considering delaying the launch of PlayStation 6 until 2028 or 2029 due to component supply problems caused by the high demand for AI hardware.