Light Field Video Streaming on GPU

Light Field Video Streaming on GPU

Scopus Article

This paper proposes an efficient encoding method for light field video rendering in real time. Each frame of the light field video consists of a grid of images capturing the scene from different camera positions. The images are encoded by a video compression algorithm. The positions of the keyframes on the grid are automatically determined. The proposed compression uses GPU-accelerated HW video decoders. Data transfer between the host and the GPU memory is minimal. Only the packets necessary for the novel view synthesis are transferred. Standard video compression methods need to decode all packets between keyframes, and other existing light field compression methods focus solely on the best compression ratio. The proposed method outperforms them in the quality/decoding time ratio, which is the most important metric for the real-time rendering. The results presented show that currently existing alternatives cannot be used efficiently for 4K light field video streaming. A proof-of-concept light field player was implemented and is available to use. The proposal solves the memory and streaming requirements that are the most crucial issues in light field rendering. The paper additionally outlines enhancements to a current light field rendering technique, which has been modified to integrate effectively with the newly proposed encoding method.

This paper proposes an efficient encoding method for light field video rendering in real time. Each frame of the light field video consists of a grid of images capturing the scene from different camera positions. The images are encoded by a video compression algorithm. The positions of the keyframes on the grid are automatically determined. The proposed compression uses GPU-accelerated HW video decoders. Data transfer between the host and the GPU memory is minimal. Only the packets necessary for the novel view synthesis are transferred. Standard video compression methods need to decode all packets between keyframes, and other existing light field compression methods focus solely on the best compression ratio. The proposed method outperforms them in the quality/decoding time ratio, which is the most important metric for the real-time rendering. The results presented show that currently existing alternatives cannot be used efficiently for 4K light field video streaming. A proof-of-concept light field player was implemented and is available to use. The proposal solves the memory and streaming requirements that are the most crucial issues in light field rendering. The paper additionally outlines enhancements to a current light field rendering technique, which has been modified to integrate effectively with the newly proposed encoding method.

light field, compression, video, GPU, streaming

light field, compression, video, GPU, streaming

CHLUBNA, T.; MILET, T.; ZEMČÍK, P.

25.06.2025

SIGNAL PROCESSING-IMAGE COMMUNICATION

2025

138

Kingdom of the Netherlands

12

https://www.sciencedirect.com/science/article/pii/S0923596525001237

http://hdl.handle.net/