Published December 18, 2023 | Version v1

Neural Rendering Techniques

Authors/Creators

Description

Neural rendering has transformed 3D scene representation and novel view synthesis by replacing explicit geometric
primitives with implicit neural representations that encode scene appearance and geometry directly in neural network
weights. This study presents a systematic evaluation of five neural rendering approaches -- Neural Radiance Fields
(NeRF), Instant-NGP (hash-encoded radiance fields), 3D Gaussian Splatting (3DGS), Mip-NeRF 360 (unbounded
scenes), and NeuS (surface reconstruction) -- across four rendering tasks: bounded object synthesis (NeRF Synthetic /
Blender dataset, 8 objects), unbounded forward-facing scenes (LLFF, 8 scenes), unbounded 360-degree scenes
(Mip-NeRF 360 dataset, 9 scenes), and surface reconstruction (DTU dataset, 15 objects). A total of 1,680 experiments
were conducted across training time, rendering quality, and geometric accuracy dimensions. 3D Gaussian Splatting
achieved real-time rendering at 68.4 +- 4.2 FPS (1080p) by representing scenes as explicit differentiable Gaussians that
rasterise without ray marching, while achieving competitive PSNR (33.2 +- 0.4 dB on Blender vs. NeRF's 31.0 +- 0.6 dB).
Instant-NGP reduced NeRF training from hours to 2.4 +- 0.4 minutes through multi-resolution hash encoding, enabling
interactive training feedback. Mip-NeRF 360 achieved the highest unbounded scene quality (PSNR = 29.4 +- 0.4 dB)
through contract mapping of infinite space into bounded representation and anti-aliasing via cone-casting. NeuS
achieved the most accurate surface reconstruction (Chamfer distance = 0.62 +- 0.08 mm on DTU) by using a signed
distance function representation with volume rendering that correctly handles occlusions. A practical neural rendering
selection guide mapping use case, scene type, and computational budget to recommended methods is proposed.

Files

826_Neural_Rendering_Techniques_Vol2023_IssueIssue4_pp1-8_BioQI_journal.pdf

Files (445.4 kB)

Additional details

Identifiers

ISSN
3117-7336

Related works

Is documented by
Journal article: 3117-7336 (ISSN)

Dates

Accepted
2023-10-09

References

  • Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R. and Srinivasan, P.P. (2021) Mip-NeRF: A multiscale representation for anti-aliasing neural radiance fields. Proceedings of ICCV 2021, pp. 5855-5864. Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P. and Hedman, P. (2022) Mip-NeRF 360: Unbounded anti-aliased neural radiance fields. Proceedings of CVPR 2022, pp. 5470-5479. Boss, M., Braun, R., Jampani, V., Barron, J.T., Liu, C. and Lensch, H. (2021) NeRD: Neural reflectance decomposition from image collections. Proceedings of ICCV 2021, pp. 12684-12694