ActRay: Online Active Ray Sampling for Radiance Fields

Abstract

Thanks to the high-quality reconstruction and photorealistic rendering, NeRF (Neural Radiance Fields) has attracted extensive attention and achieved continuous improvement. Despite its impressive visual effect, the prohibitive reconstruction time limits its practical application in real life. Although significant acceleration has been achieved, there is still a considerable gap from real-time reconstruction, due to the need for tens of thousands of iterations. In this paper, a feasible solution thus is to reduce the number of required iterations by always training the underfitting rays samples. To this end, we propose an online active ray sampling strategy, ActRay. Since the states of rays are partially observable and outdated, a rendering-gradient-based loss propagation algorithm is presented to fully estimate the ray states. To balance exploration and exploitation, a UCB-based online reinforcement learning algorithm is proposed to adaptively select the most valuable and potential rays. We evaluate ActRay on both real-world and synthetic scenes, and the promising results show that it accelerates radiance fields training to 651% of the original speed. Besides, we test ActRay under all kinds of NeRF representations (implicit, explicit, and hybrid), demonstrating that it is general and effective to different representations. We believe this work will contribute to the practical application of radiance fields, because it has taken a step closer to real-time radiance fields reconstruction.