Deep Shading: Convolutional Neural Networks for Screen Space Shading

By Oliver Nalbach, Elena Arabadzhiyska, Dushyant Mehta, Hans-Peter Seidel, Tobias Ritschel. 2016

175 citations recorded by the source corpus.

In computer vision, convolutional neural networks (CNNs) achieve unprecedented performance for inverse problems where RGB pixel appearance is mapped to attributes such as positions, normals or reflectance. In computer graphics, screen space shading has boosted the quality of real‐time rendering, converting the same kind of attributes of a virtual scene back to appearance, enabling effects like ambient occlusion, indirect light, scattering and many more. In this paper we consider the diagonal problem: synthesizing appearance from given per‐pixel attributes using a CNN. The resulting Deep Shading renders screen space effects at competitive quality and speed while not being programmed by human experts but learned from example images.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

3D Shape Induction from 2D Views of Multiple Objects (2016, background): “Recent work on sceen-space shading using con-vnets are promising [22].”
VConv-DAE: Deep Volumetric Shape Learning Without Object Labels (2016, neutral): “Recently, using up-sampling to produce spatial output [22,23,24] - has also seen first applications for computer graphics.”
Neural Point-Based Graphics (2019, background): “More recently, deep rendering approaches [13,14,15,16,17] aim to replace physics-based rendering with a generative neural network, so that some of the mistakes of the modeling pipeline can be rectified by the rendering network.”
Deep blending for free-viewpoint image-based rendering (2018, background): “…possible avenues of research to bridge the remaining gap, e.g. , using fewer mosaics, more eicient networks architectures [Howard et al. 2017], model compression [Kim et al. 2015; Molchanov et al. 2016], or porting the blending network to GLSL [Nalbach et al. 2017]. et al. 2017], lower is beter.”
Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning (2019, background): “Recent advances in 3D face reconstruction [40, 42, 41, 43, 9], material inference [28, 7] and other 3D reconstruction tasks [51, 38, 34, 14, 23, 35, 39] have leveraged some other forms of differentiable rendering layers to obtain gradient flows in the neural networks.”
Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction (2019, background): “[30] propose a general deep shading network which learns the direct mapping from a variety of deferred shading buffers to corresponding shaded images.”
Learning to reconstruct shape and spatially-varying reflectance from a single image (2018, methodology): “The idea of using a network to predict global illumination has also been adopted by [Nalbach et al. 2017], but no prior method has done this for inverse problems.”
Artificial intelligence in the creative industries: a review (2020, methodology): “online content creators. AI has also been employed for rendering objects and scenes. This includes the synthesis of 3D views from motion capture or from monocular cameras (see Section 3.4.4), shading [49] and dynamic texture synthesis [50]. Creating realistic lighting in animation and visual eects h”
FastNeRF: High-Fidelity Neural Rendering at 200FPS (2021, background): “Recently, neural rendering [10, 27, 17, 25, 19] has offered a disruptive alternative: involve a neural network in the rendering pipeline to output either images directly [24, 18, 27] or to model implicit functions that represent a scene appropriately [5, 23, 29, 25, 40].”