Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification

By Julie Chang, Vincent Sitzmann, Xiong Dun, Wolfgang Heidrich, Gordon Wetzstein. Scientific Reports, 2018

516 citations recorded by the source corpus.

Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/b06d7f7742ed94078f512011968ae123e576b59f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

Citations in the corpus

Inference in artificial intelligence with deep optics and photonics (2020, methodology): “Recent research has demonstrated that this interpretation allows for a single convolutional layer 15 , a fully connected layer or an otherwise parameterized 14 layer of a deep network to be implemented in optics.”
The physics of optical computing (2023, background): “…if the output is ultimately going to be electronic anyway — one can adopt the strategy of using optics to pre-process the optical image data 70,125 , intelligently encoding it so that the output conversion from optics to electronics has much lower bandwidth than naively digitizing the images…”
Large-Scale Optical Neural Networks based on Photoelectric Multiplication (2018, background): “In addition, optics performs a single-channel (C = C ′ = 1) convolution, and while extending to multiple output channels is possible by tiling kernels [82], multiple input and output channels may be difficult.”