Scientists at the Max Planck Institute for the Science of Light have proposed a new way of implementing a neural network: an optical system, which could make machine learning and AI tasks more sustainable in the future.
Currently, increasingly complex neural networks—some with billions of parameters—are required. But rapid growth of neural network size has put the networks on an unsustainable path due to their exponentially growing energy consumption and training times, say the researchers in a statement.
For example, training a large language model like GPT-3 is estimated to use just under 1,300 megawatt hours (MWh) of electricity—about as much power as consumed annually by 130 US homes. This trend has created a need for faster, more energy- and cost-efficient alternatives.
Light vs electrons
The researchers' new idea is to perform the required mathematical operations physically by light in a potentially faster and more energy-efficient way. Optics and photonics are particularly promising platforms for neuromorphic computing, since energy consumption can be kept to a minimum. Computations can be performed in parallel at very high speeds, only limited by the speed of light.
But there are two challenges: realizing the necessary complex mathematical computations requires high laser powers and the lack of an efficient general training method for such physical neural networks.
Both challenges could be overcome with a new method proposed by Max Planck Institute for the Science of Light researchers in a new paper in Nature Physics. The method avoids the complicated physical interactions needed by the required mathematical functions. But the authors have demonstrated in simulations that their new approach can be used to perform image classification tasks with the same accuracy as digital neural networks.
The authors plan to collaborate with experimental groups to explore the implementation of their method. Their proposal significantly relaxes the experimental requirements, so it can be applied to many physically very different systems. That opens up new possibilities for neuromorphic devices—allowing physical training over a broad range of platforms.
Citation: Wanjura, C. C., & Marquardt, F. (2024). Fully nonlinear neuromorphic computing with linear wave scattering. Nature Physics, 1-7. 10.1038/s41567-024-02534-9 (open access)