Max Planck Institute Develops Gravitational Wave Detector Reinforcement Learning System
“One of the benefits of MATLAB and Simulink was that I could iterate across different reinforcement learning algorithms quickly, which greatly reduced the time it took to reach the final choice.”
Key Outcomes
- Reinforcement learning agent developed with MATLAB and Simulink outperformed humans at improving astrophysical sensitivity to binary neutron stars
- First-ever implementation of reinforcement learning–based MIMO control for optomechanical systems demonstrated at a gravitational-wave detector
- Reinforcement learning agent trained and validated in simulation prior to deployment using Simulink
A deployed reinforcement learning agent outperforms humans at improving the astrophysical sensitivity to binary neutron stars. (Image credit: arXiv)
Researchers at the Max Planck Institute for Gravitational Physics in Hannover, Germany, study various areas of physics, including general relativity, quantum optics, and astrophysics. An important, ongoing project at the institute is detecting the gravitational waves caused by large-scale astrophysical events—like colliding black holes—with laser interferometry observatories, such as GEO600 in Germany or LIGO in the United States.
A Laser Interferometer Gravitational-Wave Observatory works by projecting laser beams on mirrors located miles apart to measure the minute fluctuations in the space-time caused by passing gravitational waves. These mirrors need controlling because gravitational waves can only be detected if all other, nongravitational changes on the path length (such as local seismic disturbance from ocean wave activity, farmers running machinery, etc.) are actively or passively suppressed. Controlling and aligning the hundreds of mirrors installed in the observatory has traditionally been done manually, with engineers and scientists studying the system and using their knowledge and intuition to create and adjust control filters.
Scientists at the Max Planck Institute for Gravitational Physics developed a reinforcement learning system in MATLAB® and Simulink® that can automatically adjust and align some of the key observatory mirrors. They used Deep Learning Toolbox™ to create a combined convolutional and long short-term memory neural network, which measures misalignments by analyzing the video from cameras placed in the optomechanical layout. The output of the neural network is the state of the reinforcement learning environment. With Simulink Design Optimization™ and Reinforcement Learning Toolbox™, they were able to automatically generate a reward signal for the reinforcement learning system based on the principles of classic control theory.
The scientists set up a simulation environment in Simulink based on the measurements they had obtained from the physical system using System Identification Toolbox™. They used this environment with Reinforcement Learning Toolbox to test different algorithms and train their reinforcement learning agent without the need to make physical changes to the equipment. Their success in using reinforcement learning at GEO600 is the first-ever implementation of neural network–based alignment sensing and control at a gravitational-wave detector.
Products Used
