This paper introduces a novel real-time non-linear predictive control approach based on a model-free reinforcement learning (RL) method, with a non-linear reconstruction relying on supervised learning (SL) using a U-net architecture. We present our on-sky and bench results obtained on the Subaru Coronagraphic Extreme Adaptive Optics (SCExAO instrument and show that our approach outperforms the standard modal integrator controllers.

To develop our two-component model, we quantify the possible improvement of a non-linear reconstruction for the pyramid wavefront sensor (PyWFS) and a wavefront prediction depending on the intrinsic delay of the system. 

This analysis provides the basis for building our two-component model of non-linear prediction with RL and reconstruction with SL. The RL component is trained online with telemetry data and the SL model is trained offline with a previously built dataset of WFS images and phases. 

The proposed model involves training (RL) and inferring (RL and SL) at high framerates (2kHz PyWFS images). To run at such speed, we propose a scheme that we integrate into the Compute and Control for Adaptive Optics real-time control package (CACAO), leveraging this framework to process the data and both infer and train at speeds that allow for increased performance. The main ideas of the scheme are TensorRT for inference, multiple GPU training for the RL model and a high degree of parallelization.

Finally, we present our results demonstrating that our model can run at the required speeds and outperform a default modal integrator controller both on-sky and on the bench.