We promote the optimization of robot performance via real-time experiments guided by dedicated teacher robots, instead of optimizing system performance guided by uncertain model-based predictions and measured data. The method we develop is where robots are used to develop controllers for robots in a somewhat analogous way as parents teach their children. The technique may deliver a transformative approach to control engineering systems, for which obtaining high-fidelity models is challenging while gathering training data is time-consuming.
”Robots Teaching Robots” as a New Optimal Control Paradigm
Optimal control is a branch of control theory that has the potential to revolutionize the creation of intelligent engineering systems – assistive robots, industrial robots, medical robots – that can improve by repeated experience, somewhat similar to humans. There are many optimal control techniques to control engineering systems. However, most currently available techniques require high-fidelity models or a large amount of measured data to mitigate the so-called simulation-reality gap; the gap between the optimal robot performance predicted by computer simulations and the non-optimal robot performance observed in real engineering applications.
We conduct fundamental research to close the simulation-reality gap when optimal control is applied to engineering systems. Model-based optimal control techniques enable efficient computation but they are subject to conservative control performance. Data-driven optimal control techniques mitigate the detrimental effect of uncertain models, but to do so, they require a large amount of training data. Therefore, scientific barriers must be overcome to realize the full application potential of optimal control techniques.
We promote the optimization of system performance via real-time experiments guided by dedicated teacher robots, instead of optimizing system performance guided by uncertain model-based predictions and measured data. Our research aims to address the knowledge gap that limits the potential and theoretical promise of optimal control theory when applied to engineering systems.
Real-time Optimal Control of Robots
Optimal controllers computed using inexact model information lead to significant performance degradation. One way to avoid this limitation is to use model-free optimal control. However, model-free optimal control approaches require a large amount of training data. The goal of this research is to introduce an optimal control formulation that uses inexact models to speed up the computation and measured data to avoid the simulation reality gap introduced by inexact model-based future prediction. The result is a real-time – half model-based and half data-driven – optimal control formulation which is suitable for real-time hardware-in-the-loop control. Experiments on a three-link direct drive torque-controlled robot demonstrate the benefits of the proposed hardware-in-the-loop optimal control method.
- Y. Chen, Y. Li and D. J. Braun, Data-Driven Iterative Optimal Control for Switched Dynamical Systems, IEEE Robotics and Automation Letters, vol. 8, no. 1, pp. 296-303, 2023.
- Y. Chen and D.J. Braun, Iterative Online Optimal Feedback Control, IEEE Transactions on Automatic Control, vol. 66, no. 2, pp. 566-580, 2021.
- Y. Chen and D.J. Braun, Hardware-in-the-Loop Iterative Optimal Feedback Control without Model-based Future Prediction, IEEE Transactions on Robotics, vol. 35, no. 6, pp. 1419-1434, 2019.
Optimal Control of Compliantly Actuated Robots
Compliant actuation is a biologically inspired actuation concept which promises mechanical adaptability and unseen agility for new-generation robots. However, compliant actuation introduces non-linearity, actuation constraints, and control redundancy which make robots challenging to control. In this research, we investigate how impedance control strategies emerge from the first principles of optimality, and how these strategies can be used to actuate compliant variable impedance robots. We used computational optimal control to make the ”DLR David” robot perform a highly dynamic ball-throwing task – similar to javelin throwing at the Olympics –which requires precise timing and perfect execution. The video below shows the result. The behavior of the robot is surprising because it is predicted by numerical optimization without the use of human data or expert demonstration.
- D.J. Braun, F. Petit, F. Huber, S. Haddadin, P. van der Smagt, A. Albu-Schaffer, and S. Vijayakumar, Robots Driven by Compliant Actuators: Optimal Control under Actuation Constraints, IEEE Transactions on Robotics, vol. 29, no. 5, pp. 1085-1101, 2013.