Real-time optimal control of robots

Optimal control computes the best possible motion for a robot, but the calculation relies on a model, and no model is exact. The motion that is optimal in simulation is therefore no longer optimal on the real machine: this is the sim-to-real gap. This research develops optimal control by experiment, in which the robot improves the motion during real experiments, so that measured data, not model predictions, decide how the robot moves.

The sim-to-real gap

Model-based optimal control computes efficiently, but because the model is uncertain, the resulting performance is conservative. Data-driven optimal control reduces the effect of an uncertain model, but needs a large amount of training data, which is slow to collect on real robots. Neither approach alone closes the gap for complex engineering systems, such as assistive, industrial, and medical robots, where accurate models are hard to obtain.

Half model-based, half data-driven

The method developed here combines both approaches. An approximate model speeds up the computation, while measured data decide each improvement, which avoids the errors of predicting future motion with an inexact model. The result is optimal control that is half model-based and half data-driven, and fast enough for real-time hardware-in-the-loop control. Experiments on a three-link, direct-drive, torque-controlled robot demonstrate the benefits of the method.

• Learning Finite-Horizon Optimal Control with Unknown Control-Affine Dynamics

  Y. Chen, Y. Li and D. J. Braun

  Systems & Control Letters, vol. 203, Art. no. 106161, 2025

  DOIBibTeX

  @article{Chen2025,
    author  = {Y. Chen and Y. Li and D. J. Braun},
    title   = {Learning Finite-Horizon Optimal Control with Unknown Control-Affine Dynamics},
    journal = SCL,
    volume  = {203},
    pages   = {106161},
    note    = {Art. no. 106161},
    year    = {2025},
    doi     = {10.1016/j.sysconle.2025.106161}
  }

• Data-Driven Iterative Optimal Control for Switched Dynamical Systems

  Y. Chen, Y. Li and D. J. Braun

  IEEE Robotics and Automation Letters, vol. 8, no. 1, pp. 296–303, 2023

  DOIVideoBibTeX

  @article{Chen2023,
    author  = {Y. Chen and Y. Li and D. J. Braun},
    title   = {Data-Driven Iterative Optimal Control for Switched Dynamical Systems},
    journal = IEEEral,
    volume  = {8},
    number  = {1},
    pages   = {296--303},
    year    = {2023},
    doi     = {10.1109/LRA.2022.3226075}
  }

• Iterative Online Optimal Feedback Control

  Y. Chen and D. J. Braun

  IEEE Transactions on Automatic Control, vol. 66, no. 2, pp. 566–580, 2021

  DOIBibTeX

  @article{Chen2021,
    author  = {Y. Chen and D. J. Braun},
    title   = {Iterative Online Optimal Feedback Control},
    journal = IEEEtac,
    volume  = {66},
    number  = {2},
    pages   = {566--580},
    year    = {2021},
    doi     = {10.1109/TAC.2020.2986211}
  }

• Hardware-in-the-Loop Iterative Optimal Feedback Control without Model-Based Future Prediction

  Y. Chen and D. J. Braun

  IEEE Transactions on Robotics, vol. 35, no. 6, pp. 1419–1434, 2019

  DOIVideo 1Video 2BibTeX

  @article{Chen2019a,
    author  = {Y. Chen and D. J. Braun},
    title   = {Hardware-in-the-Loop Iterative Optimal Feedback Control without Model-Based Future Prediction},
    journal = IEEEtr,
    volume  = {35},
    number  = {6},
    pages   = {1419--1434},
    year    = {2019},
    doi     = {10.1109/TRO.2019.2929014}
  }

Optimal control of compliantly actuated robots

Compliant actuators, inspired by muscles and tendons, give robots mechanical adaptability and agility. They also make robots harder to control: the actuators are nonlinear, have limits on range, speed, and force, and offer more ways to move than the task requires. This research shows how impedance control strategies follow from the first principles of optimality, and how to use them on robots with variable impedance actuators. On the DLR Hand Arm System, a humanoid arm now called David, optimal control produced a dynamic ball throw that requires precise timing. The motion emerged from numerical optimization alone, without human data or demonstrations, and the transfer from simulation to the real robot took two weeks.

• Robots Driven by Compliant Actuators: Optimal Control under Actuation Constraints

  D. J. Braun, F. Petit, F. Huber, S. Haddadin, P. van der Smagt, A. Albu-Schaffer and S. Vijayakumar

  IEEE Transactions on Robotics, vol. 29, no. 5, pp. 1085–1101, 2013

  2013 King-Sun Fu Memorial IEEE Transactions on Robotics Best Paper Award

  DOIVideoBibTeX

  @article{Braun2013,
    author  = {D. J. Braun and F. Petit and F. Huber and S. Haddadin and P. van der Smagt and A. Albu-Sch\"{a}ffer and S. Vijayakumar},
    title   = {Robots Driven by Compliant Actuators: Optimal Control under Actuation Constraints},
    journal = IEEEtr,
    volume  = {29},
    number  = {5},
    pages   = {1085--1101},
    year    = {2013},
    doi     = {10.1109/TRO.2013.2271099}
  }