News

Optimal control by experiment: NSF Grant

Aug 2020

The best plan on paper

Optimal control computes the best possible motion for a robot: the fastest, the most efficient, the most precise. The calculation relies on a model of the robot, and no model is exact. Friction, flexibility, and small differences between parts make the real robot behave differently from the simulated one, so the motion that is optimal in simulation is no longer optimal on the real machine. This difference is known as the sim-to-real gap. A golfer faces the same problem: physics can describe the ideal swing, but the swing is perfected on the course, shot by shot, from what the ball actually does.

Closing the sim-to-real gap

This National Science Foundation grant supported research on a way to close this gap: optimal control by experiment. Instead of relying on the predictions of an uncertain model, the robot improves the motion during physical experiments, much like the golfer on the course. An approximate model still has a role: it speeds up the computation. Measured data, however, decide each improvement. The method is half model-based and half data-driven, and fast enough to run in real time, while the robot operates.

What the research has shown

The project developed new methods for optimal control by experiment:

  • Optimal control without a model of the dynamics: a method that learns the optimal controller over a fixed time horizon from interaction alone, with a proof that it converges to the optimal solution (Systems & Control Letters, 2025).
  • Optimal control for robots that switch between modes of motion, such as a hopping robot that alternates between flight and ground contact: the method determines both the control inputs and the switching times directly from measured trajectories (IEEE RA-L, 2023).
  • In numerical studies, the new methods outperformed feedback control, model-based reinforcement learning, and model predictive control, especially for underactuated and unstable robots.

Where it could lead

  • Complex robots with many degrees of freedom, such as humanoid and legged robots, which are difficult to model precisely.
  • Factory robots that improve a task directly on the production line, without time-consuming reprogramming.

This work was supported by the National Science Foundation grant “Robots Teaching Robots: Real-Time Optimal Control of Complex Engineering Systems” (Award Number 2029181) under the Dynamics, Control, and Systems Diagnostics program within the Division of Civil, Mechanical, and Manufacturing Innovation.