All Transform 2021 sessions are available on demand now. Look now.

At the end of last week, OpenAI confirmed that it had closed its robotics division in part due to difficulties in collecting the data needed to cross technical barriers. After years of research into machines capable of learning to perform tasks such as solve a Rubik’s Cube, company co-founder Wojciech Zaremba said it made sense for OpenAI to focus on other areas, where training data is more readily available.

Beyond the commercial motivations of abandoning robotics in favor of media synthesis and natural language processing, OpenAI’s decision reflects a growing philosophical debate in AI and robotics research. Some experts believe that simulation training systems will be sufficient to build robots capable of performing complex tasks, such as assembling electronic components. Others stress the importance of collecting real-world data, which can provide a more solid baseline.

A long-standing challenge in simulations involving real data is that each scene must respond to the movements of a robot, even if those that may not have been recorded by the original sensor. Whatever angle or point of view that is not captured by a photo or video, it must be rendered or simulated using predictive models, which is why simulation has historically relied on computer-generated graphics and physics-based rendering that somewhat crudely represents the world.

But Julian Togelius, an AI and games researcher and associate professor at New York University, notes that robots pose challenges that don’t exist within the confines of simulation. Batteries run out, tires behave differently when hot, and sensors need to be recalibrated regularly. Plus, robots break and tend to be slow – and cost a pretty dime. The Shadow Dexterous Hand, the machine that OpenAI used in its Rubik’s Cube experiments, has a starting price of several thousand. And OpenAI had to improve the robustness of the hand by reducing its tendon stress.

“Robotics is an admirable business, and I have great respect for those who try to tame mechanical beasts,” Togelius wrote in a tweet. “But that’s not a reasonable way to do reinforcement learning, or any other kind of episode-heavy learning. In my humble opinion, the future belongs to simulations.

Train robots in simulation

Gideon Kowadlo, co-founder of Cerenaut, an independent research group developing AI to improve decision making, argues that no matter how much data is available in the real world, there is more data in the simulation – data that is easier to control, ultimately. Simulators can synthesize different environments and scenarios to test algorithms under rare conditions. Additionally, they can randomize variables to create various training sets with varying objects and environment properties.

Indeed, Ted Xiao, a scientist in Google’s robotics division, argues that OpenAI’s abandonment of working with physical machines should not mean the end of the lab’s research in this direction. By applying techniques such as reinforcement learning to tasks such as understanding language and code, OpenAI might be able to develop better performing systems that can then be reapplied to robotics. For example, many robotics labs use humans holding controllers to generate data to train robots. But a general AI system that includes controllers (i.e. video games) and robotics video feeds with cameras could learn to teleoperate quickly.

Recent studies suggest how a simulation-based approach to robotics might work. In 2020, Nvidia and Stanford developed a technique that breaks down vision and control tasks into machine learning models that can be trained separately. Microsoft has created an AI drone navigation system that can reason the correct actions to take from the camera images. A scientist from DeepMind has trained a cube stacking system learn from observation in a simulated environment. And a Google team detailed a frame which takes a motion capture clip of an animal and uses reinforcement learning to form a control policy, using an adaptation technique to randomize the dynamics in the simulation, for example by varying the mass and the friction.

In a blog post in 2017, OpenAI researchers wrote that they believe general-purpose robots can be built by training entirely in simulation, followed by a small amount of real-world self-calibration. . Increasingly, this would appear to be the case.

For AI coverage, send advice on Kyle wiggers – and don’t forget to subscribe to AI Weekly Bulletin and add our AI channel to your favorites, The machine.

Thanks for reading,

Kyle wiggers

IA personal writer


VentureBeat’s mission is to be a digital public place for technical decision-makers to learn about transformative technology and conduct transactions. Our site provides essential information on data technologies and strategies to guide you in managing your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the topics that interest you
  • our newsletters
  • Closed thought leader content and discounted access to our popular events, such as Transform 2021: Learn more
  • networking features, and more

Become a member


Please enter your comment!
Please enter your name here