Welcome to ∇Q

If I could use one word to describe the advancement of ML or AI in the past couple of years it would be scale. When GPT-31 was released in 2020 and ChatGPT2 in 2022, I was impressed with the performance and thought it was essentially a step towards generalisation in Natural Language Processing (NLP), similar to how AlexNet3 allowed for generalization in image classification. I did not fully grasp the significance Large Language Models (LLMs) would have on robotics and ML in general....

Imagine teaching a robot to pick up a coffee cup in a simulation or video game. In this perfect virtual world, the cup’s weight is precisely known, the lighting is consistent, and the robot’s sensors provide exact measurements. Now try the same task in the real world. The cup might be heavier than expected, it’s surface more slippery, the lighting creating unexpected shadows, and the robot’s sensors noisy. This disconnect between simulation and reality, known as the reality gap, is a fundamental challenge in robotic learning....

In this post, we’ll explore the fundamental methods used to teach robots new skills. The three main paradigms we’ll explore are: Imitation Learning: Teaching robots by showing them what to do Reinforcement Learning: Letting robots discover solutions through experience Supervised Learning: Using labeled data to build core perception and planning capabilities Each of these approaches tackles the fundamental challenges of robotic learning in different ways, and modern systems often combine them to leverage their complementary strengths....

To understand why robot learning is fundamentally different from traditional machine learning, let’s start with a simple example. Imagine teaching a robot to pick up a coffee cup. While a computer vision system needs only to identify the cup in an image, a robot must answer a series of increasingly complex questions: Where exactly is the cup? How should I move to grasp it? How hard should I grip it? What if it’s fuller or emptier than expected?...

Robot learning combines robotics and machine learning to create systems that learn from experience, rather than following fixed programs. As automation extends into streets, warehouses, and roads, we need robots that can generalise, taking skills learned in one situation and adapting them to the countless new scenarios they’ll encounter in the real world. This series explains the key ideas, challenges, and breakthroughs in robot learning, showing how researchers are teaching robots to master flexible, adaptable skills that work across the diverse and unpredictable situations of the real world....