New Computer Program Can Predict How Things Will Move, Just Like Us

To some extent, we humans are pretty good at predicting the future. Not the big stuff of course, but small things, like how something will move under different forces and conditions. Now, researchers are trying to give computers that same ability.

Let's say you have a heavy block and a rubber ball sitting at the top of a steep ramp, with you holding each one in place. What happens if you just let go (no pushing allowed)?

We can predict that if you let go of the block, it's probably not going to move down the hill as fast as the ball, if it moves at all. You know that round things roll, and things with edges generally don't. Even though physics governs the actions of the two objects, you don't have to have a physics background to make the guess, you just know. How do you know? Because as a kid, you probably played with blocks and balls and ramps. All your experiences helped you make that prediction in a split second.

But computers generally aren't sent outside to play, so they don't learn how objects interact with the world. Until now. A group of scientists at MIT's Computer Science and Artificial Intelligence Lab (CSAIL) have developed a computer model called Galileo that is able to watch videos of different objects interacting in various situations (like a block sliding down a ramp to crash into something else) to estimate how heavy the objects are, and predict what they'll do in other situations.

"From a ramp scenario, for example, Galileo can infer the density of an object and then predict if it can float," postdoctoral researcher Ilker Yildirim, co-author of the research, said in a statement ^[1]. "This is just the first step in imbuing computers with a deeper understanding of dynamic scenes as they unfold."

Yildirim and his co-authors first showed Galileo 150 videos before adding some human intuition. Or, rather, computer intuition. They linked Galileo with Bullet, computer software used by video games and movies as a 'physics engine' capable of making animated graphics look incredibly real by simulating how physics works in the real world. Then they added algorithms that allowed Galilleo to learn from its previous experiences, just like humans, and tested it against people, having both the computer and humans predict how an object would move during experiments. They found that humans and the computer had very similar predictions.

You can test how you compare to Galileo using this website ^[2] created by CSAIL. After watching a short clip of an object sliding down a ramp to hit a block, click on the object that you think is heavier. You can check and see if your answer is correct, and whether Galileo managed to guess correctly.

Next, the researchers want to go even further, working with Galileo on more complicated predictions involving fluids or springs, and eventually getting to a point where it can make predictions in the natural world even faster than we can.

"Imagine a robot that can readily adapt to an extreme physical event like a tornado or an earthquake," co-author Joseph Lim said. "Ultimately, our goal is to create flexible models that can assist humans in settings like that, where there is significant uncertainty."