\n<\/p>\n
Last September roboticist Benjie Holson posted the \u201cHumanoid Olympic Games\u201d: a set of increasingly difficult tests for humanoid robots that he demonstrated himself while dressed in a silver bodysuit. The challenges, such as opening a door with a round doorknob, started out easy, at least for a human, and progressed to \u201cgold medal\u201d tasks such as properly buttoning and hanging up a men\u2019s dress shirt and using a key to open a door.<\/p>\n
Holson\u2019s point was that the hard tasks aren\u2019t the dazzling ones. While other competitions feature robots playing sports and dancing, Holson argued that the robots we actually want are the ones that can do laundry and cook meals.<\/p>\n
He expected the challenges to take years to resolve. Instead, within months, robotics company Physical Intelligence completed 11 of the 15 challenges\u2014from bronze to gold\u2014with a robot that washed windows, spread peanut butter and used a dog poop bag.<\/p>\n
If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\n
Scientific American spoke to Holson about why vision-only, or camera-based, systems are outperforming his expectations and how close we are to a genuinely useful machine. He has since released a new, more difficult set of challenges.<\/p>\n
[An edited transcript of the interview follows.]<\/p>\n
You designed these challenges to be hard. Were you surprised by how quickly the results came in?<\/p>\n
It was so much faster than I was expecting. When I chose the challenges, I was trying to calibrate them so some bronze ones would get done in the first month or two, then silver and gold in the next six months, and the most difficult ones might take a year or a year and a half. To have them do basically almost all of them in the first three months is wild.<\/p>\n
What made that possible?<\/p>\n
I started with the premise that we have things that look impressive at a fairly narrow set of tasks\u2014vision-only, no touch, simple manipulator, not incredible precision. That limits what you can be good at. I tried to think of tasks that would require us to break forward out of that set. It turns out I wildly underestimated what\u2019s possible with vision-only and simple manipulators.<\/p>\n
When I visited Physical Intelligence, I learned they don\u2019t have any force sensing. They\u2019re doing all of that 100 percent vision-based. The key-insertion task, the peanut butter spreading\u2014I thought those would require force inputs. But apparently you just throw more video demonstrations at it, and it works.<\/p>\n
How exactly do you train a robot to do that without coding it line by line?<\/p>\n
It\u2019s all learning from demonstration. Somebody teleoperates the robot doing the task hundreds of times, they train a model based on that, and then the robot can do the task.<\/p>\n
There is a lot of confusion about whether large language models (LLMs) are useless for robots. Are they?<\/p>\n
I used to be fairly dubious of the utility of LLMs in robotics. The problem they were good at solving two or three years ago was high-level planning\u2014\u201cIf I want to make tea, what are the steps?\u201d Ordering the steps is the easy part. Picking up the teapot and filling it is the really challenging thing.<\/p>\n
On the other hand, we\u2019ve started doing vision-action models using the same transformer architecture [as that used in LLMs]. You can use transformers for text in, text out, images in, text out\u2014but also images in, robot actions out.<\/p>\n
The neat thing is they\u2019re starting with models pretrained on text, images, maybe video. Before you even start training your specific task, the AI already understands what a teapot is, what water is, that you might want to fill a teapot with water. So while training your task, it doesn\u2019t have to start from, \u201cLet me figure out what geometry is.\u201d It can start with, \u201cI see, we\u2019re moving teapots around\u201d\u2014which is wild that it works.<\/p>\n
How did you come up with the \u201cOlympic\u201d tasks?<\/p>\n
So part of it was a challenge and part of it was a prediction. I tried to think of the next set of things that we can\u2019t do now that someone\u2019s going to be able to do soon.<\/p>\n
Humans rely on touch to do things such as finding keys in a pocket. How do we get around that in robotics?<\/p>\n
That\u2019s a very good question we don\u2019t know the answer to yet. Touch technology is way worse, more expensive, delicate and far behind cameras. Cameras, we\u2019ve been working on for a long time.<\/p>\n
The big question is: Are cameras enough? Both Physical Intelligence and Sunday Robotics [which completed the bronze-medal task of rolling matched socks] have made the bet that putting a camera on the wrist, very close to the fingers, lets you kind of see forces by seeing how everything smushes. When the robot grabs something, it sees the fingers have some rubber that deflects; the object deflects, and it infers forces from that. When smearing peanut butter on bread, the robot watches the knife deflect down and crush the bread and judges forces from that. It works way better than I expected.<\/p>\n
What about safety?<\/p>\n
The energy needed to stay balanced is often quite high. If a robot is falling, that\u2019s a very fast, hard acceleration to get the leg in front in time. Your system has to inject a lot of energy into the world\u2014and that\u2019s what\u2019s unsafe.<\/p>\n
I\u2019m a huge fan of centaur robots\u2014mobile wheel base with arms and a head. For safety, that\u2019s such an easier way to get there quickly. If a humanoid loses power, it\u2019s going to fall down. The general plan seems like it\u2019s to make a robot so incredibly valuable that we as a society create a new safety class for it\u2014like bicycles and cars. They\u2019re dangerous but so valuable that we tolerate the risk.<\/p>\n
Have these results changed your time line?<\/p>\n
I used to think home robots were at least 15 years away. Now I think at least six. The difference is I thought it would be much longer before doing a useful thing in a human space, even as a demo, was plausible.<\/p>\n
But roboticists have seen time and again there\u2019s a long road between \u201cit worked in a lab and I got a video\u201d and \u201cI can sell a product.\u201d Waymo was driving on roads in 2009; I couldn\u2019t buy a ride until 2024. It takes a long time to get reliability squared away.<\/p>\n
What\u2019s the biggest bottleneck left?<\/p>\n
Reliability and safety\u2014the stuff Physical Intelligence shows is incredibly impressive, but if you put it on a different table with different lighting and use a different sock, it might not work. Each step toward generalization seems to take an order of magnitude more data, turning days of data collection into weeks or months.<\/p>\n","protected":false},"excerpt":{"rendered":"
Last September roboticist Benjie Holson posted the \u201cHumanoid Olympic Games\u201d: a set of increasingly difficult tests for humanoid robots that he demonstrated himself while dressed in a silver bodysuit. The challenges, such as opening a door with a round doorknob, started out easy, at least for a human, and progressed to \u201cgold medal\u201d tasks such<\/p>\n","protected":false},"author":1,"featured_media":45668,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[50],"tags":[1514,1845,10729,5463,585,977,4962],"class_list":{"0":"post-45667","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-environment","8":"tag-everyday","9":"tag-expected","10":"tag-faster","11":"tag-humanoid","12":"tag-learning","13":"tag-robots","14":"tag-tasks"},"_links":{"self":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/45667","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=45667"}],"version-history":[{"count":0,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/45667\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/media\/45668"}],"wp:attachment":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=45667"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=45667"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=45667"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}