The two leaders were at an Australia Day event when Aboriginal rights protesters started banging on the glass building, shouting “shame” and “racist”.
The protest lasted 20 minutes until at around 2.30pm, around 50 riot police came to escort Gillard and Abbott away from the Lobby restaurant near Old Parliament House.
Gillard lost her footing and a shoe as she was ushered, shaken but not hurt, to a waiting car by armed officers.
The protesters banged on the roof and bonnet as it drove Gillard and Abbott away.
The protest is said to have began after Abbot questioned the relevance of a makeshift Aboriginal embassy in a tent, which was celebrating its fortieth anniversary.
Between 100 and 200 protesters gathered at the embassy, which is on the lawn of the Federal Parliament, close to the restaurant Gillard and Abbott were dining at.
Aboriginal community leader Fred Hooper said the event was peaceful until Abbott made the comments.
He said: “The opposition leader on national television made a comment to tear down something that have built over 40 years, which is sacred to us.”
“So what do you expect us to do when we’re 200 yards away from the person that makes that comment? Do you expect us to say, ‘yeah Tony we’re gonna do that now? We’re gonna rip it down?'”
Abbot had said earlier that day, in reference to the Tent Embassy: ”I think the indigenous people of Australia can be very proud of the respect in which they are held by every Australian and yes, I think a lot has changed since then and I think it probably is time to move on from that.”
Protester Kalara Gilbert said: “What Tony Abbott said was disgraceful. Our people are living in awful third world conditions, they’re dying and this government just continues to turn a blind eye to it.”
Gillard said later: “Oh I’m fine. I’m fine. The only thing that really kind of angers me about it is that it disrupted such a wonderful event for great people.”
“I am made of pretty tough stuff and the police did a great job.”
