Unite confirmed that it has given 21 London bus companies notice of a ballot, which will involve 21,000 Unite members in London.
The union is asking for a £500 payment to recognise the “massive increase” in workload during the Games in the capital.
It added that 800,000 extra passengers were expected to travel on buses during the Olympics.
Unite official Peter Kavanagh said that bus workers would be going “above and beyond the call of duty” over the Olympics, and accused the London Mayor of failing to intervene on the workers’ behalf.
“Boris Johnson has sat on his hands for almost a year while the bus operators have refused to even talk to Unite about their workers’ extra contribution to the Olympics,” he said.
Transport for London’s managing director Leon Daniels said of the dispute: “London bus drivers are employed by private bus companies and their pay and conditions are set by those companies.
“If bus drivers are required to work additional hours they are always paid overtime accordingly.”