Success for the Tigers, follows on from victories over the West Indies and last year against the touring England side.
In a low-scoring test match, Australia were set 265 to win after bowling out Bangladesh for 260 and 221.
But after losing early wickets in the second innings, opener David Warner, compiled a superb 112 as Australia were on course at 158-3.
Until spinner Shakib Al Hasan took 5-85 and spin his team to victory and a possible first series win over one of the big nations in world cricket.
Chittagong hosts the final test of the series and Australia will be keen to win and level up against the 9th ranked team in the world rankings.
