Hayles nodded home Tony Capaldi's free-kick from close range just when it looked as though teenager Levi Porter's first career goal would be enough to earn Leicester their fifth home win of the season.
Iain Hume had earlier given the Foxes the lead before City old boy Lilian Nalis levelled just 90 seconds later.
Porter then struck ten minutes after coming on as a 65th minute substitute. But Hayles popped up from six yards with the seconds ticking down to spoil the 18-year-old's big day and keep Argyle on the fringe of the play-offs.
Plymouth boss Ian Holloway kept the same side that lost to Birmingham last week, whereas Leicester manager Rob Kelly was forced to make five changes through injury and suspension.
But it was the hosts who made the brighter start and visiting keeper Luke McCormick was forced to push Stephen Hughes' powerful drive around the post in the third minute after Plymouth could only half clear a corner. Hume then angled a header narrowly wide before making amends on 25 minutes with the game's first goal.
The Canadian international latched on to Momo Sylla's perfectly weighted throughball and rounded McCormick just inside his area to tap into an empty net.
However, the visitors were level within 90 seconds and it was Leicester old boy Nalis who did the damage. Capaldi's corner was headed straight back to him by team-mate Nick Chadwick and the Northern Ireland international squared for an unmarked Nalis to drill home from 15 yards.
An uninspired second period finally swung into life on 71 minutes when Chadwick struck the bar with a header from a rare Plymouth corner.
And seconds later he was made to pay when Porter capitalised on a slip by Marcel Seip and advanced into the penalty area and beat McCormick with a cool finish.
Plymouth looked a beaten side, but Hayles beat Nils-Eric Johansson to Capaldi's deep free-kick and although Conrad Logan got a hand to the ball, he couldn't keep it out.