Taylor was given a frosty reception on his return to the club where he spent more than £20million in just 15 months, resulting in just 19 wins from the 54 games he was in charge of the East Midlands outfit.
Leicester enjoyed the better of the opening exchanges but had to wait until the ninth minute for their first chance when Boaz Myhill found Iain Hume's long-range shot too hot to handle and with the Hull defence unable to clear Alan Maybury drove the loose ball wide.
Hull's response was a chance for Stuart Green which he bundled wide and a tame header from Damien Delaney.
But the crowd was stirred from its slumbers with a well-crafted goal on the half-hour mark when Hume registered his eighth of the season.
His clever run into the penalty area was spotted by Gudjonsson who played a perfect pass through the Hull defence for Leicester's top-scorer to sweep the ball home from 10 yards.
Their joy was short lived however with the visitors leveling on 35 minutes when sloppy defending allowed Mark Noble to cross to the far post where the unmarked Stuart Elliott rose unchallenged to plant an easy header beyond Paul Henderson.
Leicester almost recaptured the lead on 40 minutes when Delaney's header from a Gudjonsson cross bounced off the crossbar and out for a corner.
The Foxes squandered a glorious chance soon after the restart when Gudjonsson's long ball picked out Matt Fryatt who, after rounding the full-back, shot weakly at the keeper with both Richard Stearman and Maybury standing unmarked in front of goal.
Hume then lifted a shot on to the roof of the net, Fryatt nodded down for Stephen Hughes to fire just wide and Fyatt's mazy run was halted by Alton Thelwell's last-ditch tackle as Leicester continued to create the better chances.
And the home side finally broke through on 64 minutes with Gudjonsson taking centre stage in spectacular fashion with a goal from the halfway line.
The Icelandic international picked up a loose ball and after spotting Myhill stranded off his line crashed in a long-range shot that rocketed over the keeper and into the roof of the net.
Henderson then covered himself in glory with a fantastic point-blank save to deny Jon Parkin but the Leicester keeper was well beaten on 75 minutes when Parkin played the ball in for Green to drill home from 15 yards.
But Leicester continued to play the more attractive football and were rewarded on 84 minutes with another magnificent strike from Gudjonsson.
This time he was picked out by Nils-Eric Johansson and from just outside the penalty he rifled home a terrific low shot.