It was a priceless strike from the lively Porter - coming just nine minutes after Lewis Gobern had cancelled out Elvis Hammond's first-half opener for Kelly's side.
The former Wolves midfielder's team had been in free-fall after a disastrous spell that had seen them pick up just one point from their previous six games.
But they looked anything but a team on that kind of run to take full advantage of lethargic Wolves and record their first win at Molineux since 1996.
It was a hugely disappointing display by Wolves who failed to build on last weekend's hard-earned win at Southend - only their second in nine games.
Molineux has been something of a fortress for them this season. But they did not get anywhere near to the heights they have hit at home as they slipped to their first defeat in front of their own fans since losing 1-0 to Derby County on September 12.
After an abject first-half display, Wolves did at least show more endeavour after the break and looked to have rescued a point when Gobern fired home from 25 yards after Leicester had only half-cleared a Michael McIndoe corner.
But Leicester were not to be denied and Porter capped a highly impressive display when he cut in from the left wing and fired home from the edge of the area in the 89th minute.
Leicester were well worth their half-time lead as they carried more of a consistent threat than Wolves in the opening period.
The surprise was that it took them until the 38th minute to break the deadlock. Hammond capitalised on a neat piece of play from his strike partner, Canadian international Iain Hume, who headed the ball into his path.
The pacy Hammond easily had the beating of Jody Craddock and surged past the Wolves skipper before drilling a low drive past stranded goalkeeper Matt Murray.
The shock of falling behind belatedly sparked Wolves into life and they were only denied an equaliser on the stroke of half-time by the crossbar when Gobern crashed his 30-yard drive against the woodwork.