But it would be unfair to single out Taylor - on loan from Arsenal - for being solely responsible for a defeat from which no Leicester player emerged with much credit.
Taylor's error came on 36 minutes when Leicester failed to clear the danger after he had dropped and then fumbled a Paul McLaren corner and in the scramble that followed, Shaun Barker bundled the ball over the line from close range.
Rotherham fully merited their lead. Leicester went close just the once in the first half when on 19 minutes Lee Morris aimed a 12-yard angled shot towards the far corner of the United goal with Mike Pollitt intervening with a brilliant fingertip save.
Apart from that the first half belonged to the visitors. Junior tested the City keeper first with a rising shot from inside the penalty area and then just before the goal Junior again threatened with a 20-yard free-kick which Taylor did well to keep out.
On the half-hour mark an appalling backpass by Keith Gillespie was seized upon by Martin Butler, whose shot beat Taylor and seemed destined for a certain goal until Dion Dublin got back to clear off the line.
City did little better after the break with David Connolly and substitute Trevor Benjamin getting in headers which were comfortably dealt with by Pollitt.
Butler almost put his side two up on 70 minutes when Taylor was forced to palm away Butler's chipped shot.
And in stoppage time Leicester were denied by the woodwork when Connolly thudded a close-range header against the bar and then Pollitt ensured Rotherham's win at the death when he flung himself backwards to block Joey Gudjonsson's goalbound shot.