Palace with a goal in each half won the game and go on to face Watford in round 4.
Paul Ifill scored 7 minutes before the break and Sean Scannell doubled the lead 10 mins into the 2nd half.
Gradel's last minute goal gave the travelling Foxes something to cheer. Palace were good for their win and good luck to them in the competition.
Leicester can now focus on promotion back to the Championship and a League meeting at Selhurst Park next season.