Lee Erwin gave Saints an early lead with his first goal for the Paisley club but Shane Duffy equalised and James Forrest put Neil Lennon's men in front before half-time.
Odsonne Edouard saw a second-half penalty saved by Zdenek Zlamal but Celtic picked up the three points to move above Hibs and into second place on goal difference.
