A Gareth McAuley own goal gave Craig Levein's side the lead after 27 minutes but Connor Goldson equalised and Alfredo Morelos had Steven Gerrard's team in front by half-time.
Rangers midfielder Scott Arfield was sent off with 18 minutes left but the visitors held on to move a point ahead of Celtic.
