A Lawrence Shankland goal after just five minutes, as he looped a header over Jack Butland from a cross, looked to have put Hearts on course for all three points at Ibrox.
As the clock ticked down, Hearts were leading 1-0, but after earlier missing a penalty, James Tavernier popped up when another was awarded and made it 1-1 with just minutes of normal time left.
Then, in nine minutes of injury time, Danilo, on off the bench, popped up to score and hand Rangers an unlikely three points, cutting the gap to Celtic to five points.
Clement was delighted to see his team not giving up in the game and admits he is watching his new players closely to see their reactions in every situation.
The Rangers boss told BBC Scotland: “The most important thing is that it is a deserved win. For me this shows the mentality, not giving up.
“I am still watching the players to see how they react.
“I expect them in every circumstance to never give up and do the things they have to do together and that is what they did today.
“Circumstances were against us, but because of the mentality of the players we got the win.
“The good teams are the teams that never give up.
“There is not one team in the world that wins every game.
“The teams that never give up, they win trophies at the end. That is our ambition that is what we went to do.
“There are still a lot of things that need improved, but they are things to talk about in my dressing room.”
Clement will be looking for his men to now make sure they see off Dundee in an away league clash on Wednesday evening to keep their momentum going.
