Substitute Gaetan Courtet fired Reims into the Ligue 1 top four with two goals in the last six minutes.
Nice looked on course to extend their unbeaten run to nine matches in all competitions when Dario Cvitanich cancelled out Diego's 15th-minute opener for the hosts.
However, Courtet came off the bench to inspire Reims, who extended their own unbeaten sequence to six games.
It has been some start for Hubert Fournier's side following their promotion from Ligue 2 last season.
They were tipped to struggle but have thus far proved their critics wrong, with their results at the Stade Auguste Delaune - this was their fourth consecutive league win at home - particularly encouraging.
They started in impressive fashion tonight, with Diego breaking the deadlock in the 15th minute.
The Brazilian latched on to Antoine Devaux's magnificent through-ball before bursting clear and slotting the ball between Nice goalkeeper Joris Delle's legs.
Yet Nice did have the chance to level proceedings in a quiet first half, with Kevin Diaz heading inches wide before thumping a free-kick just outside the area crashing off the crossbar.
Reims failed to take advantage of the let-offs, though, and Cvitanich deservedly equalised for the visitors in the 62nd minute.
Anthony Weber could only clear Valentin Eysseric's ball into the path of Cvitanich, who fired home.
It was no less than Claude Puel's men deserved but it seemed to bring Reims to their senses as they dominated thereafter.
Delle was forced to make two fine saves from Algeria international Kamel Ghilas before Reims were rewarded for the endeavour four minutes from time.
Courtet, who had been introduced off the bench in place of Nicolas Fauvergue in the 53rd minute, capitalised on a poor clearance to beat Delle with a left-footed strike.
And the 23-year-old sealed the win in the second minute of injury time when he smartly finished at the back post following Devaux's long throw into the area.