Slovenia went two up against USA after Valter Birsa and Zlatan Ljubijankic goals, but Landon Donovan got one back before Michael Bradley rescued a point. Maurice Edu looked to have scored a winner for the Americans but it was controversially disallowed.
It leaves all to play for in Group C going into the final round of matches. There was a very early stoppage when Clint Dempsey caught Ljubijankic with an elbow to the head as the pair challenged for a high ball near the halfway line.
The Gent striker was fine to continue and Malian referee Koman Coulibaly opted to take no action against Dempsey.
Birsa created the first chance with a whipped cross from the right which Milivoje Novakovic failed to connect with on the volley, but the classy Slovenia number 10 took charge himself after 12 minutes. He was allowed to turn and take stock before curling a shot past the helpless Tim Howard from 25 yards, with no American defender going to close him down.
Francisco Torres tested Handanovic with a left-foot free-kick from a wide angle, with the Udinese goalkeeper probably expecting a ball into the box rather than a cross.
It lifted the US, who had Robbie Findley booked for a handball in the Slovenia box which ruled him out of the final group match against Algeria, and Slovenia full-back Miso Brecko had to get a vital touch to deny Donovan a far-post tap-in after a flowing move involving Altidore, Findley and Dempsey.
But in a flash Bob Bradley's men were 2-0 down as Novakovic slotted a pass through to strike partner Ljubijankic, who beat the offside trap and calmly rolled a right-foot shot under Howard. The US coach responded by bringing on Rangers midfielder Edu for Torres and Benny Feilhaber for Findley at the start of the second half.
Within three minutes of the restart the Americans had halved the deficit when right-back Steve Cherundolo's ball down the flank evaded Bostjan Cesar and Los Angeles Galaxy star Donovan raced into the box at an angle before drilling the ball hard and high past Handanovic at his near post.
Marko Suler then injured himself as he climbed over Altidore and from the resultant free-kick Dempsey's glancing header from Donovan's inswinging ball was just wide.
Howard then had to claw away a Birsa free-kick as Suler threatened to head it home and Suler was then involved at the other end to tackle Altidore after the former Hull loanee looked set to get clear of him just before the hour mark.
The game was now evenly poised and after Suler manhandled Altidore once too often for the referee's liking and was booked, the burly striker fired a shot straight at Handanovic. Andraz Kirm joined Suler in the book for going in late
on Cherundolo and so too did Bojan Jokic for a clumsy foul on Donovan.
Bradley's final change was to bring on forward Herculez Gomez for defender Oguchi Onyewu as the US pressed for the leveller - and it was the coach's son who got it. Altidore climbed high to nod down a long ball into the box and Bradley junior sprinted into the box to fire past Handanovic and send the majority of the Ellis Park crowd into hysterics with eight minutes to go.
They thought they had a winner when Edu turned in Donovan's free-kick but the referee disallowed it, apparently seeing a tug in the box although both sides appeared equally guilty of that. Slovenia pushed forward for the glory goal and Aleksandar Radosavljevic forced an excellent parry from Howard from 25 yards as time ticked away.