Kyle Patterson pounced on a terrible late error from Gateshead goalkeeper Jak Alnwick to fire Tamworth into the FA Cup third round courtesy of a 2-1 victory.
Francino Francis put the Lambs ahead after 29 minutes when he capitalised on Alnwick's first mistake, before Kieron St Aimie missed from the penalty spot after defender Chris Carruthers had seen red for a foul on Danny Mills.
Ten-man Gateshead regrouped at the break and Nathan Fisher levelled just after the hour mark, but Blue Square Bet Premier rivals Tamworth booked their place in the next round when Alnwick again dropped a 90th-minute corner and Patterson finished.
Heed fired the first shot in anger when Chris Moore cut infield and shot over after 20 minutes yet it was Tamworth who took the lead when Francis pounced on an error by Alnwick.
The young stopper spilt Samuel Habergham's corner and Francis poked the ball home from two yards, Danny Thomas then wasting a glorious chance to double the lead after Alnwick had denied Mills.
Gateshead's task was made even harder just before the break when Carruthers saw red for hauling down Mills in the box, but Alnwick redeemed his earlier error by saving St Aimie's penalty and Mills headed over from the rebound.
The Lambs were punished for their wasteful finishing when Eddie Odhiambo found Fisher in the box and he swivelled before lashed home his second goal of the season, but Patterson was on hand to win it for the visitors.