Southern League Premier Division side Stourbridge pulled off an FA Cup upset over 10-man Plymouth to book their place in the second round for the first time in their history.
Ryan Rowe: We had belief
Second-half strikes from Paul McCone and Sean Evans sealed a famous replay win for the Glassboys over League Two's bottom side, who had Paul Bignot sent off before the break.
McCone's 52nd-minute header put the home side in front and Evans sealed Stourbridge's passage to a second-round home tie against Stevenage when he bundled home a 73rd-minute rebound.
Grimsby were also celebrating a giantkilling after the former Football League club - now in the Conference Premier - beat League Two Port Vale 1-0 after Serge Makofo's 59th-minute strike.
On a night of upsets, AFC Wimbledon were one of three League Two sides to overturn higher-ranked opposition with Luke Moore's 66th-minute goal giving them a 1-0 win over Scunthorpe at Glanford Park.
Liam Dickinson volleyed a 55th-minute winner as Southend saw off Preston by the same score at Roots Hall, and second-half strikes from Garry Richards and Stefan Payne helped Gillingham hang on for a 3-2 home win over Bournemouth.
Conference Premier leaders Wrexham reached the second round with goals from Mathias Pogba and Danny Wright nudging them to a 2-1 win over Cambridge, who replied late through Michael Wylde.
And Conference Premier Tamworth booked a second-round trip to Gateshead after a dramatic 1-0 win over Hinckley, Kieron St Aimie bundling home a Liam McDonald corner in the dying moments.
Aldershot will travel to Sheffield Wednesday in round two after goals from Alex Rodman and Luke Guttridge gave them a 2-0 win over Maidenhead of the Conference South.
Ben Bradbury's extra-time winner ensured Isthmian Division One North side Redbridge - the lowest ranked side left in the competition - reached round two with a 2-1 replay win at Oxford City.
Lee Steele put the hosts ahead on 20 minutes but Nathan Gordon equalised on 70 and Bradbury netted the winner in the 102nd minute to book his side a trip to League Two side Crawley.