Michael Moffat was 10-man Ayr's matchwinner, as his stoppage-time goal cut Stranraer adrift at the foot of the Irn-Bru Scottish Second Division table with a 2-1 home victory.
The hosts were looking for revenge over their south-west rivals following a 2-0 defeat earlier in the season, and were inches away from taking the lead when Mark Roberts' 11th-minute effort hit the post.
Stephen Aitken's men lost 5-0 on their last visit to Somerset Park, but took the lead when Michael Moore struck from close range after half an hour.
Moffat's 57th-minute penalty levelled the scores however, after Grant Gallagher upended Roberts.
Marc Twaddle struck the crossbar seven minutes later as the Honest Men mounted second-half pressure, before Ryan McStay was dismissed for violent conduct nine minutes from time to give the away side a numerical advantage.
But the visitors were unable to hold onto their point as Moffat's 92nd-minute finish earned a late victory.