Lee Currie scored twice as Berwick won 3-0 at Albion to move to within three points of the play-off places.
Albion could have moved into the top four themselves had they won, but never recovered from falling behind after just four minutes.
It was Paul Currie who opened the scoring, firing high into the top corner after being set up by Ross Gray inside the penalty area.
And 11 minutes later the visitors were 2-0 up as Lee Currie slotted a low effort past Neil Parry.
Lee Currie then wrapped up the game with his second from the spot. Barry Russell was booked after tripping Gray in the area and Lee Currie tucked in his ninth goal of the campaign.