Stuart Attwell has lost his place on the list of Select Group of referees and will return to Football League duty on the National List.
The move, which is said to come by mutual consent, comes after Attwell made a string of high-profile errors after being promoted to the elite group of referees at the start of the 2008-09 season aged just 25.
Only weeks into that season, Attwell, now 29, attracted heavy criticism for awarding Reading a goal against Watford when John Eustace had kicked the ball over the byline four yards wide of the goal before it was put back into play by Noel Hunt. Attwell's linesman Nigel Bannister signalled for a goal.
More recently in December, Attwell sent off Gary Cahill, then of Bolton, for fouling Scott Parker only yards over the halfway line, saying he had denied Tottenham a clear goalscoring opportunity. The red card was overturned on appeal.
In a statement, Professional Game Match Officials Limited general manager Mike Riley backed Attwell to improve as a referee and return to the top level.
"Throughout his career in the Select Group Stuart has demonstrated great courage and mental strength in responding to the challenges that he has faced," Riley said. "He has shown many of the attributes of a top flight official but the Premier League is an exacting standard with huge demands. It is imperative that PGMOL deliver the best officiating possible to the English game at all levels.
"This is not the end for Stuart. There are several match officials who have returned to the National List and benefited from it. Stuart has a high level of maturity and responsibility and I'm convinced that he has a long-term future as a referee at the very highest level.
"PGMOL sincerely hopes to be welcoming Stuart back to the Select Group in the future."
Attwell said: "I have learned a great deal from my involvement in the Select Group over the last four years and I am now looking forward to building on that valuable experience.
"I strongly believe I have a lot to offer refereeing, and I now look forward with commitment and enthusiasm to the new challenge that I am undertaking. This is supported by a determination to continue my career development, and return to the Select Group as soon as possible."
Attwell spent only one year on the national list prior to his promotion to the select group four years ago.
He made his Premier League debut as a referee in the 1-1 draw between Blackburn and Hull in August 2008.
Following the Watford-Reading game, further controversies followed when he was accused of "losing control" of a Derby-Nottingham Forest derby, having disallowed two Derby goals in the final stages, booking eight players and sending off Forest's Lewis McGugan.
He has also attracted the ire of Wigan manager Roberto Martinez, who accused the referee of lying about his reasons for sending off Gary Caldwell in April 2010, while Stoke's Danny Higginbotham claimed Attwell could be too easily influenced by players.
Later that year, he awarded Liverpool a highly controversial goal against Sunderland. He had awarded the Black Cats a free-kick inside their own half and Michael Turner touched the ball back to Simon Mignolet, apparently for the goalkeeper to take the set-piece, but Attwell ruled the ball active, allowing Fernando Torres to steal it and set up Dirk Kuyt to finish into an empty net.