The Ewood Park side had a loan agreement with a £10m permanent move in the event of promotion wrapped up with Forest.
O’Brien was put through his medical paces by Blackburn, but the Championship side missed some paperwork in their submission to the EFL.
They appealed against the decision in order to be able to complete the capture of O’Brien, but after considering their case, the EFL has rejected it.
The EFL has not accepted Blackburn’s appeal and O’Brien will return to Nottingham Forest.
O’Brien could well have to play Under-21s football with Nottingham Forest for the rest of the season due to the error as he may not be included back in the Tricky Trees’ Premier League squad.
Blackburn boss Jon Dahl Tomasson was hoping to have O’Brien in his squad to face Wigan Athletic on Monday.
He will now have to make do without the influential midfielder and the transfer error could cost Blackburn dearly.
