The Swiss great travelled to the Spanish capital for the tournament but was unable to practice on Monday and announced his withdrawal.
Federer, however, said he would be back for the Rome Masters next week.
"I don't want to take more chances as I know I'm not going to be fully ready for Wednesday," he added, referring to his opening match.
"I would rather play it safe and rest up now and get ready for Rome. I'm sorry to the tournament for coming and leaving without playing.
Game of throws: Big four to reunite in Madrid Masters
"I arrived and I was okay and then I practised on Saturday and hurt my back a little bit and stopped early. I'm very disappointed to say the least," said Federer.
"I changed my schedule around and practised well in Switzerland before coming here."
Federer said he was not worried about his preparations for the French Open that starts on May 22.
High targets: Federer’s haul at back of my mind, says Djokovic
"I've been doing a lot of practice on clay," the world number three explained.
"I don't always need a lot of matches to feel 100 per cent ready.
"With my experience and the way I feel about big tournaments, if I have matches, great. If I don't, I trust in my game, in my mind that I'll be fine regardless of the preparations."
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ