Sources also confirmed Madoff was treated at Duke University Medical Center in Durham last Friday and discharged earlier this week.
He is serving a 150-year sentence at the federal prison in Butner after admitting to cheating investors out of billions of dollars.
According to the sources, Madoff came to Duke with facial fractures, broken ribs and a collapsed lung.
Prison officials won't confirm that. They say the 71-year-old is now in the medical clinic at the prison in Butner.
According a report by the Associated Press, Madoff's lawyer - Ira Sorkin - said his client was not assaulted, but "had some dizzy spells and some high blood pressure."
The AP reported Duke University Medical Center earlier said Madoff had been treated there, but later a spokeswoman said the hospital had never had a patient by that name.