Firefighters responded at about 4 a.m. to the blaze in the basement and on the first floor of a private home on Taylor Avenue.
"I gotta leave the nervous away and do what I got to do," Eddie Herasme said.
So when Eddie Herasme saw two people holding two babies screaming on a balcony he knew he had to get them down.
"I told them to throw me the baby first, throw the babies," he said.
"They weren't crying. I just wrapped them up in the coats until the mom took them away from me," Francine Valmont said.
Next, Herasme says out came a man with only one arm and one leg dangling off the balcony.
"He was hanging. When he came down, we both went to the floor and he rebounded on me," he said.
Herasme tried to run in to get his teenage daughters, but the smoke was too thick.
They are among the 12 now recovering at Jacobi Medical Center.
The FDNY says the fire was caused by candles left burning.
That was the cause of a fire last Friday also in the Bronx which killed three young children. Both homes did not have working smoke detectors.
On Monday, the FDNY took to the streets of the Bronx to remind people about fire safety.