Dozens of people were also left homeless by the massive blaze, which started around 4 a.m. Sunday. Crews from 10 different fire departments responded, but by the time the first group arrived, 2 of the structures were already almost gone.
"I was sleeping and I heard a boom, boom, boom and when I came outside everyone was screaming and crying," Perth Amboy resident Luis Gonzalez said.
The fire was too much to bare for Jose Nunez, and his son, Randy. The top floor, where the 15-year-old was sleeping is completely gone. Randy's knees are bandaged and bruised. He fell trying to get away from the flames and had the scare of his life in all of the chaos. While some people were able to piece things back together, Jose wasn't as lucky.
"I lost everything, I don't have anything, this is all I have," he said.
The three injured people - a couple and their infant daughter - were hospitalized in serious but stable condition.
Roughly 75 firefighters battled the stubborn blaze, which damaged several other homes and burned for several hours before it was brought under control.
Witnesses said it appeared to be "raining ashes" as crews worked to extinguish the massive flames.
It was a five-alarm fire, and authorities were concerned it could jump across the street to other homes in the neighborhood.
A wide radius of at least 30 homes and 50 residents were evacuated.
"I haven't seen that amount of fire in a very very long time and I've been doing this for more than 30 years. I was very very concerned about this fire moving in multiple directions," said Perth Amboy Fire Chief David Volk.
The cause of the fire remains under investigation.
Most of the displaced residents were taken to a shelter set up at a school. Some later left to stay with family and friends, but Red Cross officials said many people would likely be spending the night at the shelter.
Get Eyewitness News Delivered