Police say the fire broke out in one home on Lincoln Street and spread to another at around 2:20 a.m.
Everyone inside made it out safely, but two people were treated for minor injuries.
A resident named Juanita Cologne and her husband and a 3-year-old were able to run out unharmed - but she is livid about the people she says caused the fire.
Boxes of used fireworks were found in front of her neighbor's house on the corner, which had no damage.
"They don't have compassion or respect for anybody," Cologne said.
Juanita says she knows that fireworks are illegal and everyone else does too and were calling the police who came out more than once - but the fireworks continued.
"Right in front of them they were throwing the firecrackers and they didn't do anything," Cologne said.
FDNY confirmed the fire was sparked from fireworks and reminded New Yorkers that fireworks are illegal in New York City.
There was disaster in Edison, New Jersey as well where officials had 40 calls during Diwali about fires, which is why the mayor and the fire chief made a video warning people one family has already lost their home.
"Bottle rockets, fire crackers, and Roman candles and aerial fireworks are all illegal and prohibited," they said.
They invited everyone to a safe Diwali celebration Saturday from 2 to 9 where there will be a fireworks show.
"Find another way of celebrating, because that's dangerous and it's not legal," Cologne said.
----------
* Get Eyewitness News Delivered
* More Manhattan news
* Send us a news tip
* Download the abc7NY app for breaking news alerts
* Follow us on YouTube