It happened just before 2:30 a.m. on Beach 91st Street in the Rockaways.
Firefighters say there was a woman trapped on the third floor, sitting out of a window and calling for help.
They say she was ready to throw her children out of the window, but firefighters rushed inside to rescue them.
"We reached the victims, we each grabbed one of the babies," said FDNY Lt. Joe Costa. "We took the two older ones that were able to walk and we made it down the interior stairs, past the fire and out the front. Then we went back in and put out the fire."
Several people were taken to the hospital with minor injuries, including a police officer who responded to the scene.
The cause of the fire is under investigation