The blaze caused extensive water and fire damage at the Al Tawheed Islamic Center.
"All the rooms inside, everything and the roof, too, its big damage," said Mohamed Osman, a mosque member.
It was around 5:30 a.m. Friday when flames and smoke were shooting through the roof of the mosque on Westside Avenue as several people were showing up for morning prayer.
Bilal Arshad never made it inside.
"All that smoke came in my face; I went away from the door. It was just, way too much to handle," Arshad said.
By late morning, members of the mosque's executive board made other arrangements for their midday prayer service.
It was held at the armory in Jersey City.
Where these congregants will worship in the coming days is unclear.
"Since this happened, there won't be a lot of prayers at the mosque, so we're probably going to have to go to other mosques until we could get the place fixed," said Hesham Amer, Al Tawheed Executive Board member.
Investigators are now trying to figure out just how the fire began.
"My investigators have not given me any indication that they're comfortable where it started yet, but we will, wherever it started, we will find out," said Jim Shea, Jersey City Public Safety Director.
No one was injured in the blaze, which reached three-alarms. The structure was destroyed.
Firefighters were working in extreme cold conditions which caused water to freeze.