It happened at Kunal Jewelry at 37-08 74th Street in Jackson Heights around 8 p.m.
"I was shocked to see and listen. Thank goodness, police came right away," said Tofael Chowdhury, a fellow merchant.
The three armed men entered the store just before it closed.
Police say they ordered two employees to the back store.
The suspects got away before police arrived.
Thomas Kashy is the wholesaler who supplies jewels to many of the stores here. He says the robbers never touched the display cases upfront, probably so they wouldn't trigger the alarm. Instead, he says, the safe in the back was wiped out.
"Whatever was in the display cases is there. No problem. What happened is that whatever was in the safe was sacked. The whole thing. Yes, the whole thing is cleared out," Kashy said.
There is an active investigation underway. Although, late Tuesday night police said that nothing was taken and the safe was empty.
Eyewitness News is told Kunal Jewelers is owned by one family who also owns Karan Jewels across the street.
Tuesday night, local merchants are calling for more police presence.
"We're like the Diamond District on 46th Street in Manhattan. We need more police presence in this area," said Zakaria Masud, President of the Business District.
No one was hurt.