There is no word yet on what spurred the man to take four hostages, including a small child, inside the apartment.
East Orange police spokesman Andrew DiElmo says 31-year-old Danny Baxter of East Orange surrendered at about 2 p.m., when police broke down the apartment door.
Baxter offered no resistance, and a 9 mm handgun was recovered there.
Inside the apartment, officers found a woman believed to be Baxter's wife and a small child. But they haven't determined if the child was related to the couple.
Baxter had released two adult hostages shortly before noon, but their identities haven't been released.
