There were reports the suspect was holding a victim in the basement apartment of a home in Secaucus.
The SWAT team was called in and officers spent hours outside the home.
Neighbors saw the activity and feared for their own safety.
"The next thing you know, I'm getting a bang on the door, and you know, we're all very scared," said Matthew Montgomery, a neighbor. "It was just the cop telling us that we had to evacuate, and that's after the big armored truck pulled up."
Police eventually learned the suspect they were looking for was not in the home, but out in a car.
The vehicle was stopped in Bayonne and the suspect was taken into custody.
The victim is being treated for her injuries.
It's not clear what charges the suspect is facing.
ALSO READ | 1 killed, 2 injured in shooting at Long Island party
* More New Jersey news
* Send us a news tip
* Download the abc7NY app for breaking news alerts
* Follow us on YouTube