"I came into this job with one mission: to fight crime and keep everybody safe," O'Neill said. "We did it and continue to do it."
Mayor Bill de Blasio held an afternoon news conference to officially make the announcement, thanking O'Neill for his service and naming Chief of Detectives Dermot Shea as his replacement.
Watch the full news conference:
The mayor met with senior police officials over the weekend, including extensive meetings with Shea and First Deputy Commissioner Ben Tucker. He called Shea, who will take office on December 1, a "proven change agent."
"This is a tremendous honor and a tremendous responsibility and I'm grateful to the mayor for this privilege to serve," Shea said. "Police Commissioner O'Neill has been a mentor and a friend to me, and I am committed to building on the incredible success of Neighborhood Policing and precision policing, while continuing my life's work to eradicate gangs and guns from our streets. Every New Yorker deserves to be safe and feel safe, and that has been my mission since I took the oath and became a police officer 28 years ago. As police commissioner, this will be what drives me."
Related: Who is Dermot Shea, New York City's next police commissioner
The 61-year-old O'Neill spent more than three decades with the NYPD before becoming commissioner in September of 2016. His tenure began with a pipe bomb exploding in Manhattan's Chelsea neighborhood on his first day.
He also saw the department through two other terror attacks: The Halloween truck attack on the West Side and the detonation of a pipe bomb beneath the Port Authority Bus Terminal.
O'Neill paid tribute to the families of officers who died in the line of duty during his tenure.
"This is not an easy job, the job of police officers," he said.
O'Neil also oversaw a continual drop in crime and a reduction in arrests, but part of his administration was haunted by difficult decisions after the death of Eric Garner, which cost him loss of faith among some in the rank and file.
There have also been a recent alarming amount of officer suicides.
He led efforts to bolster community policing and repair the department's relationship with minority communities that had complained about innocent black and Hispanic men being caught up in aggressive enforcement of minor crimes.
"This is a safer city and fairer city," de Blasio said in crediting O'Neill for transforming the police department to one focused on neighborhood policing that brought officers closer to the communities they serve.
O'Neill moved the department away from the controversial "broken windows" theory of law enforcement, which viewed low-level offenses as a gateway to bigger crimes.
"On behalf of all New Yorkers, I want to express deep gratitude to Jimmy O'Neill for dedicating his entire career to keeping our city safe," de Blasio said in a statement. "Jimmy transformed the relationship between New Yorkers and police, and helped to make the Department the most sophisticated and advanced in the country."
In August, he brought closure to one of the NYPD's lowest moments, firing Officer Daniel Pantaleo for Garner's chokehold death. The city's largest police union responded by calling for O'Neill's immediate resignation.
Asked in recent weeks about rumors of his retirement, he said he had the "best job in the world."
O'Neill said Monday that the decision to fire Pantaleo weighed heavily on him, but did not factor into his retirement.
"This is the right time for me," O'Neill said. "This job comes with a lot. It comes with a lot of pressure. This (job) is all I have thought about for the last 38 months - 24 hours a day, 7 days a week. It's all you think about, is keeping the people of this city safe, and it was an honor to serve."
De Blasio released a statement on Twitter, saying O'Neill "is the architect of neighborhood policing. He drove crime to record lows while working tirelessly to bring police and communities together. He leaves behind a city that's safer than it's been in decades. I'm lucky to have worked with as good a man as Jimmy O'Neill."
O'Neill is expected to take a job in the private sector.
De Blasio focused on Shea's experience and roots in introducing the new leader of a department. Shea, who has served the force in varying roles for 28 years, is the son of Irish immigrants, growing up in Queens with four siblings in a one-bedroom apartment.
"Treat people with dignity, respect, treat people the same," Shea said in speaking of values instilled in him by his mother. "We cannot and will not rest until all New Yorkers feel safe."
Shea's brother Jim is the director of public safety across the River in Jersey City. His brother Paul is deployed with the US Army.
(The Associated Press contributed to this report)
* More New York City news
* Send us a news tip
* Download the abc7NY app for breaking news alerts
* Follow us on YouTube