The FBI raided two locations on Tuesday tied to Ed Mullins: the headquarters of the Sergeants Benevolent Association (SBA) in Lower Manhattan and Mullins's home in Port Washington.
Agents were seen leaving SBA headquarters on Worth Street Tuesday carrying boxes of potential evidence.
Sources say Mullins is being investigated in connection with expense report irregularities.
In a letter to its members obtained by ABC News, the union representing 13,000 current and retired police sergeants stressed that Mullins is the only target of the investigation.
The SBA executive board says it asked Mullins to resign given "severity of this matter and uncertainty of its outcome," and he agreed.
There was no immediate comment from Mullins.
For nearly two decades, Mullins has been an outspoken and controversial member of the NYPD.
He has also been harshly critical of the de Blasio administration.
Aside from the federal investigation, he is under NYPD departmental review for allegedly tweeting information about the mayor's daughter's arrest during last year's Black Live Matter protests.
In response to Tuesday's resignation, de Blasio tweeted this:
"Ed Mullins dishonored his uniform, his city and his union more times than I can count. It was just a matter of time before his endless hatred would catch up with him. That day has come."
Ed Mullins dishonored his uniform, his city and his union more times than I can count. It was just a matter of time before his endless hatred would catch up with him. That day has come. https://t.co/J03dKk0a1g— Mayor Bill de Blasio (@NYCMayor) October 6, 2021
It is not clear who will replace him.
Along with Mullins' periodic appearances on cable networks like Fox News and Newsmax - including one in which he was pictured in front of a QAnon mug - perhaps the union's most powerful megaphone is its 45,000-follower Twitter account, which Mullins runs himself, often to fiery effect.
In 2018, amid a rash of incidents in which police officers were doused with water, Mullins suggested it was time for then-Commissioner James O'Neill and Chief of Department Terence Monahan to "consider another profession" and tweeted that "O'KNEEL must go!"
The Associated press contributed to this report.
ALSO READ | Suspect arrested in case of woman pushed into subway train in Times Square
* Get Eyewitness News Delivered
* Follow us on YouTube
* More local news
* Send us a news tip
* Download the abc7NY app for breaking news alerts Submit a News Tip