It happened just before 5 a.m. Sunday in the Pelham Gardens section.
The Pelham Bay Diner parking lot that the shooting happened in was put on lockdown for hours following the shooting.
The off-duty officer was reportedly involved in a fight in the parking lot.
Investigators say the officer was there with five other men who became involved in an argument with six other men outside.
The off-duty officer apparently was trying to get his friends into the car when he was shot in the buttocks.
At least two shots were fired.
The off-duty officer was rushed to Jacobi Medical Center where he is expected to survive.
Police say they are searching for the gunman and two accomplices who remain on the run.
Customers and workers inside the diner were held for hours as investigators took their statements.