The man was found with a stab wound to the chest at the Lexington Grocery on Lexington Avenue just after 5 a.m.
He was rushed to Harlem Hospital, where he was pronounced dead on arrival.
No arrests were immediately made, but police say they have recovered the murder weapon.
Police have not revealed any of the circumstances that led to incident, only that there was an altercation.
The investigation is ongoing.
Get Eyewitness News Delivered