He cited research showing asymptomatic people could be spreading the virus without realizing it.
"When you put on that face covering, you're protecting everyone else," he said.
The mayor said it could be a scarf or a bandanna or anything homemade, but it should not be a surgical mask needed by first responders and healthcare workers.
A recent study by researchers in Singapore became the latest to estimate that somewhere around 10% of new infections may be sparked by people who carry the virus but have not yet suffered symptoms.
New York state officials said Friday there is no data to support the effectiveness of face masks.
"There is no clear evidence to suggest that face masks, whether made out of cloth or even the general public using face masks. but we are continuing ot look at the data all the time that is coming in and we are examining that right now," Health Commissioner Dr. Howard Zucker said.
A bandanna might not prevent someone from coming into contact with the virus, but it could help a person who has it not give it to others when the sneeze, cough or breathe.
"It's fair to say the masks couldn't hurt unless they gave you a false sense of security," New York Governor Andrew Cuomo said.
On Friday, the Centers for Disease Control and Prevention, in a reversal, is also advising Americans wear non-medical masks in public, adding it was voluntary.
The mayor of Los Angeles also told everyone in the city to start wearing masks on Wednesday.
MORE CORONAVIRUS COVID-19 COVERAGE
Where are the testing centers
See how our communities are making a difference
Free educational resources for parents and children
How you can help victims of coronavirus
Coronavirus news and live updates in New York
Coronavirus news and live updates in New Jersey
Coronavirus news and live updates in Connecticut
New York City updates
Long Island updates
How coronavirus is leaving ghost towns in its path
Coronavirus prevention: how clean are your hands?
Social distancing: What is it and how does it stop the spread of coronavirus?
Coronavirus closures and cancelations
Coronavirus tips: What Americans need to know
Centers for Disease Control and Prevention on coronavirus
Total count of COVID-19 cases based on patient address