Giuliani, who led New Yorkers through Sept. 11 and its aftermath and whose opinion on the mosque could carry considerable clout, made his comments as the imam leading plans for the community center toured the Middle East promoting religious tolerance.
"If you are a healer, you do not go forward with this project," Giuliani said on NBC's "Today" show, referring to the center's leader, Imam Feisal Abdul Rauf. "If you are a warrior, you do."
Developers want to build the $100 million community center, including a mosque, at a building two blocks north of where Islamic extremists brought down the World Trade Center in 2001. Muslims have been holding prayer services at the building since last year.
Support is growing for a possible land swap to provide an alternate site for what's called the Park51 project, Gov. David Paterson said.
"One of the problems the cultural center is going to have is just a constant point of antagonism, which I don't think is what they want," Paterson told WOR Radio on Thursday.
Paterson said he had the support of Islamic clergy, New York Archbishop Timothy Dolan and Giuliani. The governor and state officials refused to say what site would be suitable for the proposed cultural center or where the state owns nearby land.
Paterson said he expects to meet with the developers in a couple of days to persuade them that a move could best assuage the "national hysteria" that has followed the project.
Sharif el-Gamal, Park51's developer, and The Cordoba Initiative, an organization that hopes to operate the community center, didn't return telephone and e-mail messages Thursday.
Buthayna Abdul Rauf, the imam's mother, said Thursday she was mystified by the controversy surrounding the project, which she only recently heard about from news reports and a friend.
"This is the first time I've heard people in America being against a mosque," she told The Associated Press in a telephone interview. "The Americans are very generous. ... Mosques have been erected everywhere in America."
She called her son, whom she hasn't seen for three months, a "very peaceful man."
Feisal Abdul Rauf, who heads Cordoba, arrived in Bahrain on Thursday for a U.S.-funded outreach trip for two weeks in the Middle East.
Rauf was expected to discuss Muslim life in America and promote religious tolerance. He will be visiting mosques, Department of State spokesman P.J. Crowley said Thursday.
"He will be involved in direct discussions to help people in the countries he'll visit understand the role of religion in our society, how American Muslims celebrate Ramadan, how we emphasize religious tolerance in our society," Crowley said.
Rauf won't be allowed to raise funds for the mosque on the trip, Crowley said.
The project has caused a political uproar, pitting national Republicans against President Barack Obama and dividing Sept. 11 families and New Yorkers.
Foes argue that the proposed mosque is offensive because it's too close to the place where the terrorists killed more than 2,700 people. Supporters led by Mayor Michael Bloomberg say the center's constitutional rights to religious freedom should be protected.
Bloomberg reiterated his support Thursday.
"I haven't changed my views. This is about the First Amendment," he said. "It's about people being able to pray to whomever they want, whenever they want, wherever they want. That's one of the fundamental tenets of our society. It's one of the things that differentiates us from other countries. ... In terms of this particular mosque, I've said I think it would add to the diversity of the city and be fine."
Both sides were on display Thursday at the site, where on the sidewalk passers-by had scribbled messages in multicolored chalk.
"Mosque Yes Hate No" read one.
Heated words were exchanged between visitor Matt Harris, of Yorba Linda, Calif., standing face to face with Matt Sky, a New Yorker who hoisted a placard that read: "Support Freedom of Religion."
"Dude! You have other mosques in New York - why here? This is lack of respect!" Harris yelled.
"We believe there is freedom of religion in this country," replied Sky, a 26-year-old resident of Manhattan's East Village neighborhood.
He added: "Islam is not terror. The guys who blew up the towers called themselves Muslims. But other Muslims did not blow up the towers."
Giuliani noted that the right to religious freedom has nothing to do with the sensitivity of locating a large community center so close to the attack site.
"They have every right to build it. The question is, should they build it?" Giuliani said, noting the group claims to be about sensitivity and healing between cultures. "All this is doing is creating more division, more anger, more hatred."
"I think Governor Paterson had the best approach," the Republican said of the Democratic governor. "Nice compromise, find another place, have a beautiful mosque there."
An expert noted that government resources have often been used to help religious organizations and their buildings.
"But the government can't simply buy property and turn it over to a religious entity where the benefits are exclusively for the members of that church," said Robert B. Ward, of the Nelson A. Rockefeller Institute of Government.
Said the governor: "Not-for-profits that are run by churches receive state resources all the time."