The rally was organized by the AFL-CIO and an association of community groups. It included a diverse mixture of union workers, activists, the unemployed, and homeowners threatened by foreclosure.
"These guys are like pirates," said small business owner Karen Casamassima, of New York, who called for "economic patriotism" and held up a jewel encrusted skull with the words, "Financial Terrorists."
The protesters, carrying signs saying "Wall Street Overdrafted Our Economy" and "Reclaim America," rallied at City Hall Park, then marched down to the Merrill Lynch Bull statue demanding good jobs and accountability from banks.
"I think Wall Street is responsible for the collapse of our economy," said Bennett Silverstein, an attorney from Brooklyn who said his savings were depleted by the recession.
Earlier in the day, noisy protesters with signs took over two bank building lobbies on Manhattan's Park Avenue in a prelude to the rally.
More than 100 people entered a midtown building housing JPMorgan Chase offices. They handed a bank executive a letter requesting a meeting with the CEO, and chanted "Bust up! Big banks!" and "People power!"
A half-hour later, they were calmly escorted outside by officers, who remained expressionless as the protesters chanted, "The police need a raise."
They then walked a few blocks up the avenue and crowded into the lobby of the Seagram Building, where Wells Fargo and Wachovia, the bank it merged with in 2008, have offices.
The protesters held up signs reading, "Save Our Jobs" and "Save Our Homes." One included a Great Depression-era photograph.
Police arrived on horseback as curious office workers watched the scene unfold from their windows.
"We're here today to stop the corporate greed that is ruining our neighborhoods," said Andrea Goldman, 59, of Springfield, Mass., who's part of a group called Alliance to Develop Power.
Fran Durst, a Wachovia spokeswoman, said the protesters wanted to deliver a letter to the bank, and they did so. She declined further comment on the letter.
The other banks did not immediately respond to requests for comment.
The Securities Industry and Financial Markets Association, which includes many Wall Street financial institutions, declined to comment.