"It's an absurd abuse of tenure," Bloomberg said.
Under the agreement with the United Federation of Teachers, most of the teachers will be given administrative or nonclassroom work while their cases are pending.
"We're going to put teachers to work instead of having them sit in rubber rooms while their cases are being resolved," Bloomberg said.
Teachers accused of serious charges including violent felonies will be suspended without pay.
About 650 educators, more than 500 of them teachers, are in the rubber rooms earning some $30 million in salaries, officials said.
The nickname refers to the padded cells of asylums, and teachers have said the name is fitting, since some of the inhabitants can become unstable.
"There are fights among teachers because some teachers are nuts," said Leonard Brown, a high school teacher who spent four years in a reassignment center in Queens. "They put crazy people in with very sane people."
The city has blamed union rules that make it difficult to fire teachers, but some teachers assigned to rubber rooms say they have been singled out because they ran afoul of a principal or they blew the whistle on someone who was fudging test scores.
Past attempts to expedite the disciplinary process have borne few results, but Bloomberg said Thursday's agreement will carry more weight because it includes deadlines for resolving cases.
Under the pact, the Department of Education will have 10 days to file incompetence charges or 60 days to file misconduct charges, depending on the nature of the case. If no charges are filed by the deadline, the teacher will be sent back to the classroom.
"Together, we and the UFT are saying enough is enough," said Schools Chancellor Joel Klein. "No more rubber rooms. They aren't good for anyone. Not for teachers, not for students, not for schools."
UFT President Michael Mulgrew said ending the rubber rooms has been a priority since he took over as head of the union last year.
"We want a faster, fairer process," he said.
Teachers who are or have been assigned to rubber rooms said they welcomed the agreement.
"We want to coach those that are not prepared for this profession to move on. However, we also want justice for those who have been accused of wrongdoing," said Orlando Ramos, who spent seven months in a rubber room in 2004-2005. "The rubber room has been the wrong answer for so long."
Ramos, who is now a middle school principal in San Jose, Calif., was an assistant principal in East Harlem when he was accused of lying at a hearing on whether to suspend a student. Ramos denied the allegation but quit before his case was resolved and moved to California.
David Suker, who is assigned to a rubber room in Brooklyn, said teachers there were waiting to hear details about how the system would be dismantled. "It's just another typical day in terms of powerlessness," Suker said.
But Philip Nobile, who has been in a Brooklyn rubber room for nearly three years, said Thursday's agreement is a step in the right direction.
"We won't have people in rubber rooms for years uncharged," he said.
Instead of going to rubber rooms, most teachers awaiting disciplinary hearings will now perform administrative work in department offices or nonclassroom work in their schools.
Klein said teachers accused of serious wrongdoing will be suspended without pay after appearing at a probable cause hearing.
Officials agreed to increase the number of arbitrators who hear teachers' cases from 23 to 39 and said they hope to catch up with backlogged cases by the end of the year.