The 41-story building had a gash in its side, 15 stories long, after it was hit by one of the falling towers on 9/11.
Crews are clearing away the few remain pieces of the building before turning the site over to the Port Authority.
The building was also the site of a blaze that killed two firefighters. The bank tower was first slated for deconstruction in 2005, when a government agency bought it to end an impasse over who would pay to take it down.
The cleanup of toxins including asbestos, lead, mercury, PCBs and dioxins was delayed multiple times by fights over how to remove the material without polluting the neighborhood. More than 700 body parts of Sept. 11 victims were recovered, mostly on the roof, along with parts of the hijacked plane. Environmental and city regulators spent years coming up with a cleanup plan that would keep the toxins in with polyurethane coverings and other protective panels.
Accidents plagued the deconstruction. In May 2007, a 22-foot pipe fell from the building and crashed into the firehouse next door, injuring two firefighters.
Three months later, a construction worker's discarded cigarette sparked a fire that tore through several stories. Firefighters faced hazards including deactivated sprinklers, stairwells that had been blocked to contain toxic debris and a broken standpipe, a crucial water conduit like a fire hydrant.
Firefighters Robert Beddia and Joseph Graffagnino were trapped on the burning 14th floor and died of smoke inhalation on Aug. 18, 2007. Prosecutors investigated every agency involved and heavily chastised the city for failing to regularly inspect the tower and make sure its dismantling was safe.
Three construction industry figures were charged in the fire. Prosecutors said Mitchel Alvo, Jeffrey Melofchik and Salvatore DePaola knew the standpipe had been cut and did nothing about it.
The three pleaded not guilty to manslaughter and other charges; their lawyers have argued that the men have been singled out unfairly when government agencies and others are to blame for the fire.
Joseph Graffagnino's widow, Linda, said she has mixed feeling about the criminal case.
"The people who are going on trial are scapegoats for higher-ups who were more responsible," Graffagnino said. "Do I really care about what happens to those people? Not really."
The Graffagninos have sued the city, the LMDC, the main contractor Bovis Lend Lease and subcontractor John Galt Corp. The parties have also sued each other over the mounting costs of the cleanup.
The fire delayed the cleanup and dismantling for a year. Removal of toxic debris started in 2008, and deconstruction resumed in late 2009.
The Port Authority owns the 16-acre trade center site and plans to place an underground truck-screening facility at the site. The spot has long been slated for the fifth of five towers planned to be rebuilt at the trade center site, although there's no timeline for it.
Assembly Speaker Sheldon Silver, whose district includes ground zero, said the toxic tower's removal is enough of a milestone for now.
"It's one more symbol that lower Manhattan will come back from Sept. 11 bigger, better and stronger than ever before," Silver said.