"And then all of a sudden someone on the street yelled, 'Oh my God! A plane has just gone into the World Trade Center.' And I yelled down, 'Are you kidding? My wife works there,'" Charles Wolf said.
Wolf was in his apartment that day. He would later learn his wife, Katherine, perished in the North Tower.
For him and so many others, what came next was painful and embarrassing.
For years nothing got done at ground zero. Some even called it Pataki's Pit.
"And so I never had any doubts, this is New York. The day you don't get criticized is the day you don't get up and breathe. But ultimately the critical thing for me was doing it right," former Governor George Pataki said.
Doing it right meant sticking to a complicated master plan. That called for a series of skyscrapers and a huge, expensive memorial.
Yet everyone kept bickering and the players at the table kept changing. Only one refused to budge - developer Larry Silverstein.
"So I've gone through four governors in the state of New York, five governors in the state of New Jersey, five executive directors of the Port Authority, two mayors of the city of New York," he said.
Silverstein was in the middle of a complicated insurance fight and a battle with the Port Authority over who should build what.
Finally construction began on the memorial in 2006. And things began to change.
While the memorial will take up about half of the 16 acre site, commercial development finally moves forward. What used to be called the Freedom Tower will eventually be home to magazine company, Conde Nast. The Port Authority is putting up that building. On the east side, Silverstein will put up three big buildings and one of them is already about 45 stories tall."
One World Trade is already more than 75 stories tall.
Construction is underway on all of the Silverstein buildings, but will they ever be as tall and elegant as planned? Silverstein says yes. He is 80-years-old and determined to move forward.
In video promoting, Lower Manhattan to prospective tenants he said, "We had to rebuild this, absolutely had to rebuild this as quickly as possible and to the best of our ability. And come up with something that's bigger and better than anything we had on 9-11 that they sought to destroy so here we are. They didn't destroy us. It's coming back. It's bigger and better than ever."
Even though they're not technically part of the memorial, what's happening here means a lot to people like Charlie Wolf.
"When I see tower one, it's over 70 stories now, and I can see it from my neighborhood. It's so heart-warming to see that growing and growing and growing. I love it," Wolf said.
A message of hope and strength, ten years later.