Now, 20 years later, he's talking about about what he remembers and how far we've come as a city and as a nation.
[Ads /]
"My daughter, who was working at Bloomberg News, called me and said, 'Turn on the TV, a plane hit one of the towers,'" he said.
RELATED | 'Eyewitness to 9/11: Behind the Lens' reveals untold stories, rare video of America's darkest day
'Eyewitness to 9/11: Behind the Lens' | Full Special
That day, Pataki shut down all bridges and tunnels, called up the National Guard, and like all of us, mourned.
Twenty years later, Pataki is still mourning.
"I just look, and the towers aren't there, 3,000 people aren't there," he said. "And I have this unbelievable sense of sorrow and loss. But what a lot of people don't realize is we had an economic catastrophe, nobody coming to New York. Tourists wouldn't come here. Visitors from America wouldn't come here. I was getting calls all the time from companies, 'We're leaving.'"
Pataki and Mayor Michael Bloomberg spearheaded a movement that transformed Lower Manhattan, with 20 million square feet of office space built back, and a memorial to those killed that day.
"I look around and then I feel pride, too," Pataki said. "I'm really proud of what we did down there."
[Ads /]
Today, Pataki is furious about what's happening in Afghanistan, where the Taliban has again regained control.
"Afghanistan as a horror for the Afghans, for the Americans who sacrificed so much over there," he said. "But it's also a frightening thing for the future for places like New York and Washington, and America."
RELATED | How reporter N.J. Burkett and his photographer escaped Twin Towers collapse
How reporter N.J. Burkett and his photographer escaped Twin Towers collapse
Despite his anger about Afghanistan, he says on this 20-year anniversary, he's focused on remembering, and on the long road to recovery.
"Yes, we're going to rebuild Lower Manhattan," he said. "Yes, we're going to bring back the economy. But we're never going to forget those we lost, and we're always going to honor the families who lost so much on that day."
Pataki also reflected on when he knew New York was going to be OK. It came when President Bush threw out that first pitch at Game 3 of the 2001 World Series.
[Ads /]
"I think people said, 'Hey, if 60,000 people and the president can be in Yankee Stadium, maybe it's OK to come back to New York.'"
CLICK HERE for more Eyewitness News reflections, photos and stories marking the anniversary of 9/11
----------
* Get Eyewitness News Delivered
* Follow us on YouTube
* More local news
* Send us a news tip
* Download the abc7NY app for breaking news alerts Submit a News Tip