The fire broke out in a room where electrical equipment was stored on the mezzanine level of the Graybar Building at about 9:30 a.m.
Officials said sprinklers extinguished the fire, but there was a heavy smoke condition on the floor.
There were six minor injuries reported, although only two people were taken to the hospital.
Our crew at the scene said at least some of the building was evacuated and hundreds of office workers were standing out on Lexington Avenue, but after about 90 minutes they were able to return to their offices.
One woman said she walked down from the 24th floor and said she saw smoke up on her floor. The smoke rose as high as the 27th floor.
Traffic was backed up on Lexington Avenue north of East 42nd Street but the streets around the terminal later reopened.
The MTA says there was no impact to Grand Central Terminal.
Here is a live report from Josh Einiger who was at the scene:
* More Manhattan news
* Send us a news tip
* Download the abc7NY app for breaking news alerts