Alex Ovechkin had a goal and two assists for the Capitals, who jumped over Atlanta and Carolina into first place in the Southeast Division. Ovechkin leads the NHL with 47 goals and 76 points.
Kolzig stopped 31 shots to earn his 296th victory, tying Ron Hextall for 23rd place on the NHL career list. Kolzig is only the eighth player, and the first goaltender, to play in 700 games with the Capitals. The 37-year-old began his career during the 1989-90 season.
"We needed Olie. He played superior today and kept us in the game," Washington coach Bruce Boudreau said.
With Kolzig leading the way, the Capitals successfully killed six penalties. Then, in overtime, Washington's maligned power play produced the game-winner.
Green scored on a shot high above the circles after taking a centering pass from Ovechkin. The shot came while New York's Ryan Hollweg was off for tripping, a penalty that occurred with 34 seconds left in regulation.
"I was thinking shoot," said Green, who has three overtime goals this season. "Their guy was screening the goalie, so as long as I got it by him, there was a good chance of it going in."
The Capitals had gone 0-for-18 with the extra man in their previous six games, including 0-for-7 in a loss to Carolina on Friday night.
Coming off two consecutive shutouts, against Philadelphia on Jan. 31 and Saturday, Rangers goaltender Steve Valiquette started on a second straight day for the first time in his career. He stopped 21 shots, but was helpless against Green's decisive blast.
New York's four-game road winning streak is over, but the Rangers remain in seventh place in the Eastern Conference, one point ahead of Boston and three behind Philadelphia. New York doesn't play again until Saturday, against Buffalo.
"Certainly, we'll pay attention to what other teams are doing, video and scoreboard watching," Rangers coach Tom Renney said. "We will practice with a purpose (but) give the guys a breather tomorrow."
Chris Drury gave New York a 2-1 lead at 9:25 of the second period with a sensational shot, a turnaround wrister from the left circle that somehow found its way through the well-positioned Kolzig. It was the team-high 18th goal for Drury and his sixth in seven games.
The Capitals pulled even with 15 seconds left in the period when Boyd Gordon got hold of a rebound and flipped a backhander past a diving Valiquette.
"That was a tough one. We weren't as tidy as we needed to be in our end," Renney said.
New York had an opportunity late in the third period to break the deadlock when Kolzig accidentally sent a clearing pass over the glass for a delay of game penalty. But the Rangers failed to score.
"Yeah, we had a lot of chances in the third to bury it," New York defenseman Marc Staal lamented.
New York had 13 shots on goal in the period, none of which found their way past Kolzig.
"He's been great for us," Gordon said. "The defense did a good job on the power plays, and he made a ton of great saves. He's always there for us."
Any hope Valiquette had of recording a third straight shutout ended at 2:28 of the first period when Ovechkin scored his NHL-high 17th power-play goal, tapping in the rebound of a shot by Eric Fehr. The goal came after Valiquette was assessed a penalty for delay of game.
Sean Avery tied it at 7:02 of the period with a rising wrist shot from the right circle.
