Mathieu Schneider staked the Ducks to a lead in the first, before Ryan Callahan tied it for the Rangers, who dropped their second straight home game to a team from Southern California.
Los Angeles topped New York 4-2 on Tuesday, and the Rangers are winless against the Western Conference this season (0-7-2). They got 33 shots at Jean-Sebastien Giguere, who came in off a pair of shutouts, but couldn't make them count.
Todd Bertuzzi made it 4-1 with 7:47 left, 15 seconds after a Rangers power play created lots of pressure but didn't click. Anaheim is 2-3-1 on its club-record, nine-game trip that continues Friday in New Jersey.
The Ducks, who hadn't played at Madison Square Garden since Oct. 28, 2003, improved to 12-3-4 against the Rangers - 6-2-2 in New York.
The pace picked up quickly in the middle period with a pair of fights 16 seconds apart. That revved up the fans, but didn't immediately lift the offenses.
Callahan changed that 5 minutes later with his first goal in 26 games, dating to opening night.
The second-year forward, who spent half this season on the injured list or in the minors, fired a shot from the right circle that smacked loudly off the left post and in at 7:42 - all while teammate Blair Betts sat in the crease.
Anaheim didn't muster many shots, but made them count. Perry put the Ducks back in front at 12:05, and Kunitz stretched the lead to 3-1 with a power-play goal at 13:32.
Henrik Lundqvist surely would want back Perry's 28th, a shot from the right circle that squeezed under the goalie's left arm and trickled over the line. Kunitz took advantage of Fedor Tyutin's hooking penalty - Anaheim's first power play - 13 seconds after the defenseman went to the box.
Kunitz got to a rebound of Chris Pronger's shot and bounced the puck off the right post and into Lundqvist, who knocked it in.
Anaheim had at least three goals for the second straight game after scoring only five in six contests. The Ducks scored three times on 14 shots through two periods.
Anaheim, which visited the White House on Wednesday in honor of last season's championship, was outshot 12-7 in the first, 11-7 in the second and 10-5 in the third.
Despite most of the early action taking place in the Anaheim end, the Ducks took a 1-0 lead on their fourth shot. Schneider wound up from the right point and sent the puck through traffic and past Lundqvist for his seventh goal, scored at 10:36.
Schneider, a former Rangers defenseman, played in his second game after missing two because of a sprained left wrist.
