"We've taken some poundings here. So it's nice to be on this side of one for a change," Aubrey Huff said.
Millar hit a three-run drive in the second and Ramon Hernandez followed with another homer off an ineffective Mike Mussina (13-7), who failed to last at least six innings for the first time since May 24.
It was the fourth time Baltimore has hit back-to-back homers this season, including three times against the Yankees. The last-place Orioles improved to 6-4 against New York, winning for only the sixth time in 20 games overall.
"For some of us, these guys are a tough matchup. They have some tough righties, which are a problem for me," Mussina said. "They always seem to be scoring runs, especially when they play us."
Huff hit his 21st homer and had three RBIs as Baltimore scored its most runs since a 15-0 win over Toronto on Aug. 19, 2006. The Orioles also matched their top hit total this season with 17.
"I think they all had good days," Mussina said. "It's a day that you want to forget."
Xavier Nady homered for his first hit with the Yankees, who had their eight-game winning streak snapped with a 9-2 loss Sunday night in Boston. Johnny Damon added a three-run shot to make it 11-4 in the seventh.
Before the game, New York announced Jorge Posada will have season-ending surgery on his right shoulder, ending his attempt to return to the lineup for the team's playoff push.
The Yankees remained three games behind AL East-leading Tampa Bay and two back of Boston.
"We've got to stop this two-game losing streak and get back on the winning track," manager Joe Girardi said.
Guthrie (7-8) shut out New York on two hits through six innings, giving Baltimore's struggling rotation its second consecutive solid outing. Garrett Olson tossed six-plus innings to beat the Los Angeles Angels 5-2 Sunday, breaking a run of eight games in which an Orioles starter failed to get through the sixth.
Guthrie, 4-1 with a 3.10 ERA in his past eight starts, recently added a minor hitch in his windup.
"I'm just hoping it adds a little wrinkle. I'm making it up as I go along," he said. "I felt good out there with all my pitches."
Still, only once in the past 22 games has a Baltimore starter gone at least seven innings. Guthrie was lifted after Nady's one-out homer in the seventh made it 11-1.
"You could tell right from the first hitter that he had finish on his pitches. He had good location," Orioles manager Dave Trembley said. "He had a game plan coming in and really attacked the guys. That's the best I've seen him pitch in a long time."
Mussina, who was 3-0 in his previous four outings, dropped to 10-8 against his former team. After not issuing a free pass in his past three starts, he walked the first batter of the game.
Melvin Mora and Luke Scott singled ahead of Millar's 15th homer in the second. Hernandez followed with a drive to center for his 1,000th hit, making it 4-0.
"It wasn't awful stuff, but it wasn't good stuff. It was a struggle from the beginning," Mussina said.
Five of Millar's homers this season have come against the Yankees, who have given up at least 10 runs to Baltimore three times. Millar is 18-for-41 (.439) with three home runs against Mussina.
Brian Roberts doubled to start the fifth before Jones tripled and scored on Huff's sacrifice fly. Mussina was pulled after allowing six runs and eight hits in five innings.
Jones hit his first career slam to left in the sixth. It was the first home run allowed by David Robertson in his professional career, which began last year.
Jones also singled in the eighth and scored three runs. He had a chance for an even bigger game, but he flied out with the bases loaded to end the ninth.
"First couple of at-bats were terrible. Then I told myself, just do what you've been doing," Jones said. "This was one of the few days this year when the balls hit hard didn't find somebody."
Millar finished a triple shy of the cycle. Jamie Walker, just off the disabled list, worked 1 1-3 scoreless innings to finish it.
Notes: Jones, who had only two RBIs in his previous 12 games, set a career high with five. He knocked in four runs May 20 at Yankee Stadium. ... Damon had gone 156 at-bats without a home run. ... Every Orioles starter had a hit. Baltimore finished with nine extra-base hits.
