The Red Sox, who face Tampa Bay six times over the next two weeks, were idle.
Kazmir (11-6) limited the Yankees to Cody Ransom's fifth inning double, walked five and struck out seven, but a high pitch-count (98) prevented him from going deeper into the game. Ransom also singled off reliever Trever Miller in the seventh for New York's second hit.
Chad Bradford worked a scoreless eighth but Tampa Bay lost its bid for its 13th shutout of the season in the ninth when Jason Hammel gave up RBI single to Ransom, who finished 3-for-3, and home runs to Jeter and Rodriguez.
Jeter's three-run shot to right field moved him within four hits of tying Babe Ruth for second on the Yankees career hit list.
Rodriguez followed with career homer No. 550 to trim New York's deficit to 7-5.
Dan Wheeler replaced Hammel and earned his 11th save in 14 opportunities, retiring Xavier Nady on a first-pitch fly ball.
Aybar capped a five-run second inning against Darrell Rasner (5-10) with a two-run single for Tampa Bay. He hit a solo homer in the fourth off Alfredo Aceves.
The Rays, who lost a series for the first time since the All-Star break, looked bad in dropping the first two games.
Manager Joe Maddon attributed the flat performances to the way the Yankees have been playing lately and uncharacteristic mistakes by the Rays - not the pressure of being in a pennant race in September for the first time.
Tampa Bay improved the best record in the majors to 85-52, but is 6-9 against New York, one of two AL teams with winning marks against the Rays this year.
Kazmir walked four in the first three innings, including A-Rod twice. He didn't give up a hit until one out in the fifth when Ransom doubled on a fly ball to shallow right that fell in when Gabe Gross tried to make a shoestring catch.
Rasner, 0-3 in nine starts since last winning on July 12, allowed five runs and six hits in 1 1-3 innings. Aybar drove in the last two runs charged to the right-hander with a bases-loaded single off Aceves.
Gross, Jason Bartlett and Akinori Iwamura also had RBIs during the five-run second against the Yankees starter.
Notes: Since taking over for injured rookie All-Star third baseman Evan Longoria, Aybar has five homers and 16 RBI's in 23 games. ... The Yankees plan to recall OF Melky Cabrera from Scranton/Wilkes-Barre on Friday. ... The Rays improved to 52-21 at home. ... Yankees RF Bobby Abreu was scratched because of a jammed left wrist, suffered Wednesday night when he slid into second base on a ninth-inning stolen base. X-rays were negative and he is day-to-day. ... New York also rested Hideki Matsui, who had been the designated hitter in 15 straight games after being activated from the 15-day disabled list on Aug. 19. He pinch-hit for Brett Gardner in the ninth and struck out.
